Questions tagged [awk]

A pattern-directed scanning and processing language.

Filter by
Sorted by
Tagged with
-2
votes
0answers
22 views

How to Print unique IP address and Date with header of Log files in Linux by using a simple script

can anyone please explain me how separate ip address and date of apache access logs in the following way unique IPs date 15.1.1.0 12/jan/2019 5.1.0.1 13/jan/2019 200.1.1.1 ...
0
votes
3answers
50 views

Using printf with AWK to format my output correctly

I'm trying to format some output using printf with awk. Basically, I want the final format to look like this (everything is tab-separated.) chr10 100000624 100000625 10:100000625_A_G ...
0
votes
3answers
45 views

Using awk to print part of one column plus additional columns

Good day. Using only Linux OS tools, how would you convert the following text from this: cat latest.csv apsga,alp1acscf0001vm001oam001p1n004v001,192.7.129.7,UVP apsga,...
0
votes
1answer
17 views

select data conversion based on value range

I have a text file with 3 columns of data. However at random times in the various files there is a change in the in the observed unit from ppn to ppb resulting in the need of a conversion factor and ...
2
votes
2answers
118 views

How to compare two files to get matched records?

I have 2 files with * delimiter, each file with 3k records. There are common fields in different positions. In file1(count=1590) the position is 1 and in file2(2707) the position is 2. file2 count and ...
0
votes
4answers
59 views

How to replace space with another value?

I am having file with | delimiter 8k records. If column#3 is empty replace with column#2 values. How do we achieve this. input: 1|100437251| 2|51414204| 3|111651604|111651604 4|8321737|8321737 5|...
0
votes
2answers
40 views

Using awk '{printf…' multiple columns

How do I print multiple columns using awk '{printf...? With awk '{print... it is easy: $ printf bar\\t5 | awk '{print $2,"baz",$1}' 5 baz bar But if I want to format the digit to show a ...
-3
votes
1answer
41 views

Adding a values columns in a text file

I have a text file with three columns as below 568234.099421 4497975.785405 0.000000 567992.699441 4498682.885472 0.100000 568738.099488 4499368.184928 0.600000 568029.798936 4499762.584742 0.600000 ...
0
votes
2answers
31 views

How to trim off specific parts of multiple directory names?

I have 100 directories within a parent directory (/home/anik/"all my directories") with the following naming structure: Zt.isolate1.Spades Zt.isolate2.Spades Zt.isolate100.Spades I want to ...
1
vote
1answer
51 views

How to get string between starting with X to ending with Y using Shell

I have dynamic string which are mentioned below, where I want to get ABCXXX.XXX.XXX string till , character. Starting character is fix (ABC)CAPITAL ONLY and ending also fix com or Xcom. String can be ...
1
vote
2answers
33 views

How to delete duplicate line within a bracket preferably using one-liner?

My intention is to remove duplicate Param within each test. There is multiple tests that might have duplicate Param and I want to remove the 2nd Param duplicate within each test. My file is as below: ...
1
vote
1answer
52 views

incrementing variables in awk if-else

I have the following code: TOTAL=0 COUNT=0 # a ratio of 10000 indicates that ILP # rejected was 0. awk -v c=$COUNT -v t=$TOTAL 'BEGIN { FS=OFS="," } NR==1 { $(NF+1)="RATIO"...
0
votes
1answer
33 views

awk catch missing columns

I have a file which looks like this: SOME,DESCRIPTIONS,OF,THE,COLUMNS 0,93,50127,0,49592,1,49592,1 100,195,107336,0,106491,2,107077,2 200,298,178492,3,170578,24,172045,13 ... 2590,0 And I'm trying to ...
-1
votes
1answer
49 views

Add and subtract values group by Unique ID

I have records like this: I have to add/subtract the values of columns 2 and 4 if there is add and subtract if it is DEL. 12345)63)ADD)3 12345)40)ADD)2 12345)40)ADD)2 12345)40)DEL)2 67892)30)ADD)5 ...
0
votes
0answers
44 views

how to properly match data in awk and fill in missind data based on a known date/time file

I have a data study running from 1/1/2002 00:00 to 12/31/19 23:00. Not all location have the same time ranges thus to make data processing easier I have a script to fill in the missing dates YYYYMMDD ...
-2
votes
3answers
64 views

Filter a word list using a stop words file

So I have a text file with a random text, and I need to list all the words present in that file, but filtering all the words common to my stop words file. What commands can I use for this?
0
votes
1answer
26 views

How to keep blank data or assign missing data when recompiuting data for unit shift

I have a series of data files with format below. I have a script to read just the data to the needed units (ppm to ppb) thus the *1000 of column $3. The issue which I am running into when awk runs and ...
1
vote
1answer
46 views

Concatenating text files based on prefix?

In a directory, I had 9792 files from 1088 groups (1088 x 9 = 9792) each group had a unique ID. I'm interested in concatenating only those files which match group ID as a prefix. OG00 - prefix for all ...
0
votes
1answer
44 views

Adding and changing columns in a file with awk

I have the file of about 300 hundreds lines TITLE cargas REMARK 1 File created by GaussView 5.0.9 HETATM 1 O 0 0.957 -0.000 -0.000 O HETATM 2 H ...
0
votes
3answers
54 views

Find multiple words and print the next numbers within a single line

I saved a single line sql result to a file. AA 10 BB 20 DD 14 GG 14 I want to count the numbers next to each words and save it as variables. And if there's none(like CC) it should be counted as 0. ...
0
votes
2answers
71 views

Creating new tables based on specific parameters from an existing table

I want to create several separate CSV files from a table.  Here is an example table: gene REF_S1_host REF_S1_FL S1_host1 S1_host2 S1_FL REF_S2_host REF_S2_FL S2_host1 S2_host2 S2_FL gene1 ...
-4
votes
0answers
59 views

Mutate a particular amino acid coordinates in protein file using grep or awk with sed command

I have a file which contains the atomic coordinates in coloumn. I have to substitute the coordinates (col 7,8,9,10,11,12) of LYS present, id 24, col 6, with PHE (col 4) res id (25, col 6). I was able ...
-1
votes
1answer
73 views

Extract columns from table using loop

I have this matrix. #chr st start PM1_r1 PM2_r1 PM3_r1 PM1_r2 PM2_r2 PM3_r2 chr1 - 10625 0 0 0 0 0 0 chr1 + 16616 0.10217712 0 0 1 0 0 chr1 - 120482 ...
1
vote
3answers
115 views

How to extract a substring starting with a word from string

I am writing a bash script and at some point I have $path="things useless things... path/another/files1/useful/path/here" So I am trying to remove all words and text before "files1"...
0
votes
3answers
70 views

Adding a column to a three columns file with awk

I would like to add a new column to an existing matrix in file.txt using awk. I have the file -1.664405 -0.019498 0.487501 -2.210156 0.326547 1.193717 -2.260318 -0....
-1
votes
5answers
103 views

Find and replace text in bash script terminal

I have a file at /home/count/1/details/info.txt as follow title1 { key1 value1 key2 value2 key3 value3 } info { name1 text post1 anything salary 1 work day1 work day2 work day3 } work { department ...
0
votes
3answers
94 views

awk getting rid of blank spaces

I am creating a script that checks to see if the first character in the second line of an output file is " " (blank or not). If it is blank I want to echo 'yes' if not echo 'no'. I am ...
-1
votes
5answers
98 views

Calculating specific average

I have a log with a long variable string and I am trying to get min, max and average from a number. String examples date time from Time: 100 ms to status code: date time Time: 1050 ms status code IP ...
-3
votes
0answers
62 views

how to sum first N numbers of a line

i have a file with a client name like client1.txt with first line represents the salary of client per month, second line represents the costs that he have per month. > 80 >> 12 10 19 20 18 16 ...
1
vote
3answers
56 views

Error in adding a suffix in a list of identifiers

this is a recurrent topic but still not duplicate since I tried all the solutions and none worked for me. I am trying to add the suffix "_1" to a list of identifiers imported from a txt file....
-1
votes
2answers
48 views

bash + print all enabled service on linux machine with up-time

I created the following simple script in order to give the list of all systemctl service with the uptime we run the script on our rhel 7.2 machine #!/bin/bash for i in ` systemctl list-unit-files | ...
4
votes
8answers
585 views

Find files with matched whole lines from a file

I have a file with this content: $ cat compromised_header.txt some unique string 1 some other unique string 2 another unique string 3 I wanted to find all files that have all the lines of above file ...
2
votes
3answers
240 views

Compare two lines in a file and delete those if pattern matches

I have a file like this. 12345 X678GHR 0 ADD 23445 HGT6787 1 ADD 12345 X678GHR 0 REM 67894 OIY5678 0 ADD 12345 OIY5678 0 ADD 12345 X678GHR 1 ADD I have to compare the lines in a file to delete the ...
0
votes
2answers
95 views

How to append a missing line only by comparing 2 columns in 2 different files

I have two files file1 (reference file) xxx xxxxx 00 xxx xxxxx 01 xxx xxxxx 02 xxx xxxxx 03 xxx xxxxx 04 xxx xxxxx 00 xxx xxxxx 01 xxx xxxxx 02 xxx xxxxx 03 xxx ...
-1
votes
2answers
68 views

Show content of specific line number from a file

So say i have a file with content on each lines, know the number of lines in advance and want to know the content of specific lines, i can do: awk 'NR==10' file But while i already know one way (...
-1
votes
1answer
42 views

Replacing numbers in a text file in each iteration with random values

I have a text file with the following content: number of apples 7 I would like to run the shell script and modify the number with random values for say 5 iterations. So I need my output to look like ...
0
votes
2answers
81 views

How to use awk instead of cmp to store comparison results to variables

Assume I have a file called checkDuplicateDownload.sh to check the downloaded file for duplicate or not, by comparing the new one and the old one. This is what I have done. duplicate=`cmp $prevFile ...
0
votes
1answer
61 views

How to sum to one column other column from a diffrent file?

I have something like this file1: element1,12 element2,10 element3,5 file2: IN element1,8 element3,5 and I want to modify file1 and obtain: element1,20 element2,10 element3,10 The first line from ...
1
vote
1answer
54 views

Get next line with awk

I am trying to calculate integral from bash. I have a file "data.log" which contains: x1 y1 x2 y2 x3 y3 ... xn yn I have created this file with awk. Now I'd like to manipulate it to get: x2 ...
-1
votes
3answers
76 views

Take the lines between two specific values from different columns in a txt file

I have a txt file (as show below) with many columns and I need to get the lines contained between the position of the second column to the position of the third column. 1 10385389 ...
-6
votes
1answer
51 views

find value and write it in 12,2 chars long format

example input: :20:210301/9944072 :25:PL63213000342008099410720012 :28C:6032563 :60F:C2101010101PLN4138,23 :61:2101010101D13,61 :86:^101 ^20Text ANY ^27YTRDTAL co. ltd. ^30 ^31 find lines that start ...
0
votes
3answers
71 views

compare the word after char occurrence in each line

I have a file with many lines like below. 123)hfd)jggg)7584)abc 123)))7584)abc )hfd)jggg)7584)cdf)))abc )hfd)))abc I have to output the lines with abc exactly after 4th occurrence of the ) [closing ...
2
votes
3answers
89 views

Grep a pattern in a line from log file and print next n lines until next patter

Suppose the file log.txt contains below content [12] 03/31/21 08:33:30.080851 T(12581) _DBG message x 1 [12] 03/31/21 08:33:30.080851 T(34897) _DBG message y 1 [12] 03/31/21 08:33:31.241167 T(12344) ...
1
vote
3answers
113 views

Get two values in each row from an existing file

I am trying to make a new file from an existing file, input.txt, using Unix commands. File input.txt type ABC=home,3,test,data type EFG=data,2,test_new,data_load type PQR=load,2,test_type,data_loader ...
0
votes
2answers
72 views

Need to merge lines for fixed width file based on length

Each line starts with FEPS. Merge the line which doesn't start with FEPS to previous line and then sort the file based on length i.e; keep the lines whose length is 23 chars Input: FEPS xxxx01 ...
0
votes
1answer
23 views

AWK - Select columns in a variable not input records

Assume that you have an AWK program file with a complex procedure: example.awk. You want to assign specific columns of a variable, rather than a whole line of the current record. #!/usr/bin/awk -f ...
-4
votes
2answers
60 views

Editing text file via awk command

I am very newbie about on that command, so far I tried few things with that but I failed every time.. I have a text file (MWE) like below: 26 6 2020 09:06:23.300 277.430 50.131 ...
0
votes
4answers
52 views

awk hostname and ip address from log files when the column numbers are different

This is my sample file. Actual data has way more data in it. wolf@linux:~$ more log* :::::::::::::: log1.csv :::::::::::::: hostname,sn,type,ip,random abc,a11,zzz,192.168.168.168,rrr def,e16,yyy,192....
-1
votes
2answers
91 views

I would like to get the names of entire headers present in another fasta file

I have a fasta file like that : >TRINITY_DN100_c0_g1_i1 len=242 path=[0:0-241] ...
0
votes
1answer
46 views

return lines containing numbers in a range in a certain column but only if a column contains a certain number

Related to this question... return lines containing numbers in a range in a certain column return lines containing numbers in a range in a certain column I know how to delete lines which contain ...

1
2 3 4 5
141