Questions tagged [awk]

A pattern-directed scanning and processing language.

Filter by
Sorted by
Tagged with
0 votes
3 answers
90 views

Awk adding corresponding rows in different columns

I have a file called presidents.txt. The content inside is as follows... First Last Age Years Geogre W 57 8.0 John A 61 4.0 Thomas J 57 8.0 James M 57 8.0 Andrew J 61 ...
user avatar
  • 11
2 votes
2 answers
86 views

How to format timestamp field with AWK/GAWK

I am processing a CSV file with GAWK that has a field with a timestamp formatted like this "18-APR-22 11:00:00". I would like to format the date like this in the output of awk "2022-04-...
user avatar
0 votes
3 answers
112 views

Average of the columns and export it in another column

I have a txt file just like below (This picture is from the .csv version of it). What I like to do is take the average of the months (starting column 7 to 10) and export it to a new column. But it ...
user avatar
0 votes
3 answers
102 views

awk decimal division

I have a file with values, some are whole numbers & some are with decimals # cat file value1 100 value2 500.10 value3 2505 value4 35.4 I want to divide each value in field2 with say a fixed ...
user avatar
  • 1,603
4 votes
6 answers
315 views

remove everything before a character in a column

I have a table like this start end chr 1 10 H300Chr01 10 50 H500Chr02 I would like to replace everything before "Chr" in column3. My desired output would be start end chr 1 10 ...
user avatar
-2 votes
2 answers
52 views

how to match value in file1 with value in file2 & get relevant info to add in file1?

I have 2 files, first row is a header row for reference file1 userId userContact parentId parentContact 200 0900200 100 - 201 0900201 100 - 300 0900300 101 ...
user avatar
  • 1,603
-1 votes
3 answers
52 views

Print whole line in red if some numeric value is higher than 0

I have an out put that shows this, is a dimm diag result. But i want to print in red only the result that are higher than 0, the rest should be in the default color. So the result only comes after ...
user avatar
-3 votes
2 answers
71 views

Copy the last column for the upcoming rows

I have a file that looks like this: FILE: 900 1 37 900 2 41 900 3 278 918 1 9999 918 2 9999 918 3 9999 And I want to obtain a new file which looks like this: 918 1 37 918 2 41 918 3 278 I want to ...
user avatar
  • 69
2 votes
5 answers
253 views

match value2 in 2 files if value 1 is exact match

I have 2 files containing list. Column 1 is userIds & column 2 is associated values # cat file1 e3001 75 n5244 30 w1453 500 #cat file2 d1128 30 w1453 515 n5244 30 e3001 55 Things to consider. ...
user avatar
  • 1,603
1 vote
2 answers
103 views

Change color of a certain lines from an output bash

I have this output from a drive diag command. Slot Number : 0 Drive's position : DiskGroup 0 Media Error : 0 Other Error : 0 Predictive Failure : 0 Raw Size ...
user avatar
0 votes
4 answers
131 views

Flexible Pattern Matching

I have a file that looks like this: FILE1: 0/28 7200/11 14400/11 21584/28 21600/11 28800/28 36000/11 36000/28 43200/11 43200/28 50400/11 57600/11 79200/28 In the left part (before / ) I have the time ...
user avatar
  • 69
-2 votes
3 answers
56 views

Search then replace it with last 4 characters

I have here a csv file with the following data 1,2,CC-12345678-20201120,6 1,3,CC-87654321-20201110,5 I need to replace the specific string after “CC-“ with its 4 last digit Expected output: 1,2,CC-...
user avatar
  • 1
1 vote
3 answers
61 views

How to delete a set of lines using awk when certain condition is met

I am using awk to delete duplicates in files, the problem is that I want to delete a set of lines if a duplicate is found, for eg- <p> This is duplicate. </p> <p> This is original. &...
user avatar
0 votes
2 answers
56 views

Remove Records with more than 30 of the same value

I have a large CSV, and want to remove the records that have the same FirstName field ($8), MiddleName field ($9) and LastName field ($10) if there is more than 30 instances of it. TYPE|10007|44|Not ...
user avatar
  • 1
1 vote
2 answers
46 views

How to remove duplicates from pipe delimited file using awk with Pipe in Values?

I am trying to remove duplicates based on multiple columns from a pipe delimited file using this How to remove duplicates based on multiple dynamic columns But I found there are pipes within the ...
user avatar
  • 15
3 votes
8 answers
656 views

Get result from awk using two delimiters in a single command

Output of just the ping command: [root@servera ~]# ping -c 4 8.8.8.8 PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. 64 bytes from 8.8.8.8: icmp_seq=1 ttl=128 time=8.04 ms 64 bytes from 8.8.8.8: icmp_seq=...
user avatar
  • 125
1 vote
1 answer
57 views

Sum of Column with Rows Specific and write that value to .txt file

I have number of text files which has 70000 rows and 2 column. I would like to sum entries of 2nd column from 40000 to 70000 row and value of sum is to written to new text file. e.g:-Suppose Data1_old....
user avatar
  • 19
-2 votes
1 answer
40 views

AWK command ignores adding newline ending "\n" for blank line and print in one single line

When I use the below command, it adds \n to all the lines but skips for a blank line. awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' Example: Input: George Washington, 1789-1797 ...
user avatar
0 votes
1 answer
39 views

Can't make awk loop over file values

Hi guys I'm trying to loop over the values of a file (or multiple files) passed as a parameter to the script as in, ./myscript file1 file2 ... but awk only recognizes the first line of each file. #!/...
user avatar
3 votes
3 answers
90 views

Sed replace with any random word in string

Hello I am trying to do a replace on a file where a string is followed by any random string using sed. My original file looks like below. access = read : [ * ], write : [ admin, sc_admin ] access = ...
user avatar
1 vote
2 answers
47 views

Join two files ordered numerically according to second column

I have two files file 1 contains 2*J=0 EXP= 0.00000 2*J=4 EXP= 1.27911 2*J=8 EXP= 1.57613 2*J=12 EXP= 1.69134 2*J=10 EXP= 2.72705 2*J=16 EXP= 4.55689 2*J=20 EXP= 5.62138 file 2 contains 2*J=0 EXC= 0....
user avatar
0 votes
1 answer
45 views

How to remove duplicates based on multiple dynamic columns

using bash(awk), how do I remove duplicates from a pipe delimited file based on multiple columns but columns need to be passed as input arguments. I have below data in the file Insert|val1|val2|val3|...
user avatar
  • 15
0 votes
3 answers
112 views

How to print the text after the first occurence of a "start pattern"-"stop pattern" pair?

I have a file that contains a bunch of certificates: -----BEGIN CERTIFICATE----- AAAAAAA AAAAAAA AAAAAAA -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- BBBBBBB BBBBBBB BBBBBBB -----END ...
user avatar
  • 3
-1 votes
5 answers
108 views

How to make sure that the last character of each line of a file ends with single quote character

I have a file which has several lines. I wish to update the file while making sure that the last visible character of each line of the file is a single quote '. In case not, then we should add the ...
user avatar
  • 297
0 votes
1 answer
78 views

How to add single quote at the end of ONLY the last line of a file [duplicate]

I want a non-perl solution where I should be able to append a single quote ' at the last visible (non-escape characters) line of a file and save it back to the file. cat example.txt var1: 'funn' var2: ...
user avatar
  • 297
0 votes
2 answers
53 views

sed and awk in sh script not replacing text

The desired work to find a string, and replace to another (all occurrencies), and rewrite/update the file too. The file is an pbxproj file. I linked a sample of this file sample of the file (please ...
user avatar
  • 13
3 votes
5 answers
316 views

find duplicate 1st field and concat its values in single line

I have a file that has entries in key: value format like the below: cat data.txt name: 'tom' tom_age: '31' status_tom_mar: 'yes' school: 'anne' fd_year_anne: '1987' name: 'hmz' hmz_age: '21' ...
user avatar
  • 297
-2 votes
2 answers
59 views

I need to update a tag <abc><\abc> with variables from another file. Output should be an xml file with all those values comma separated within abc tag

I need help updating an xml file, file1.xml which has a tag name <abc> <\abc>. The file2.txt has numbers fixed length numbers (1 number in each row): 1234 5678 1435 I need output to a ...
user avatar
  • 11
0 votes
1 answer
40 views

Add value to a string

I have a large bash file that has a lot of lines like this: ... --env COMP_LABELS="service:service1" ... Where service1 differs in every line. (sometimes it's service:mgmt or service:admin)....
user avatar
0 votes
4 answers
47 views

concise but readable way to extract the numerical value after the substring "mode:" in a shell script

I have this string: DMT mode 4: 640x480 @ 60Hz 4:3, clock:25MHz progressive I want to extract from it the mode number (the part before the first :), which in this example is 4. As you might ...
user avatar
  • 191
0 votes
3 answers
91 views

convert a new line to a tab formatted file

I have multiple files namely SRR3384742.Gene.out.tab SRR3384743.Gene.out.tab SRR3384744.Gene.out.tab and many more in that order. I am extracting first and fourth columns from these files and store it ...
user avatar
  • 63
0 votes
2 answers
62 views

awk matching fields in 2 separate files using command containing {BEGIN FS = "|"} returns output of blank lines

Two files: data1 Name |formula |no. |dose|days|cost |msg|em|notes Fname-Lname|BXXXT+GG |8262|4 |14 |57.78 | | |sq Fname-Lname|SJXXT+GG |8263|4¾ |...
user avatar
  • 25
0 votes
1 answer
49 views

What does " records[rec] = (rec in records) ? records[rec] : func(rec) " command in awk mean?

Can anyone tell me what this expression means in awk? records[rec] = (rec in records) ? records[rec] : func(rec)
user avatar
3 votes
3 answers
239 views

Grep or awk between 2 large files where file1 is a pattern file and file2 contains the data

File1 has numbers of fixed length, with one number in each row.: 1234 5678 1345 File2 contains data fields in "key=value" format: abc:def=1999,xyz=1234; abc:def=5678,xyz=1234; abc:def=1234,...
user avatar
  • 33
0 votes
4 answers
66 views

Sed Merge split lines between two different patterns

I want to merge lines between ^pattern2 and its ";. Change this: pattern2 "xxx xxxxxx xxxxxxxx"; pattern2 "xxxx xxxxxxx xxxxxxxxx xxxxxxxxx"; pattern2 "xxxx xxxxxxx ...
user avatar
0 votes
1 answer
47 views

ubuntu20.04 awk Variable assignment

I want to get two numbers in two lines awk '/\!/ {E=$5} $1=="PWSCF" {printf"%4d %s %s\n",'$ecut',E,$3}' $name.$ecut.out >> calc-ecut.dat.3 and i got the error message awk: ...
user avatar
  • 5
0 votes
7 answers
111 views

AWK: Concatenate field from different records

Given file: 2018-03-22 foo/bar/baz 2020-09-30 Lorem/ipsum/dolor 2021-10-01 yadda/yadda/yadda 2022-03-14 blah/blah/blah (actual files contain thousands of such lines) How to obtain the string 2018-03-...
user avatar
2 votes
1 answer
53 views

awk print does not work in bash -c

I noticed that this works fine (will print only the PID of unattended-upgrades process): lslocks | awk '/unattended-upgrades/ { print $2 }' But this effectively ignores the print part (will print ...
user avatar
  • 189
2 votes
3 answers
228 views

Delete three lines in specific step

I have data like that (this is only a small part): 454SOL OW20704 0.317 0.251 5.525 454SOL HW120705 0.388 0.322 5.531 454SOL HW220706 0.229 0.290 5.553 462SOL ...
user avatar
  • 165
1 vote
2 answers
72 views

replace a string by using part of the string in the same line

I have a file like the following on a Linux machine, and the gene_id is wrong. So I would like to replace the string "PB" in the gene_id with the transcript_id from the same line. $ cat try....
user avatar
  • 25
0 votes
2 answers
55 views

Append specific value to end of line between patterns

I would like to move a line of text to the end of the line below, and replicate the value moved for x lines, until a new match for the pattern is found. All the other text processing for my files I ...
user avatar
1 vote
3 answers
68 views

How to extract path names from a log file?

This is similar to eg this , but not quite; I have the output from strace saved into a file, like this: ... 3691 fcntl(2, F_GETFD) = 0 3691 access("/etc/suid-debug", F_OK) ...
user avatar
  • 335
0 votes
3 answers
49 views

Append data from one file to another

So I have a script that creates a CSV file looking like this: Timestamp,Latitude,Longitude,MinSeaLevelPressure,MaxIntensity 1800 UTC MAY 16, 0000 UTC MAY 17, 0600 UTC MAY 17, 1200 UTC MAY 17, 1800 UTC ...
user avatar
4 votes
4 answers
797 views

Extracting sea level pressure from KML using Shell

I'm currently working on a little project; in a kml file called weatherdata.kml, I would like to extract the sea level pressure for each <Placemark> element. I'm trying to parse the information ...
user avatar
0 votes
3 answers
44 views

sed, awk to copy paste pattern1 line into following pattern2 lines

I'm looking to copy -city- line to end of "all" following -business- lines This: -country-france -city-Paris, Marseille, Lyon, Toulouse, Nice, Nantes -business-TotalEnergies -business-AXA -...
user avatar
0 votes
2 answers
77 views

How to use awk $1 $2 in bash shell?

cat << EOF > test.sh cat $FILEA $FILEB | sort -k1,1 -k2,2n | mergeBed -i - | awk -F\\t '{print $1 "\t" NR "\t\t" $2 "\t" $3 "\t\t.\t\t" NR}' > ...
user avatar
1 vote
2 answers
59 views

Regexp to fix GPS data

I'm looking to fix GPS data in text file on Linux Incorrect string model: 49:29:37N 005:28:39E 2or3digits:2or3digits:2or3digits(N|S) 2or3digits:2or3digits:2or3digits(E|W) Correct string model: 49:29:...
user avatar
  • 13
-2 votes
3 answers
96 views

How can I remove a character {only found between curly brackets}, and take the brackets along for the ride?

For example, if I'm removing only spaces between brackets, the title of this post would be: How can I remove a character onlyfoundbetweenbrackets, and take the brackets along for the ride? I've only ...
user avatar
  • 19
-2 votes
1 answer
73 views

awk commande-line [closed]

please, help for this commande. so I write this commande awk -F":" '{tab[$5]+=1} END { for (i in tab) {print i,tab[i]}}' AnnuaireBis.txt who give this output: Ketou 4 Anneho 4 Panhouignan 4 ...
user avatar
2 votes
3 answers
133 views

Insert space character before a certain character

I have a file which looks like this: 120E+00 360E+01 258E-09-784E+02 -847E+00-360E+01 242E-08 420E+05 14E+00 360E+01 254E+00 78E+02 -120E+00-254E-01-223E-09-786E-02 E+00,E+01,E-01 etc are in ...
user avatar
  • 69