Questions tagged [awk]
A pattern-directed scanning and processing language.
7,688
questions
0
votes
3
answers
90
views
Awk adding corresponding rows in different columns
I have a file called presidents.txt. The content inside is as follows...
First Last Age Years
Geogre W 57 8.0
John A 61 4.0
Thomas J 57 8.0
James M 57 8.0
Andrew J 61 ...
2
votes
2
answers
86
views
How to format timestamp field with AWK/GAWK
I am processing a CSV file with GAWK that has a field with a timestamp formatted like this "18-APR-22 11:00:00". I would like to format the date like this in the output of awk "2022-04-...
0
votes
3
answers
112
views
Average of the columns and export it in another column
I have a txt file just like below (This picture is from the .csv version of it). What I like to do is take the average of the months (starting column 7 to 10) and export it to a new column. But it ...
0
votes
3
answers
102
views
awk decimal division
I have a file with values, some are whole numbers & some are with decimals
# cat file
value1 100
value2 500.10
value3 2505
value4 35.4
I want to divide each value in field2 with say a fixed ...
4
votes
6
answers
315
views
remove everything before a character in a column
I have a table like this
start end chr
1 10 H300Chr01
10 50 H500Chr02
I would like to replace everything before "Chr" in column3.
My desired output would be
start end chr
1 10 ...
-2
votes
2
answers
52
views
how to match value in file1 with value in file2 & get relevant info to add in file1?
I have 2 files, first row is a header row for reference
file1
userId userContact parentId parentContact
200 0900200 100 -
201 0900201 100 -
300 0900300 101 ...
-1
votes
3
answers
52
views
Print whole line in red if some numeric value is higher than 0
I have an out put that shows this, is a dimm diag result.
But i want to print in red only the result that are higher than 0, the rest should be in the default color. So the result only comes after ...
-3
votes
2
answers
71
views
Copy the last column for the upcoming rows
I have a file that looks like this:
FILE:
900 1 37
900 2 41
900 3 278
918 1 9999
918 2 9999
918 3 9999
And I want to obtain a new file which looks like this:
918 1 37
918 2 41
918 3 278
I want to ...
2
votes
5
answers
253
views
match value2 in 2 files if value 1 is exact match
I have 2 files containing list. Column 1 is userIds & column 2 is associated values
# cat file1
e3001 75
n5244 30
w1453 500
#cat file2
d1128 30
w1453 515
n5244 30
e3001 55
Things to consider.
...
1
vote
2
answers
103
views
Change color of a certain lines from an output bash
I have this output from a drive diag command.
Slot Number : 0
Drive's position : DiskGroup 0
Media Error : 0
Other Error : 0
Predictive Failure : 0
Raw Size ...
0
votes
4
answers
131
views
Flexible Pattern Matching
I have a file that looks like this:
FILE1:
0/28
7200/11
14400/11
21584/28
21600/11
28800/28
36000/11
36000/28
43200/11
43200/28
50400/11
57600/11
79200/28
In the left part (before / ) I have the time ...
-2
votes
3
answers
56
views
Search then replace it with last 4 characters
I have here a csv file with the following data
1,2,CC-12345678-20201120,6
1,3,CC-87654321-20201110,5
I need to replace the specific string after “CC-“ with its 4 last digit
Expected output:
1,2,CC-...
1
vote
3
answers
61
views
How to delete a set of lines using awk when certain condition is met
I am using awk to delete duplicates in files, the problem is that I want to delete a set of lines if a duplicate is found, for eg-
<p>
This is duplicate.
</p>
<p>
This is original.
&...
0
votes
2
answers
56
views
Remove Records with more than 30 of the same value
I have a large CSV, and want to remove the records that have the same FirstName field ($8), MiddleName field ($9) and LastName field ($10) if there is more than 30 instances of it.
TYPE|10007|44|Not ...
1
vote
2
answers
46
views
How to remove duplicates from pipe delimited file using awk with Pipe in Values?
I am trying to remove duplicates based on multiple columns from a pipe delimited file using this How to remove duplicates based on multiple dynamic columns
But I found there are pipes within the ...
3
votes
8
answers
656
views
Get result from awk using two delimiters in a single command
Output of just the ping command:
[root@servera ~]# ping -c 4 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=128 time=8.04 ms
64 bytes from 8.8.8.8: icmp_seq=...
1
vote
1
answer
57
views
Sum of Column with Rows Specific and write that value to .txt file
I have number of text files which has 70000 rows and 2 column. I would like to sum entries of 2nd column from 40000 to 70000 row and value of sum is to written to new text file.
e.g:-Suppose
Data1_old....
-2
votes
1
answer
40
views
AWK command ignores adding newline ending "\n" for blank line and print in one single line
When I use the below command, it adds \n to all the lines but skips for a blank line.
awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}'
Example:
Input:
George Washington, 1789-1797
...
0
votes
1
answer
39
views
Can't make awk loop over file values
Hi guys
I'm trying to loop over the values of a file (or multiple files) passed as a parameter to the script as in, ./myscript file1 file2 ... but awk only recognizes the first line of each file.
#!/...
3
votes
3
answers
90
views
Sed replace with any random word in string
Hello I am trying to do a replace on a file where a string is followed by any random string using sed. My original file looks like below.
access = read : [ * ], write : [ admin, sc_admin ]
access = ...
1
vote
2
answers
47
views
Join two files ordered numerically according to second column
I have two files
file 1 contains
2*J=0 EXP= 0.00000
2*J=4 EXP= 1.27911
2*J=8 EXP= 1.57613
2*J=12 EXP= 1.69134
2*J=10 EXP= 2.72705
2*J=16 EXP= 4.55689
2*J=20 EXP= 5.62138
file 2 contains
2*J=0 EXC= 0....
0
votes
1
answer
45
views
How to remove duplicates based on multiple dynamic columns
using bash(awk), how do I remove duplicates from a pipe delimited file based on multiple columns but columns need to be passed as input arguments.
I have below data in the file
Insert|val1|val2|val3|...
0
votes
3
answers
112
views
How to print the text after the first occurence of a "start pattern"-"stop pattern" pair?
I have a file that contains a bunch of certificates:
-----BEGIN CERTIFICATE-----
AAAAAAA
AAAAAAA
AAAAAAA
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
BBBBBBB
BBBBBBB
BBBBBBB
-----END ...
-1
votes
5
answers
108
views
How to make sure that the last character of each line of a file ends with single quote character
I have a file which has several lines.
I wish to update the file while making sure that the last visible character of each line of the file is a single quote '. In case not, then we should add the ...
0
votes
1
answer
78
views
How to add single quote at the end of ONLY the last line of a file [duplicate]
I want a non-perl solution where I should be able to append a single quote ' at the last visible (non-escape characters) line of a file and save it back to the file.
cat example.txt
var1: 'funn'
var2: ...
0
votes
2
answers
53
views
sed and awk in sh script not replacing text
The desired work to find a string, and replace to another (all occurrencies), and rewrite/update the file too.
The file is an pbxproj file. I linked a sample of this file
sample of the file (please ...
3
votes
5
answers
316
views
find duplicate 1st field and concat its values in single line
I have a file that has entries in key: value format like the below:
cat data.txt
name: 'tom'
tom_age: '31'
status_tom_mar: 'yes'
school: 'anne'
fd_year_anne: '1987'
name: 'hmz'
hmz_age: '21'
...
-2
votes
2
answers
59
views
I need to update a tag <abc><\abc> with variables from another file. Output should be an xml file with all those values comma separated within abc tag
I need help updating an xml file, file1.xml which has a tag name <abc> <\abc>.
The file2.txt has numbers fixed length numbers (1 number in each row):
1234
5678
1435
I need output to a ...
0
votes
1
answer
40
views
Add value to a string
I have a large bash file that has a lot of lines like this:
... --env COMP_LABELS="service:service1" ...
Where service1 differs in every line. (sometimes it's service:mgmt or service:admin)....
0
votes
4
answers
47
views
concise but readable way to extract the numerical value after the substring "mode:" in a shell script
I have this string:
DMT mode 4: 640x480 @ 60Hz 4:3, clock:25MHz progressive
I want to extract from it the mode number (the part before the first :), which in this example is 4. As you might ...
0
votes
3
answers
91
views
convert a new line to a tab formatted file
I have multiple files namely SRR3384742.Gene.out.tab SRR3384743.Gene.out.tab SRR3384744.Gene.out.tab and many more in that order. I am extracting first and fourth columns from these files and store it ...
0
votes
2
answers
62
views
awk matching fields in 2 separate files using command containing {BEGIN FS = "|"} returns output of blank lines
Two files: data1
Name |formula |no. |dose|days|cost |msg|em|notes
Fname-Lname|BXXXT+GG |8262|4 |14 |57.78 | | |sq
Fname-Lname|SJXXT+GG |8263|4¾ |...
0
votes
1
answer
49
views
What does " records[rec] = (rec in records) ? records[rec] : func(rec) " command in awk mean?
Can anyone tell me what this expression means in awk?
records[rec] = (rec in records) ? records[rec] : func(rec)
3
votes
3
answers
239
views
Grep or awk between 2 large files where file1 is a pattern file and file2 contains the data
File1 has numbers of fixed length, with one number in each row.:
1234
5678
1345
File2 contains data fields in "key=value" format:
abc:def=1999,xyz=1234;
abc:def=5678,xyz=1234;
abc:def=1234,...
0
votes
4
answers
66
views
Sed Merge split lines between two different patterns
I want to merge lines between ^pattern2 and its ";.
Change this:
pattern2
"xxx xxxxxx xxxxxxxx";
pattern2 "xxxx xxxxxxx xxxxxxxxx xxxxxxxxx";
pattern2
"xxxx xxxxxxx ...
0
votes
1
answer
47
views
ubuntu20.04 awk Variable assignment
I want to get two numbers in two lines
awk '/\!/ {E=$5} $1=="PWSCF" {printf"%4d %s %s\n",'$ecut',E,$3}' $name.$ecut.out >> calc-ecut.dat.3
and i got the error message
awk: ...
0
votes
7
answers
111
views
AWK: Concatenate field from different records
Given file:
2018-03-22 foo/bar/baz
2020-09-30 Lorem/ipsum/dolor
2021-10-01 yadda/yadda/yadda
2022-03-14 blah/blah/blah
(actual files contain thousands of such lines)
How to obtain the string 2018-03-...
2
votes
1
answer
53
views
awk print does not work in bash -c
I noticed that this works fine (will print only the PID of unattended-upgrades process):
lslocks | awk '/unattended-upgrades/ { print $2 }'
But this effectively ignores the print part (will print ...
2
votes
3
answers
228
views
Delete three lines in specific step
I have data like that (this is only a small part):
454SOL OW20704 0.317 0.251 5.525
454SOL HW120705 0.388 0.322 5.531
454SOL HW220706 0.229 0.290 5.553
462SOL ...
1
vote
2
answers
72
views
replace a string by using part of the string in the same line
I have a file like the following on a Linux machine, and the gene_id is wrong. So I would like to replace the string "PB" in the gene_id with the transcript_id from the same line.
$ cat try....
0
votes
2
answers
55
views
Append specific value to end of line between patterns
I would like to move a line of text to the end of the line below, and replicate the value moved for x lines, until a new match for the pattern is found. All the other text processing for my files I ...
1
vote
3
answers
68
views
How to extract path names from a log file?
This is similar to eg this , but not quite; I have the output from strace saved into a file, like this:
...
3691 fcntl(2, F_GETFD) = 0
3691 access("/etc/suid-debug", F_OK) ...
0
votes
3
answers
49
views
Append data from one file to another
So I have a script that creates a CSV file looking like this:
Timestamp,Latitude,Longitude,MinSeaLevelPressure,MaxIntensity
1800 UTC MAY 16,
0000 UTC MAY 17,
0600 UTC MAY 17,
1200 UTC MAY 17,
1800 UTC ...
4
votes
4
answers
797
views
Extracting sea level pressure from KML using Shell
I'm currently working on a little project; in a kml file called weatherdata.kml, I would like to extract the sea level pressure for each <Placemark> element. I'm trying to parse the information ...
0
votes
3
answers
44
views
sed, awk to copy paste pattern1 line into following pattern2 lines
I'm looking to copy -city- line to end of "all" following -business- lines
This:
-country-france
-city-Paris, Marseille, Lyon, Toulouse, Nice, Nantes
-business-TotalEnergies
-business-AXA
-...
0
votes
2
answers
77
views
How to use awk $1 $2 in bash shell?
cat << EOF > test.sh
cat $FILEA $FILEB | sort -k1,1 -k2,2n | mergeBed -i - | awk -F\\t '{print $1 "\t" NR "\t\t" $2 "\t" $3 "\t\t.\t\t" NR}' > ...
1
vote
2
answers
59
views
Regexp to fix GPS data
I'm looking to fix GPS data in text file on Linux
Incorrect string model: 49:29:37N 005:28:39E
2or3digits:2or3digits:2or3digits(N|S) 2or3digits:2or3digits:2or3digits(E|W)
Correct string model: 49:29:...
-2
votes
3
answers
96
views
How can I remove a character {only found between curly brackets}, and take the brackets along for the ride?
For example, if I'm removing only spaces between brackets, the title of this post would be:
How can I remove a character onlyfoundbetweenbrackets, and take the brackets along for the ride?
I've only ...
-2
votes
1
answer
73
views
awk commande-line [closed]
please, help for this commande. so I write this commande
awk -F":" '{tab[$5]+=1} END { for (i in tab) {print i,tab[i]}}' AnnuaireBis.txt
who give this output:
Ketou 4
Anneho 4
Panhouignan 4
...
2
votes
3
answers
133
views
Insert space character before a certain character
I have a file which looks like this:
120E+00 360E+01 258E-09-784E+02
-847E+00-360E+01 242E-08 420E+05
14E+00 360E+01 254E+00 78E+02
-120E+00-254E-01-223E-09-786E-02
E+00,E+01,E-01 etc are in ...