Questions tagged [awk]
A pattern-directed scanning and processing language.
5,716
questions
1
vote
4answers
75 views
Print value from the next line
I have file looks like that
User Charts
User ID:
40944827
User Name:
Joe, Neo
Gender:
M
DOB:
3/20/2000 12:00:00 AM - Age: 20 yr. 10 mo. old
I want to use awk or sed to get results in format of
...
1
vote
1answer
25 views
Transpose columns into rows with first column data repeating using awk
I've the input in test.txt as
month,var1,var2
June,10,a
Sep,20,b
Dec,30,c
I want the output as
month,variable,value
June,var1,10
June,var2,a
Sep,var1,20
Sep,var2,b
Dec,var1,30
Dec,var2,c
Note ...
0
votes
4answers
67 views
SED specific column replace
How to replace A to RA in column 4 using Sed command? (between these columns it contain more than one spaces)
ATOM 32 P A 2 6.882 -5.338 6.560 1.00 0.00 P
ATOM ...
-2
votes
2answers
40 views
Replace item4 and item5 with item3
If I have a data like this,
1 item1,
2 item2,
3 item3,
4 item4,
5 item5
I want to replace item4 and item5 with item3
Is there any other than this?
awk -F ' ' '{gsub("item4","item3",$2);gsub("item5"...
-1
votes
3answers
67 views
AWK How to take value from header and print it in columns?
I have to get some values from header in txt files and duplicate them as column the number of rows change from file to file , here is an example.
Input file:
PROJECT : PRK33
X_COR : 258524
Y_COR : ...
0
votes
2answers
43 views
Perform series of commands on every value in a field
I have a tsv file. For a specific value in a column 5, I want to extract all the rows, and then cut three columns and then count the unique lines. For example for string 'abc' in column 5, I want
awk ...
2
votes
5answers
73 views
Compare 2 files based on the first column and print the not matched
File# 1:
test1,1
test2,2
test3
File# 2:
test2
test1
test4
Desired Output:
test4
0
votes
2answers
24 views
Speedtest-CLI output for monitoring
So I'm pretty new to Linux and I really can't figure it how to.
So I wanted to make a useful output for our monitoring Tool with speedtest-cli. We have to monitor download and upload speeds for ...
-1
votes
4answers
59 views
If find 4 digit number print the number and next row value in csv
I have data in csv format:
125867569,98652343,7012,879456217,20121221,7065,758964231,856421345
My desired output:
7012,879456217,7065,758964231
How can I do this?
1
vote
1answer
23 views
Find row containing string, then return that row and all following rows of text file with awk
Apologies for any duplication, but most questions I've come across relate to getting a specific value from a field in a row, or using tail to get n tailing lines from a file, where n is known a priori....
2
votes
2answers
36 views
Search for a String in a file and save the string after it to a variable
Example:
Lets say this is in a file:
AB: Test CD: Testing EF: Rest
I want a script to search for CD and save "Testing" into a variable.
I've tried something with awk but somehow I'm not making any ...
-1
votes
3answers
46 views
Remove and Replace all the header sequences
kindly your comment highly appreciated
have more than 4k header sequences look like:
>LTR_retrotransposon100_Gypsy?
cgtgcaccccaatgttcatagcagcactatttacaatagccaagacatggaaacaaccta ...
0
votes
2answers
49 views
Substitute pattern in a column
I have a requirement to substitute pattern in a file from different file as input.
Lets say file1 content:
ab 10
bc 20
cd 30
de 40
file2:
server1;10 feb 2020;disk5;123455678;comment;10;1;desc;...
0
votes
3answers
40 views
How to add new columns in text file with multiplication of several sequence?
I have a text file which has 4 columns with 300 lines. I want to add 5th columns where 1st hundred lines will be multiplication of 1*0.02, 2*0.02... and from hundred one lines 2.2, 2.4 ...
0.02
0....
0
votes
2answers
70 views
Print Above line and below four lines if a pattern matchs
I have a text file say input.txt which contains below data
$ cat input.txt
text1
error
text2
text3
text4
xyz
asd
asdf
text10
error
text11
text12
text13
text14
def
ghi
jkl
text16
error
text17
text18
...
0
votes
0answers
67 views
help with awk command
Appreciate all your effort and help, if you could get the following output from a given input file.
Input:
Chr01 JGI gene 82773 86941 . - . ID=Potri.001G000900;Name=Potri.001G000900
...
0
votes
2answers
45 views
Matching columns of File1 to columns of file 2 and printing them in a file using awk
I am new to awk and trying to compare ":" separated columns 1(eg., chr10) and 2 (eg 10000003) between file1 and file 2 ; and write the matching lines into a new file using awk.
File 1:
chr10:...
0
votes
1answer
26 views
sum different column values with each other using awk
Could you please help me through the below issue:
I have below input:
pmNoNormalRabReleaseSpeech 1467 1281 1121 1046
pmNoRabEstablishAttemptSpeech 1479 1282 1128 1026
...
0
votes
4answers
43 views
How to multiply different numbers in specific column in specific rows?
I want to multiply 5, 4, 3 in the 2nd column (1st 5 rows), (6th to 8th rows) and (9th to 10th rows) respectively.
I can only multiply one number using below command
awk '{print $1, $2*5, $3,$4}' ...
0
votes
1answer
51 views
Awk: Bypass header without writing code
Here is an expression I type on a routine basis because everyone uses headed CSV files, and I must pass around the header for my purposes as well:
cat foo.csv | awk -F',' 'BEGIN{start=0}{if(start==0){...
-1
votes
0answers
57 views
AWK gsub function is failing if substitute string exceeds 500 characters [closed]
I am using gsub() in my awk construct, and it fails if the string value by which the substitution is to happen exceeds 500 characters.
The reason why I am using gsub() is that I am going to perform ...
0
votes
1answer
79 views
Count occurence of raw for users
I have tab delimited file is as follows:
Presence For Month 01/2020
Sl No Salary no Name 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 ...
2
votes
1answer
60 views
Repeat external command for each line in awk
With the following content in the clipboard:
curl "foo" --compressed ;
curl "bar" --compressed ;
curl "baz" --compressed ;
The following command creates one line for each clipboard-line:
pbpaste | ...
2
votes
3answers
169 views
convert one line values to multiple lines with numbering order
The following variable include for example this values
echo $SERVERS
server1,server2,server3,server4,server5
and when I want to pipe them on different lines then I do the following
echo $SERVERS | ...
0
votes
1answer
32 views
Cutting out specific text from specific lines in real time
So I have a log running in real time and saving to a text file named 'test.txt' also in real time. Now I want to set up a process that constantly polls that text file for changes and cuts out a ...
1
vote
3answers
47 views
Awk to sum all numbers only after a character string
I've a file with input as:
abc
def
ghi
23
54
12
tuy
12
iuy
56
67
I want to add all numbers after a character line and print it in front of character, like below:
abc
def
ghi 89
tuy 12
iuy 123
Can ...
0
votes
1answer
29 views
arpscan to get Mac Address, suppress first 2, last 3 lines
I currently get the result I want by using this:
arp-scan -I eno1 -l | cut -f2-3 | tr [':lower:'] [':upper:'] | sed 1,2d | head -n -3
Someone told me I am spawning too many processes and that it's ...
2
votes
2answers
57 views
Add all the row values from file1 to file2, if a column value in file1 exists in file2
I have two files
FileA
Aacs 4498.5 778.5 995.5 1263 530
Abracl 2824 19.31 58.45 55.67 33.075
Ackr3 1294.5 19.5 47.5 38.5 4.5
Actb 246564 4472.46 11134.1 12272.6 ...
0
votes
0answers
23 views
Bash array: invalid subscript expression [duplicate]
I am new to bash so this bug might be silly. Below is my code and it gives me a syntax error around every curly braces in the awk command. Can someone point out where it goes wrong? Thanks a lot!
...
1
vote
4answers
63 views
Separate multiple sections/lines from a file
I have a file with below sample texts:
THIS(
is first
Line);
THAT(
is second line);
THIS(
is third
line);
THAT(
is
fourth
line);
If you see the file, each sections are either starts with "THIS" or "...
1
vote
3answers
71 views
awk — replace once after immediate pattern in same line!
We have a 1.5MB file.
One particular line has about 160000 characters.
In the line we need to change "false" into "true" only once immediate after PATTERN.
Following awk only changes first ...
0
votes
1answer
59 views
join csv with many fields without disturbing order
I have 2 csv files. The first file is big (>400 fields and many rows > 1 mil) and needs to have another field appended to it, via a matched join.
I want to join on a field $170
I have tried
gawk '...
1
vote
2answers
68 views
How to select lines of file based on multiple conditions of another file?
I have 2 genetic datasets. I filter file1 based on a column in file2. However, I also need to account for a second column in file2 and I'm not sure how to do this.
The condition for file 1 line ...
0
votes
1answer
39 views
Column transposition according to two values
I have a file with the following structure:
Numero_di_conferimento Numero_Campione Identificazione Valore Data_del_prelievo Data Analisi
358172 1 80_T1_Glu_Gln 0,14 17/10/2019 Granulociti_basofili_(...
0
votes
4answers
57 views
Awk: Insert line after N lines following the pattern
I'd like to insert a line at the 3rd line following the pattern. Example: insert the word z 3 lines after each match of a
From
a
b1
c1
d1
a
b2
c2
d2
...
to
a
b1
c1
z
d1
a
b2
c2
z
d2
...
0
votes
1answer
24 views
In CSV file, Based on match pattern,fetch the other value matched pattern
Need your expert advice on this
AMOUNT,30,/AMOUNT,MESS,Am,/MESS,AMOUNT,30,/AMOUNT,TXN,209,/TXN
MESS,SU,/MESS,TXN,200,/TXN,AMOUNT,70,/AMOUNT
TXN,200,/TXN,AMOUNT,90,/AMOUNT,MESS,SUM,/MESS,...
0
votes
1answer
15 views
Duplicate columns of a bed file to produce another bed file
I have a bed file genome_cov.bed specifying the coverage of a genome at every position on the genome. The columns are; scaffold, position, coverage.
scaffold_1 1 0
scaffold_1 2 ...
0
votes
1answer
33 views
How to select rows from a file based on conditions of another file?
I have 2 genetic datasets, both have a column for chromosome position. I am trying to filter file 1 based on each line's chromosome position in comparison to the chromosome positions in file 2. The ...
0
votes
4answers
44 views
Extract column starting with uppercase followed by next column starting with lower case
I have a file with multiple columns, separated by an underscore (_).
Most of the columns started with an uppercase for the first character, some with lower case.
I am intended to extract the string ...
0
votes
2answers
48 views
SSV/CSV Manipulation: Calculate Ratios
Note that I am aware of datamash and an experienced awk user. I am looking for something a bit simpler/streamlined than awk. Suppose I have the following:
// data_file
foo bar biz
10 100 1000
11 ...
0
votes
0answers
35 views
finding repetations on first column [duplicate]
I have pipe delimited text file like below
AAAAAAAAA|100.00|SALES|OK
BBBBBBBBB|250.00|ACCOUNT|OK
CCCCCCCCC|200.00|MARKETING|OK
DDDDDDDDD|175.00|ADMIN|OK
AAAAAAAAA|150.00|SALES|OK
BBBBBBBBB|200.00|...
0
votes
1answer
23 views
How to replace a positioned value in a file in Linux /Solaris
I have the file containing lines which are pipe separated like below.
45|abc|NULL|hol|
Gha|ghsk|NULL|gah|
How can I replace 3rd column values (Null) with something like a hyphen(-). The value in ...
0
votes
2answers
55 views
How can I produce serial no for repeating numbers
I have a pipe delimited text file like below
user |amount|dept|flag
AAAAAAAAA|100.00|SALES|OK
BBBBBBBBB|250.00|ACCOUNT|OK
CCCCCCCCC|200.00|MARKETING|OK
DDDDDDDDD|175.00|ADMIN|OK
AAAAAAAAA|150.00|...
4
votes
4answers
270 views
awk start printing based on a condition
I have test data in a file text.txt
a
b
test
test2
1,2
3,3
I want to output the file starting from the line number where test is + 2. I need this to be a oneliner usable in gnuplot, i have comeup ...
0
votes
2answers
55 views
Merge two columns of a gzipped file and replace the merged column with another column
I have a gzipped file which looks like this :
1 rs571093408 rs571093408 13380 C G
I want to merge the first and fourth column with : inbetween and replace the second column with the new content, so ...
0
votes
3answers
70 views
Compare data by different column
I have a file (tab separated) containing rows with a different number of columns. Such like this:
Bin_37:_Pelotomaculum_sp._DTU098 GH3 GH57 GH15 GH18 GT2 GT4 GT28 ...
0
votes
1answer
114 views
awk: Extracting characters one by one from a string
Someone often writes like this when extracting characters one by one from the beginning to the end of a string.
For example:
function rev1(str, i, ret) {
for (i = 1; i <= length(str); i++) ...
0
votes
3answers
56 views
loop over one or more matches in an awk column
The problem is that in one column of a csv file there is one or more email addresses. The output needs there to be one line for each email address. How do I do a for loop on the email addres to ...
1
vote
5answers
99 views
sed/awk: replace numbers in a line after last occurance of '.'
I have the following tcpdump stream:
Current:
07:36:03.848461 IP 172.17.3.41.33101 > 172.17.3.43.17408: UDP, length 44
07:36:03.848463 IP 172.17.3.42.33101 > 172.17.3.43.17409: UDP, length 44
...
0
votes
4answers
62 views
Split single line into multiple line until matches as pecific pattern
I'm trying to split every single line of data from c1.txt into multiple lines and copy it to new file c2.txt as shown in Expected result set. But ended up with no difference between c1.txt and c2.txt ...
