Questions tagged [awk]
A pattern-directed scanning and processing language.
5,831
questions
0
votes
4answers
76 views
Awk: Insert line after N lines following the pattern
I'd like to insert a line at the 3rd line following the pattern. Example: insert the word z 3 lines after each match of a
From
a
b1
c1
d1
a
b2
c2
d2
...
to
a
b1
c1
z
d1
a
b2
c2
z
d2
...
0
votes
1answer
24 views
In CSV file, Based on match pattern,fetch the other value matched pattern
Need your expert advice on this
AMOUNT,30,/AMOUNT,MESS,Am,/MESS,AMOUNT,30,/AMOUNT,TXN,209,/TXN
MESS,SU,/MESS,TXN,200,/TXN,AMOUNT,70,/AMOUNT
TXN,200,/TXN,AMOUNT,90,/AMOUNT,MESS,SUM,/MESS,...
0
votes
1answer
22 views
Duplicate columns of a bed file to produce another bed file
I have a bed file genome_cov.bed specifying the coverage of a genome at every position on the genome. The columns are; scaffold, position, coverage.
scaffold_1 1 0
scaffold_1 2 ...
0
votes
1answer
34 views
How to select rows from a file based on conditions of another file?
I have 2 genetic datasets, both have a column for chromosome position. I am trying to filter file 1 based on each line's chromosome position in comparison to the chromosome positions in file 2. The ...
0
votes
4answers
51 views
Extract column starting with uppercase followed by next column starting with lower case
I have a file with multiple columns, separated by an underscore (_).
Most of the columns started with an uppercase for the first character, some with lower case.
I am intended to extract the string ...
0
votes
2answers
55 views
SSV/CSV Manipulation: Calculate Ratios
Note that I am aware of datamash and an experienced awk user. I am looking for something a bit simpler/streamlined than awk. Suppose I have the following:
// data_file
foo bar biz
10 100 1000
11 ...
0
votes
0answers
35 views
finding repetations on first column [duplicate]
I have pipe delimited text file like below
AAAAAAAAA|100.00|SALES|OK
BBBBBBBBB|250.00|ACCOUNT|OK
CCCCCCCCC|200.00|MARKETING|OK
DDDDDDDDD|175.00|ADMIN|OK
AAAAAAAAA|150.00|SALES|OK
BBBBBBBBB|200.00|...
0
votes
1answer
23 views
How to replace a positioned value in a file in Linux /Solaris
I have the file containing lines which are pipe separated like below.
45|abc|NULL|hol|
Gha|ghsk|NULL|gah|
How can I replace 3rd column values (Null) with something like a hyphen(-). The value in ...
0
votes
3answers
66 views
How can I produce serial no for repeating numbers
I have a pipe delimited text file like below
user |amount|dept|flag
AAAAAAAAA|100.00|SALES|OK
BBBBBBBBB|250.00|ACCOUNT|OK
CCCCCCCCC|200.00|MARKETING|OK
DDDDDDDDD|175.00|ADMIN|OK
AAAAAAAAA|150.00|...
4
votes
4answers
277 views
awk start printing based on a condition
I have test data in a file text.txt
a
b
test
test2
1,2
3,3
I want to output the file starting from the line number where test is + 2. I need this to be a oneliner usable in gnuplot, i have comeup ...
0
votes
2answers
56 views
Merge two columns of a gzipped file and replace the merged column with another column
I have a gzipped file which looks like this :
1 rs571093408 rs571093408 13380 C G
I want to merge the first and fourth column with : inbetween and replace the second column with the new content, so ...
0
votes
3answers
73 views
Compare data by different column
I have a file (tab separated) containing rows with a different number of columns. Such like this:
Bin_37:_Pelotomaculum_sp._DTU098 GH3 GH57 GH15 GH18 GT2 GT4 GT28 ...
0
votes
1answer
120 views
awk: Extracting characters one by one from a string
Someone often writes like this when extracting characters one by one from the beginning to the end of a string.
For example:
function rev1(str, i, ret) {
for (i = 1; i <= length(str); i++) ...
0
votes
2answers
72 views
loop over one or more matches in an awk column
The problem is that in one column of a csv file there is one or more email addresses. The output needs there to be one line for each email address. How do I do a for loop on the email addres to ...
1
vote
5answers
107 views
sed/awk: replace numbers in a line after last occurance of '.'
I have the following tcpdump stream:
Current:
07:36:03.848461 IP 172.17.3.41.33101 > 172.17.3.43.17408: UDP, length 44
07:36:03.848463 IP 172.17.3.42.33101 > 172.17.3.43.17409: UDP, length 44
...
0
votes
4answers
62 views
Split single line into multiple line until matches as pecific pattern
I'm trying to split every single line of data from c1.txt into multiple lines and copy it to new file c2.txt as shown in Expected result set. But ended up with no difference between c1.txt and c2.txt ...
1
vote
1answer
29 views
sort a text file based on another file excepting header
I have a file in which the first and second column are individuals ids and the rest are genotype information (I should mention there are TWO rows per individual). this file has a header line but only ...
1
vote
2answers
162 views
How many decimal places could awk's rand() give you?
How many decimal places will rand() give?
I assume rand() value can't be any completely arbitrary number from 0 up to excluding 1 and that it's limited to a certain amount of decimal places or ...
-1
votes
2answers
64 views
How to echo color using AWK
I have this:
cm_env(){
env | grep 'cm_' | while read line; do
echo "$line" | awk -F'=' '{print $1 $2}'
done;
}
I want to print something like this:
env_var1=this_is_in_blue
env_var2=...
1
vote
2answers
50 views
How to find and count matching data for an ID column in two files?
I've got two genetic datasets which have matching chromosome position IDs. I want to count how many times file 1's chromosome position IDs appear in file 2.
For example my data looks like:
File 1 (...
4
votes
5answers
587 views
Using awk/sed to flip around some email address?
So I have a long list of email address and I would like to sort first by domain, so I would like to take a line like this:
email: vgnerov@yandex.ru
email: petebll@josephay905s.changeip.com
and get ...
0
votes
1answer
39 views
How Do I Select awk Columns Using Regex?
I have lots of java processes running on a server and since this is a dev server I have more than one env running.
If I want to list the processes for a particular environment I do this to give a ...
1
vote
3answers
61 views
Use awk to prepend the line with a specific field of the line
Using awk, I'm trying to print the fourth field of an input line ($4) also at the begining of the line.
I have tried the following code, but it prints the string above the line:
awk -F"[--]" '{print ...
-1
votes
1answer
51 views
matching a string and appending a new one to it
I have the following file:
NbV1Ch01 transdecoder gene 802292 803490 . + . ID=STRG.1;Name=ORF%20type%3A5prime_partial%20len%3A222%20%28%2B%29%2Cscore%3D55.19%...
3
votes
1answer
102 views
How to select and extract matching data using awk?
I have file 1 which only contains one column which is list of genes and file 2 which has a longer list of genes and the genes' statistical data. I am trying to find the statistical data which matches ...
0
votes
1answer
51 views
How to dynamically pass values in awk
I'm trying to split a csv file with too many rows into chunks of user defined no. of rows, using the following script using awk but its not generating any output, the file is created but its blank. ...
0
votes
2answers
42 views
Use awk to list all values seen in a column [duplicate]
I have a large data file with genetic information in it:
head Homo_sapiens.GRCh38.93.chr.onlyGene.gff3
1 ensembl gene 100000637 100000739 . - . ID=...
0
votes
3answers
125 views
bash script to get MAC address and paste it to ifconfig file after HWADDR
Is there a way to get the MAC address from ifconfig and append after HWADDR in ifconfig-eth0 file? I have used awk command to show MAC address but couldn't figure it out to hold that information and ...
12
votes
2answers
449 views
What is POSIX awk's stance on null byte in variables/printf?
Is it legal to print null bytes using awk's printf function according to POSIX? The POSIX standard of awk does not seem to explicitly mention it either way. Real world implementations differ in how ...
4
votes
5answers
387 views
Changing the format of two columns using AWK
I have the below data as input:
A 1,2
B 3,2,5
C 6,7
D 1,3,5,8
How can I get the below output using AWK?
A 1
A 2
B 3
B 2
B 5
C 6
C 7
D 1
D 3
D 5
D 8
1
vote
2answers
64 views
awk extract lines that matches specific range of numbers from multiple text files
For instance,
I have the following in a tab separated text file with two columns:
blah ABC_DE_23154_e53G_12
blah DEF_GH_23165_f35H_36
blah CED_BF_56412_c56T_21
I am using the following awk ...
1
vote
3answers
37 views
Bash - Filter rows with a certain proportion of columns occupied
So, I've got a large number of files, each one with 8 columns and a lot of rows. Here's a head from one of them for an example.
ID Ct 1 2 3 4 5 ...
3
votes
1answer
52 views
awk print pattern matched lines, and if a line with pattern1 has pattern2, then print line with pattern1 and nth line after as a single line
I currently have the output of a log, like the following.
20200124_075926.795633 [24_1859] [ERROR ] [PID] error running program:
...
...
actual error message from n lines below
...
20200124_075929....
1
vote
2answers
80 views
How to remove CRLF from awk $NF column
I'm not sure how to phrase the question as most of the answers are about removing \r\n from a file.
I have a unique problem where compressed files are numbered randomly and in order to associate them ...
2
votes
5answers
109 views
how to find common number from multiple file?
I want to extract common number present in all file. I have 1000 file in folder. I Want to compare all file number and find out common number in 1000 file. I have used below code:
for ((i=2;i<=...
0
votes
3answers
42 views
convert part of a line into hex
I have a file like so:
some line
some other line
DisplayChoices=SAMSUNG
yet another line
ChipManufacturer=LG
and yet another line
( there may be lots more of lines in the file, but the = sign is ...
3
votes
3answers
71 views
Print first word of the output
I've been looking around and haven't found what I'm trying.
I have to say I'm petty poor with grep, sed and awk though.
I have an alias:
alias upgradable='apt list --upgradable'
and it gets me ...
0
votes
1answer
43 views
Grep different kinds of patterns
I have a file that looks like this:
1>>>PROKKA_00001 Transcriptional regulator PadR-like family protein - 137 aa
>>tr|E5G076|E5G076_ARAHY Ara h 1 allergen OS=Arachis hyp (619 aa)
37.3%...
0
votes
1answer
30 views
replace strings in columns basing on another file values
input file:
col01 20083d70-1bf0-40f4-9df9-6247d587c085 text text
col02 d233d76d-e60d-442e-9b44-f0e1dc9ebefb text text
col03 20083d70-1bf0-40f4-9df9-6247d587c085 text text
col04 20083d70-1bf0-40f4-...
1
vote
1answer
50 views
bash bug - unexpected EOF while looking for matching `"' on tab completion
I know this question has been answered many times like here and here.
But all of them had places where there was an extra erroneous quote.
I am just running a simple awk script in this fashion:
awk ...
0
votes
2answers
44 views
Script to parse fields, changing format of the nth
I'm trying to take the contents of a pipe-delimited text file with a bunch of fields. Field 10 is a timestamp, and it either has the format yyyy-MM-dd hh:mm:ss, or yyyy-MM-dd hh:mm:ss.SSS. If it's the ...
0
votes
1answer
44 views
Why doesn't using a shell variable in awk work?
I'm trying to use awk to solve some problem, but I cannot understand why theese lines don't work.
name="root"
ps -eo user:20,rss | awk "{if(\$1==$name) print \$2}"
Generally in my system there are a ...
0
votes
1answer
70 views
Best way to substitute text
I have the following text in a file
/etc/pam.d/systemd-user|S.5....T.
/etc/bluetooth|not owned
/etc/pam.d/crond|S.5....T.
/etc/pam.d/crond|S.5....T.
/var/log/dmesg|.M.......
the text after the | ...
0
votes
3answers
157 views
awk - Print all remaining lines
I am trying to filter a log file using awk. Now the filtering is based on time, log file entries that are out of the time range are dropped and entries within the range are kept. Now once I encounter ...
-1
votes
2answers
46 views
Need to add Spaces and commas in specific places
Need your expert advice
Input:
2020-01-1713:57:5888823,aJ,MO,ZXC,adf
2020-01-1713:58:3878823,sJ,MO,AXC,Ldf
2020-01-1713:59:1898823,DJ,LO,AZC,NDf
Desired Output
2020-01-17 13:57:58,88823,...
2
votes
2answers
65 views
I want to extract particular column from the file, column number is given in different file using shell scripting?
I want to extract particular column from the file1 by comparing with file2 in which column number is given that should be extracted from file1.
File 1 (source data) look like this:
1 2 3 4 5 ...
2
votes
2answers
74 views
Extracting a json value from bash shell using awk without having to resort to both awk and sed
The aws command returns a json string to stdout inside a bash shell.
$ aws ssm get-parameter --name /mysite/dev/email
{
"Parameter": {
"Name": "/mysite/dev/email",
"Type": "...
-1
votes
1answer
32 views
Replace a value in particular position in a file
Can someone please help me here?
How do I replace 3 digit number value at poison 25 (25, 26, 27) from 478 to 645 in Unix script using AWK command. Thanks in advance.
0
votes
2answers
55 views
replace yaml field with content of file
I recently asked a question very similar to this one but failed to provide enough feedback for what would be a solution for this particular case. Opening this question with further information and ...
2
votes
3answers
348 views
BEGIN: command not found in a bare simple awk script when run via sourcing
I'm learning awk today, but I cannot succeed in having the most simple scripts to work.
#!/usr/bin/env -S awk -f
BEGIN { }
{ }
END { }
this outputs BEGIN: command not found
or even
#!/usr/bin/env -...
