Questions tagged [awk]

A pattern-directed scanning and processing language.

Filter by
Sorted by
Tagged with
0
votes
4answers
76 views

Awk: Insert line after N lines following the pattern

I'd like to insert a line at the 3rd line following the pattern. Example: insert the word z 3 lines after each match of a From a b1 c1 d1 a b2 c2 d2 ... to a b1 c1 z d1 a b2 c2 z d2 ...
0
votes
1answer
24 views

In CSV file, Based on match pattern,fetch the other value matched pattern

Need your expert advice on this AMOUNT,30,/AMOUNT,MESS,Am,/MESS,AMOUNT,30,/AMOUNT,TXN,209,/TXN MESS,SU,/MESS,TXN,200,/TXN,AMOUNT,70,/AMOUNT TXN,200,/TXN,AMOUNT,90,/AMOUNT,MESS,SUM,/MESS,...
0
votes
1answer
22 views

Duplicate columns of a bed file to produce another bed file

I have a bed file genome_cov.bed specifying the coverage of a genome at every position on the genome. The columns are; scaffold, position, coverage. scaffold_1 1 0 scaffold_1 2 ...
0
votes
1answer
34 views

How to select rows from a file based on conditions of another file?

I have 2 genetic datasets, both have a column for chromosome position. I am trying to filter file 1 based on each line's chromosome position in comparison to the chromosome positions in file 2. The ...
0
votes
4answers
51 views

Extract column starting with uppercase followed by next column starting with lower case

I have a file with multiple columns, separated by an underscore (_). Most of the columns started with an uppercase for the first character, some with lower case. I am intended to extract the string ...
0
votes
2answers
55 views

SSV/CSV Manipulation: Calculate Ratios

Note that I am aware of datamash and an experienced awk user. I am looking for something a bit simpler/streamlined than awk. Suppose I have the following: // data_file foo bar biz 10 100 1000 11 ...
0
votes
0answers
35 views

finding repetations on first column [duplicate]

I have pipe delimited text file like below AAAAAAAAA|100.00|SALES|OK BBBBBBBBB|250.00|ACCOUNT|OK CCCCCCCCC|200.00|MARKETING|OK DDDDDDDDD|175.00|ADMIN|OK AAAAAAAAA|150.00|SALES|OK BBBBBBBBB|200.00|...
0
votes
1answer
23 views

How to replace a positioned value in a file in Linux /Solaris

I have the file containing lines which are pipe separated like below. 45|abc|NULL|hol| Gha|ghsk|NULL|gah| How can I replace 3rd column values (Null) with something like a hyphen(-). The value in ...
0
votes
3answers
66 views

How can I produce serial no for repeating numbers

I have a pipe delimited text file like below user |amount|dept|flag AAAAAAAAA|100.00|SALES|OK BBBBBBBBB|250.00|ACCOUNT|OK CCCCCCCCC|200.00|MARKETING|OK DDDDDDDDD|175.00|ADMIN|OK AAAAAAAAA|150.00|...
4
votes
4answers
277 views

awk start printing based on a condition

I have test data in a file text.txt a b test test2 1,2 3,3 I want to output the file starting from the line number where test is + 2. I need this to be a oneliner usable in gnuplot, i have comeup ...
0
votes
2answers
56 views

Merge two columns of a gzipped file and replace the merged column with another column

I have a gzipped file which looks like this : 1 rs571093408 rs571093408 13380 C G I want to merge the first and fourth column with : inbetween and replace the second column with the new content, so ...
0
votes
3answers
73 views

Compare data by different column

I have a file (tab separated) containing rows with a different number of columns. Such like this: Bin_37:_Pelotomaculum_sp._DTU098 GH3 GH57 GH15 GH18 GT2 GT4 GT28 ...
0
votes
1answer
120 views

awk: Extracting characters one by one from a string

Someone often writes like this when extracting characters one by one from the beginning to the end of a string. For example: function rev1(str, i, ret) { for (i = 1; i <= length(str); i++) ...
0
votes
2answers
72 views

loop over one or more matches in an awk column

The problem is that in one column of a csv file there is one or more email addresses. The output needs there to be one line for each email address. How do I do a for loop on the email addres to ...
1
vote
5answers
107 views

sed/awk: replace numbers in a line after last occurance of '.'

I have the following tcpdump stream: Current: 07:36:03.848461 IP 172.17.3.41.33101 > 172.17.3.43.17408: UDP, length 44 07:36:03.848463 IP 172.17.3.42.33101 > 172.17.3.43.17409: UDP, length 44 ...
0
votes
4answers
62 views

Split single line into multiple line until matches as pecific pattern

I'm trying to split every single line of data from c1.txt into multiple lines and copy it to new file c2.txt as shown in Expected result set. But ended up with no difference between c1.txt and c2.txt ...
1
vote
1answer
29 views

sort a text file based on another file excepting header

I have a file in which the first and second column are individuals ids and the rest are genotype information (I should mention there are TWO rows per individual). this file has a header line but only ...
1
vote
2answers
162 views

How many decimal places could awk's rand() give you?

How many decimal places will rand() give? I assume rand() value can't be any completely arbitrary number from 0 up to excluding 1 and that it's limited to a certain amount of decimal places or ...
-1
votes
2answers
64 views

How to echo color using AWK

I have this: cm_env(){ env | grep 'cm_' | while read line; do echo "$line" | awk -F'=' '{print $1 $2}' done; } I want to print something like this: env_var1=this_is_in_blue env_var2=...
1
vote
2answers
50 views

How to find and count matching data for an ID column in two files?

I've got two genetic datasets which have matching chromosome position IDs. I want to count how many times file 1's chromosome position IDs appear in file 2. For example my data looks like: File 1 (...
4
votes
5answers
587 views

Using awk/sed to flip around some email address?

So I have a long list of email address and I would like to sort first by domain, so I would like to take a line like this: email: vgnerov@yandex.ru email: petebll@josephay905s.changeip.com and get ...
0
votes
1answer
39 views

How Do I Select awk Columns Using Regex?

I have lots of java processes running on a server and since this is a dev server I have more than one env running. If I want to list the processes for a particular environment I do this to give a ...
1
vote
3answers
61 views

Use awk to prepend the line with a specific field of the line

Using awk, I'm trying to print the fourth field of an input line ($4) also at the begining of the line. I have tried the following code, but it prints the string above the line: awk -F"[--]" '{print ...
-1
votes
1answer
51 views

matching a string and appending a new one to it

I have the following file: NbV1Ch01 transdecoder gene 802292 803490 . + . ID=STRG.1;Name=ORF%20type%3A5prime_partial%20len%3A222%20%28%2B%29%2Cscore%3D55.19%...
3
votes
1answer
102 views

How to select and extract matching data using awk?

I have file 1 which only contains one column which is list of genes and file 2 which has a longer list of genes and the genes' statistical data. I am trying to find the statistical data which matches ...
0
votes
1answer
51 views

How to dynamically pass values in awk

I'm trying to split a csv file with too many rows into chunks of user defined no. of rows, using the following script using awk but its not generating any output, the file is created but its blank. ...
0
votes
2answers
42 views

Use awk to list all values seen in a column [duplicate]

I have a large data file with genetic information in it: head Homo_sapiens.GRCh38.93.chr.onlyGene.gff3 1 ensembl gene 100000637 100000739 . - . ID=...
0
votes
3answers
125 views

bash script to get MAC address and paste it to ifconfig file after HWADDR

Is there a way to get the MAC address from ifconfig and append after HWADDR in ifconfig-eth0 file? I have used awk command to show MAC address but couldn't figure it out to hold that information and ...
12
votes
2answers
449 views

What is POSIX awk's stance on null byte in variables/printf?

Is it legal to print null bytes using awk's printf function according to POSIX? The POSIX standard of awk does not seem to explicitly mention it either way. Real world implementations differ in how ...
4
votes
5answers
387 views

Changing the format of two columns using AWK

I have the below data as input: A 1,2 B 3,2,5 C 6,7 D 1,3,5,8 How can I get the below output using AWK? A 1 A 2 B 3 B 2 B 5 C 6 C 7 D 1 D 3 D 5 D 8
1
vote
2answers
64 views

awk extract lines that matches specific range of numbers from multiple text files

For instance, I have the following in a tab separated text file with two columns: blah ABC_DE_23154_e53G_12 blah DEF_GH_23165_f35H_36 blah CED_BF_56412_c56T_21 I am using the following awk ...
1
vote
3answers
37 views

Bash - Filter rows with a certain proportion of columns occupied

So, I've got a large number of files, each one with 8 columns and a lot of rows. Here's a head from one of them for an example. ID Ct 1 2 3 4 5 ...
3
votes
1answer
52 views

awk print pattern matched lines, and if a line with pattern1 has pattern2, then print line with pattern1 and nth line after as a single line

I currently have the output of a log, like the following. 20200124_075926.795633 [24_1859] [ERROR ] [PID] error running program: ... ... actual error message from n lines below ... 20200124_075929....
1
vote
2answers
80 views

How to remove CRLF from awk $NF column

I'm not sure how to phrase the question as most of the answers are about removing \r\n from a file. I have a unique problem where compressed files are numbered randomly and in order to associate them ...
2
votes
5answers
109 views

how to find common number from multiple file?

I want to extract common number present in all file. I have 1000 file in folder. I Want to compare all file number and find out common number in 1000 file. I have used below code: for ((i=2;i<=...
0
votes
3answers
42 views

convert part of a line into hex

I have a file like so: some line some other line DisplayChoices=SAMSUNG yet another line ChipManufacturer=LG and yet another line ( there may be lots more of lines in the file, but the = sign is ...
3
votes
3answers
71 views

Print first word of the output

I've been looking around and haven't found what I'm trying. I have to say I'm petty poor with grep, sed and awk though. I have an alias: alias upgradable='apt list --upgradable' and it gets me ...
0
votes
1answer
43 views

Grep different kinds of patterns

I have a file that looks like this: 1>>>PROKKA_00001 Transcriptional regulator PadR-like family protein - 137 aa >>tr|E5G076|E5G076_ARAHY Ara h 1 allergen OS=Arachis hyp (619 aa) 37.3%...
0
votes
1answer
30 views

replace strings in columns basing on another file values

input file: col01 20083d70-1bf0-40f4-9df9-6247d587c085 text text col02 d233d76d-e60d-442e-9b44-f0e1dc9ebefb text text col03 20083d70-1bf0-40f4-9df9-6247d587c085 text text col04 20083d70-1bf0-40f4-...
1
vote
1answer
50 views

bash bug - unexpected EOF while looking for matching `"' on tab completion

I know this question has been answered many times like here and here. But all of them had places where there was an extra erroneous quote. I am just running a simple awk script in this fashion: awk ...
0
votes
2answers
44 views

Script to parse fields, changing format of the nth

I'm trying to take the contents of a pipe-delimited text file with a bunch of fields. Field 10 is a timestamp, and it either has the format yyyy-MM-dd hh:mm:ss, or yyyy-MM-dd hh:mm:ss.SSS. If it's the ...
0
votes
1answer
44 views

Why doesn't using a shell variable in awk work?

I'm trying to use awk to solve some problem, but I cannot understand why theese lines don't work. name="root" ps -eo user:20,rss | awk "{if(\$1==$name) print \$2}" Generally in my system there are a ...
0
votes
1answer
70 views

Best way to substitute text

I have the following text in a file /etc/pam.d/systemd-user|S.5....T. /etc/bluetooth|not owned /etc/pam.d/crond|S.5....T. /etc/pam.d/crond|S.5....T. /var/log/dmesg|.M....... the text after the | ...
0
votes
3answers
157 views

awk - Print all remaining lines

I am trying to filter a log file using awk. Now the filtering is based on time, log file entries that are out of the time range are dropped and entries within the range are kept. Now once I encounter ...
-1
votes
2answers
46 views

Need to add Spaces and commas in specific places

Need your expert advice Input: 2020-01-1713:57:5888823,aJ,MO,ZXC,adf 2020-01-1713:58:3878823,sJ,MO,AXC,Ldf 2020-01-1713:59:1898823,DJ,LO,AZC,NDf Desired Output 2020-01-17 13:57:58,88823,...
2
votes
2answers
65 views

I want to extract particular column from the file, column number is given in different file using shell scripting?

I want to extract particular column from the file1 by comparing with file2 in which column number is given that should be extracted from file1. File 1 (source data) look like this: 1 2 3 4 5 ...
2
votes
2answers
74 views

Extracting a json value from bash shell using awk without having to resort to both awk and sed

The aws command returns a json string to stdout inside a bash shell. $ aws ssm get-parameter --name /mysite/dev/email { "Parameter": { "Name": "/mysite/dev/email", "Type": "...
-1
votes
1answer
32 views

Replace a value in particular position in a file

Can someone please help me here? How do I replace 3 digit number value at poison 25 (25, 26, 27) from 478 to 645 in Unix script using AWK command. Thanks in advance.
0
votes
2answers
55 views

replace yaml field with content of file

I recently asked a question very similar to this one but failed to provide enough feedback for what would be a solution for this particular case. Opening this question with further information and ...
2
votes
3answers
348 views

BEGIN: command not found in a bare simple awk script when run via sourcing

I'm learning awk today, but I cannot succeed in having the most simple scripts to work. #!/usr/bin/env -S awk -f BEGIN { } { } END { } this outputs BEGIN: command not found or even #!/usr/bin/env -...