Questions tagged [gawk]

gawk is the GNU implementation of awk. The awk utility interprets a special-purpose programming language that makes it possible to handle simple data-reformatting jobs with just a few lines of code.

1
vote
3answers
80 views

If line contains pattern, print variable to column

I have the file DEMLIR-GEO_OPT-1-distance-1.coordLog: REQUESTED STRUCTURE DATA Distance vector r(i,j) between the atom i and j in ANGSTROM r(1,5) = 0.944776 0.190651 1....
1
vote
3answers
64 views

Organizing data by a header

I have two csv files. File 1 contains header. File 2 contains data file 1 format : file1.csv id,abc,xyz,aaa,bbb,ccc file 2 format : file2.csv id,source,value 1,abc,100 1,xyz,...
1
vote
1answer
20 views

(g)awk: Substituting expression from another script

I am building a generic script which will be re-used from several other scripts. For each case, I need to extract Nth field in the generic script; in some cases the double quotes should be removed ...
1
vote
1answer
192 views

Gawk: backspace preceding a newline doesn't work

I'm using gawk (4.1.3) and it seems I have struck upon a trivial problem. The backspace character doesn't seem to work if it is immediately followed by a newline. awk 'BEGIN{for(i=1;i<=3;i++) ...
1
vote
2answers
108 views

awk - processing two files

I have two files which I need to compare by some fields: reference file: 42:B:0 43:A:1 44:A:1 45:A:1 target file: 42:!:1 43:B:0 44:A:1 45:B:2 What I need I can already achieve with combined '...
1
vote
3answers
175 views

text processing (reading and computing from two files) [awk, script]

I have one text like the following: (words, word domains) car transport car machine bank economy bank politics bank parks God religion ... There are numerous words, some words have different domains ...
1
vote
2answers
1k views

how to use \n( and )\n as delimiter in awk

I have a file which contains below data 7[label = "ScanStep: T_b0\n(T.a = 1)\na\nb\nc\nd\ne\ndob\ntimestamp1\nUnc: Integer(8)\nUnc: Integer(8)\nUnc: Varchar(80)\nUnc: Numeric(10,2)\nUnc: Varbinary(80)...
1
vote
2answers
249 views

Save AWK FNR as a Bash variable to continue search from the line

I'm trying to get to run this on a sorted file. This returns the last line which is less than or equal to the given value comparing the first 9 characters. This would be done multiple times and VALUE ...
1
vote
1answer
17 views

awk printing out an associative array with OFS

I have the following awk code. It first determines the field numbers of interest in the header line and then it prints the output of those fields. The issue is that I am using an associative array to ...
1
vote
1answer
36 views

How to replace substrings with values taken from a dictionary indexed by the substrings

I need to parse a big file replacing possibly occurring substrings, that are part of possibly occurring strings that match a regex, with a value taken from an array whose indexes are the substrings in ...
1
vote
1answer
75 views

Using NF with FPAT – what am I doing wrong?

When NF is used with FPAT regex, the comma is considered a field. I prefer using NF and FPAT: 1) NF – to limit the output to the actual number of fields for the record 2) FPAT – to handle an ...
1
vote
1answer
97 views

Gawk partial match

My gawk script isn't matching all potential results, it's only matching one of them.. Example - script: gawk -F: "FNR==NR{a[$2]=$1;next} $1 in a{print a[$1] FS $2}" username.email.txt email.phone....
1
vote
2answers
49 views

gawk field seperator as extended ascii

I am using gawk 3.1.7 on RHEL 6.x I have large CSV files to parse with field seperator "Logical negation symbol / ALT + 170" (View ASCII table), but I could not use it as field seperator in gawk. I ...
1
vote
2answers
190 views

add and calculate percentage

I have two columns of data, I want add the number of repeating records in column A and then calculate the number of tallies in column B and then calculate the percentage of tallies. Example: 494 1 ...
1
vote
2answers
434 views

How to pipe sort to gawk?

I'm having trouble trying to pipe a sort with a gawk. I currently have: gawk -f p3a01.awk unsortedNames.txt | sort | gawk -f p3a01.awk this is however giving me the wrong results. The gawk seems to ...
1
vote
2answers
211 views

Using AWK to combine files

I have a folder with 179 tab-separated text files. They all have the same format. Here's a link to sample data: http://www.mvabl.com/public/sample_data.txt (This sample has the 5 headers, 5 lines of ...
1
vote
1answer
706 views

How can I replace the last match with a string

Given a regular expression and a replacement string, how can I use awk to implement the following: if a line contains no match, does nothing; if a line contains more than one matches, replace the ...
1
vote
1answer
141 views

awk - change value

File contains TIMEOUT=abc TIMEOUT=5 TIMEOUT= xTIMEOUT=5 I need to change from "TIMEOUT=5" to "TIMEOUT=9" This command works. But changes 4th line too, which is not desired awk '$0 ~ "TIMEOUT=...
1
vote
1answer
114 views

Combine range of columns with the same key

Input file tab separated: A |B | C | KEY | D| E| F |G | H | I | J | k | ----|----|----|------|---|---|-----|------|---|---|-----|-----| word|word|word| one | 8| 8|...
1
vote
1answer
174 views

Extracting trace information using gawk

I am using gawk to extract trace information from a mrt file in order to use it further for analysis. I have successfully been able to extract trace information from a pcap file format but cannot ...
1
vote
1answer
75 views

Improve performance when using “system” call (shell escape) processing large files in awk

I have an awk script that processes very large files that look something like this: K1353 SF3987.7PD833391.4 KARE K1353 SF3987.2KD832231.4 MEAKE K1332 IF4987.7RP832231.2 LEAOS K1329 SF2787.7KD362619....
1
vote
1answer
112 views

socat and gawk piped

I'm trying to manipulate a TCP stream using gawk. The TCP server streams out a text string separated by whitespaces. I want to add an extra string at the beginning and then separate the remaining ...
1
vote
1answer
31 views

Using GAWK to sum certain numbers line by line

My lines will look as such X 2.63766751981802 Y 1.68326762144808 Z 0 XX 2.63577810877244 YY 1.67475013544567 ZZ 0 Using GNU awk, how do I print (X-XX) + (Y-YY) on the end of every line? Just to ...
1
vote
0answers
445 views

strings concatenation on variable

how to collect add do existing "b" values next matching? input: 1:2:3:4:5 6:7:8:9:0 1:3:2:4:5 $ echo -e "1:2:3:4:5\n6:7:8:9:0\n1:3:2:4:5"|awk -F: '$1=="1"{b="line "NR" "$2;print b}' line 1 2 line 3 ...
1
vote
0answers
603 views

How to speed up awk script that uses several large gzip files?

I have two data files: file_1.in, containing over 2k lines like "12 AB0001": 10 AB0001 11 AC0002 12 AD0003 ... A list of *.gz gzipped files (about 1 to 3 million lines) I should extract and parse ...
1
vote
1answer
186 views

Gnu AWK split by pattern and insert columns , but get more delimiter

I have used same awk file to process two different string. So wired. cat test.awk BEGIN{FPAT="([^,])*|(\"[^\"]+\")";OFS=","}{$4="TDP,-1,-1,0,0"OFS$4;print $0} echo "a,b,b,b,b,b,b,b,b,b,b,b,\"a,b\"" ...
1
vote
0answers
185 views

How do split and patsplit work?

From Gawk's manual patsplit(string, array, fieldpat, seps) Divide string into pieces defined by fieldpat and store the pieces in array and the separator strings in the seps array. Is it ...
1
vote
1answer
113 views

Can a string be used as index in array of arrays in gawk?

Let's look at this file: 9=foo 3=bar 84=baz 30=bin 71=bon 9=goo 3=gar 84=gaz 30=gin 71=gon 9=soo 3=sar 84=saz 30=sin 71=son Running this gawk line: gawk ' { split($0,arr) for(i=1;i<=...
1
vote
1answer
67 views

Word frequency gawk memory leak

I have the following bash script: #!/usr/bin/env bash grep -e '^[a-zA-Z]\{4,8\}$' data/words3.txt | tr '[:upper:]' '[:lower:]' | sort -u > data/passphrase-words.txt function wordfrequency() { ...
1
vote
0answers
35 views

Selecting the columns from source file based on control file [duplicate]

I have requirement where I need to construct a file based on control file. The first row of my source file contains column names. The control file contains the columns which I need to output into a ...
1
vote
1answer
159 views

Find matches from an index file without exact matching and print the last field

I have a large file (~6mill rows) with 2 columns which looks like this: 1111.aaaaabbb.b.cccc.c ValueA 2222.dddddeee.e.ffff.f ValueB 3333.gggghhhh.h.iiii.i ValueC I want to use that as my index ...
0
votes
2answers
470 views

awk find lines with two matches

File contains Today is 1 nice day This is a perfect day Tomorrow is 1/2 cool 1/2 hot day What is Toy Story Tonight is a dark night How do I find lines starts with "To" and end with "day" None of ...
0
votes
3answers
1k views

Rounding a number in awk

I’m trying to find a way to round up a few numbers. The topic I found: https://stackoverflow.com/questions/2395284/round-a-divided-number-in-bash I am using the following command in my bash script.. ...
0
votes
2answers
1k views

patsplit vs split in gawk

In the gawk manual, I found two functions patsplit and split. I am wondering what their differences are? I understand the purpose of split, but what is the purpose of patsplit? Thanks.
0
votes
2answers
83 views

Search for a non exact match and return column 1 of that file

I have a tab seperated file which looks like this: Adam Group1,Group2,Group8,Group9,Group10 James Group1,Group11,Group12,Group7 Kelly Group18 Alfred Group1,Group8,Group10 Manuel Group2,...
0
votes
1answer
145 views

awk print from passwd but only with userid > 1000

Ok I've done: cat /etc/passwd | awk -F: '{ print $1 }' -F ==> Specify a file seperator for delineating data fields in a line $1 ==> Represents the first field in the input but how to only print ...
0
votes
3answers
98 views

resume reading a log file from the point I left it last time [duplicate]

I have a log file which is continuously updating(new line added) after few time period. I am fetching only error messages from the file in every 10 minutes. Initially, at 1st time I fetched all line ...
0
votes
1answer
88 views

processing multiple files with awk problem

I've the input files with fields separated with colon: main: one:111:222:333 fiv:333:222:333 two:123:234:500 file1: one:111:222:333 two:123:234:501 file2: one:111:222:333 thr:-:234:232 fiv:999:...
0
votes
1answer
33 views

setting more variables insite awk if ? :

I get error in such simple test with two variables: $ echo test|awk '{$2 != ""? o="ABC" r="123" : o="123 r="ABC"}END{print "o:"o" r:"r}' awk: cmd. line:1: {$2 != ""? o="ABC" r="123" : o="123 r="ABC"}...
0
votes
1answer
129 views

gnuwin bash If .txt file contains string copy file

okay so i know of grep -rhi, which extrats the line if string is found, but I want to copy the entire file so for example, file.txt > isexample #new iesameplxpele ieamama #new now if I used grep -...
0
votes
1answer
777 views

How to assign one variable per word of a string

How would you suggest to consecutively assign one variable per word of string? I am looking for the short command: one-liner and without loop if possible. It should have of the form echo "<sring&...
0
votes
2answers
165 views

Awk command failing during the redirection with null sting value

I want to split the large file into pieces. Filename and the number records per file after split will be provided as arguments/variables to awk. EG: FILENAME=A_20 (say it has 100 records), ...
0
votes
2answers
2k views

awk: How to use different field separators, and operate on different columns [closed]

For multiple files, I'm trying to extract columns containing a specific string, but the field separators in the files are different, and the string will appear in a different column depending on the ...
0
votes
2answers
207 views

filter date from a partiicular column (from the last 3rd column) of a csv file. Also if the last 3rd column is empty then print lines

Input csv file is 23,54,one,tree,56,23,7/20/2017 17:00,iii,89 23,54,one,tree,56,23,7/20/2017 17:00,iii,89 23,54,one,tree,56,23,7/21/2017 17:00,iii,89 23,54,one,tree,56,23,7/30/2017 17:00,iii,89 Need ...
0
votes
1answer
510 views

is a directory error

I'm having some problems running my current program. I need to take a file of customers information (each line is a new customer) and substitute it into a template. Each customer info will created a ...
0
votes
2answers
52 views

Why are the fields not separated by a whitespace in the output of this example?

When applying awk -F'[,/]' '{sub(/\/[0-9]+/, "/" ($2+3) ); print $0}' to input Statements and Functions/399,Black,notBold why is the output Statements and Functions/402,Black,notBold instead of ...
0
votes
2answers
104 views

Grep output concerns

Okay so I'm using grep on a external HDD example, M:/ grep -rhI "bananas" . > out.txt which would output any lines within " M:/ " containing " bananas " However I would like to output the ...
0
votes
2answers
66 views

Centos gawk command line usage

I am trying to run the following command on Centos however I get an error? Unexpected token near '(' gawk { if($0=="id323"){print $0} } doc.txt
0
votes
2answers
61 views

How to find a word and delete it with the following line

I've been using awk in an Expect script to edit a text file that contains information about a switch, so far the text file looks like this: Device ID Local Intrfce BIOTERIO Gig 1/0/6 N7K-LAN(...
0
votes
1answer
55 views

Awk script only works if i edit the text file manually

I'm using awk to organize the information in a text file (It contains CDP neighbors information of a switch taken from terminal) , the file looks like this: Device ID Local Intrfce BIOTERIO Gig 1/0/6 ...