Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [gawk]

gawk is the GNU implementation of awk. The awk utility interprets a special-purpose programming language that makes it possible to handle simple data-reformatting jobs with just a few lines of code.

48
votes
1answer
31k views

Difference between gawk vs. awk

Trying to understand the differences between the two functions gawk vs. awk? When would one use gawk vs awk? Or are they the same in terms of usage? Also, could one provide an example?
30
votes
3answers
63k views

Skip the first 6 lines/rows in a text file with awk

How can I skip the first 6 lines/rows in a text file (input.txt) and process the rest with awk? The format of my awk script (program.awk) is: BEGIN { } { process here } END { } My text file is ...
9
votes
4answers
843 views

awk assign to multiple variables at once

I'm trying to pull two numerical values out of a string and assign them to variables using awk (gawk is what I'm using specifically). I want to pull the major and minor version numbers out of a tmux ...
9
votes
2answers
690 views

gawk inplace and stdout

Is it possible to use gawk's -i inplace option and also print things to stdout? For example, if I wanted to update a file, and if there are any changes print the name of the file and the changed ...
8
votes
3answers
758 views

Separator between statements in awk

From Gawk's manual: When awk statements within one rule are short, you might want to put more than one of them on a line. This is accomplished by separating the statements with a semicolon (‘;’)...
8
votes
1answer
3k views

awk condition of true and false

I came across that if we use awk 0 inputfile, it will not print anything cause 0 means false of the condition. If we use awk 1 inputfile, it will print everything as 1 means true for each line awk ...
8
votes
2answers
377 views

Handling 3 Files using awk

Consider following files: file1: boo,8,1024 foo,7,2048 file2: foo,0,24,154 noo,0,10,561 file3: 24,154,7,1024,0 What I need is to go to File1 and check if $2==7; if true, take $1, $2 and $3 ...
7
votes
4answers
436 views

Why does regex `“\.pdf”` match `/…/pdf…/…` in gawk, but not in mawk?

From How can I extract only the pid column and only the pathname column in the lsof output? awk '{ for (i=9; i<=NF; i++) { if ($i ~ "string" && $1 != "wineserv" && $5 == "REG" &...
7
votes
3answers
2k views

Surprising result using awk floating point arithmetic

I've been trying to get awk to do some trivial arithmetic, which involves carrying some values from one line to the next. Here is a minimal example pair, for comparison. The first example is expected ...
7
votes
2answers
3k views

Batch rename image files by age plus add date and variable to filename

The question is completely rewritten due to things learned from the first two answers and comments that I was unaware of when first asking. After a photo shoot I come home with files that look like ...
7
votes
1answer
149 views

Regression with unary plus

Previously, numeric conversions could be triggered by either adding zero (+0), unary plus (+) or unary minus (-): $ gawk --version GNU Awk 4.1.4, API: 1.1 (GNU MPFR 3.1.6-p1, GNU MP 6.1.2) ...
7
votes
1answer
270 views

Make bash exit != 0 when called by AWK and interrupted with ^C

I'm having an issue with the following piece of (G)AWK script: do { ... } while (system("sleep 10")) My intention is to break the loop when the user presses ^C during the sleep, but it's not ...
7
votes
3answers
2k views

String replacement using a dictionary

What is a good way to do string replacements in a file using a dictionary with a lot of substituend-substituent pairs? And by a lot, I actually mean about 20 – not much, but many enough so that I want ...
6
votes
6answers
12k views

Convert string into date time stamp in gawk or awk

I want to convert following string (20140805234656) into date time stamp (2014-08-05 23:46:56).I am new to gawk and I don't know the exact syntax,how can I put - at every 5,8 and : at every 14,17 and ...
6
votes
4answers
415 views

How to find the number of unordered pairs from a list

I want to count the number of unordered pairs from a list. In other words, I have a list: ab ba ac bc and I want to display: 2 ab 1 ac 1 bc
5
votes
3answers
898 views

Extracting data with awk when some lines have empty/missing values

I have a sample as shown here: input.txt USERS position ref rslt usr1 X B usr2 2980 C usr3 3323 P ...
5
votes
1answer
910 views

How to use awk to print only lines containing 5 columns

I have a file which contains pipe delimited data. I want to print the lines which have only 5 columns. I tried the below. It does not work. It simply prints all the lines. Could you please let me know ...
5
votes
1answer
170 views

Why can't a constant regular expression be put on the left side of an ~ operator in gawk?

Why can't I put a regular expression on the left side of the ~ operator when using gawk? For example, given the file below with fields delimited with tabs(\t): $ cat cats siberian 1970 73 ...
5
votes
3answers
2k views

The ERE regex to split() string between a delimiter and end-of-word

I'm using a long gawk 3.1.6 script to do a complex conversion of Zim markdown text into GtkDialog code and am stuck on the following problem... Sample ASCII input... [[link|label label]] [[link]] @...
5
votes
1answer
91 views

Why does gawk treat `0123` as a decimal number when coming from the input data?

According to $ man gawk, the strtonum() function can convert a string into a number: strtonum(str) Examine str, and return its numeric value. If str begins ...
5
votes
3answers
771 views

Escape sequences passed as args to gawk not interpreted

I want to be able to pass an argument on the command line to gawk that is evaluated for escape sequences. The issue: $ gawk 'BEGIN { print ARGV[1]; }' '\t' \t Instead, I would like to get an actual ...
4
votes
4answers
18k views

Current date in awk

Is there a constant variable in awk, that store today's date? if no, is there a way that can store today's date for daily use? let's say we have below file: boo,foo,2016-08-30 foo,boo,2016-07-31 ...
4
votes
1answer
951 views

Speed up text processing

I have the following script to process a file with some data: First, the header is printed to the output file. Then 60000 lines from the input are randomly picked and printed to the output (the ...
4
votes
2answers
257 views

GNU awk ERRNO not set on command failure

I was trying run a few commands using getline() function of GNU awk and print the error number (errno) value returned. But for simple failure cases of non existent directory/file the variable doesn't ...
4
votes
3answers
1k views

Grep strings with random numeric characters

I want to grep lines containing - 0 amount - 122,000,000 amount - 50,000 amount I've tried to use grep -rhI " - * amount ". but it will output anything containing - regardless of whether ...
4
votes
2answers
945 views

How to preserve mixed linefeed formats in gawk?

I have an input file which contains both Unix (LF) and Windows (CR/LF) style newlines. (Specifically, it's XML from a Linux system, but it contains some raw HTTP headers, and HTTP prefers CRLF for ...
4
votes
4answers
381 views

How to create a new file with required columns from different multiple files in linux?

I have a directory ballgown, in which there are around 1000 subdirectories as sample names. Each subdirectory has a file t_data.ctab. The filename is the same in all subdirectories. ballgown |...
4
votes
4answers
3k views

display the missing icmp_seq count | ping command

I am looking for something which shows me the missing ICMP package when running ping command or which can display failed in the output. Example below Current output where we are missing the icmp_seq ...
3
votes
4answers
3k views

Can I determine type of an awk variable?

I have the gawk version of awk. In this part of gawk manual, it is stated that awk variables have "attributes", which are used to determine how to treat them in various operations. For example, a ...
3
votes
4answers
771 views

Conditional if clause in awk

Please consider below file: foo,1000,boo,A foo,1000,boo,B foo,1001,boo,B foo,1002,boo,D And we have below rules: If $2 equal 1000, $4 should be equal A If $2 equal 1001, $4 should be equal B If $...
3
votes
4answers
10k views

how to print quote character in awk

I have a file str.txt with the following sample records. 31,2713810299,1,11-Aug-15 19:52:10 32,2713810833,1,11-Aug-15 21:36:18 Now I want to print output with awk as below. cat str.txt|awk -F, '{...
3
votes
3answers
476 views

How to repeat a sequence of numbers in a column using awk command

My input file contains data like this. //address: 9050004 data: 1 id: 1 len: 0 size: 2 //address: 9050008 data: 1 id: 2 len: 0 size: 2 //address: 902e004 data: 1 id: 3 len: 0 size: 2 ... //address: ...
3
votes
3answers
531 views

Replace the data in a column in a file with the data in a row from another file

I'm new working with linux enviroment and i need to write a script to process some data. I have a file that looks like this: File A: 1 X 1 1.0 2 X 1 1.0 3 X 1 1.0 4 Y 2 ...
3
votes
3answers
889 views

Output to stdout works, fails when redirected to a file or a fifo

I have a Wifi antenna and I want to optimise it's position. For that propose I'd like to visualise the current latency in real time. I wrote a one-line script to output the ping latency: lat.sh: ...
3
votes
1answer
267 views

Clone complex array in awk

I have an array like this ... a["a"] = "aa" a["b"] = "ab" a["c"] = "ac" a["d"]["a"] = "ada" a["d"]["b"]["a"] = "adba" a["e"]["a"]["a"]["a"] ...
3
votes
1answer
362 views

awk - dynamically format tab-separated columns

I have a file with dynamic length columns (four) separated with tabs (a column can have have spaces) COL1 COL2 COL2 COL2 COL3 COL3 COL4 COL1 COL2 COL2 COL3 COL4 COL4 COL1 COL1 ...
3
votes
4answers
417 views

Convert a column in-place on CSV

I need some help on date conversion in-place on a CSV. Basically, I was capable to convert the column and save in a new file, but I was unable to save it on the original CSV. I have a CSV with date ...
3
votes
1answer
417 views

Split out individual characters using the null string

I read this in the Gawk manual: GNU EXTENSIONS [...] The ability to split out individual characters using the null string as the value of FS, and as the third argument to split(). ...
3
votes
2answers
630 views

Portable check for array

Gawk has "isarray": if (isarray(x)) print "is array" else print "is scalar" However Mawk and "gawk --posix" do not: fatal: function 'isarray' not defined This can cause problems: x x[1] fatal:...
3
votes
1answer
564 views

how to send output of curl request to 2 separate commands

I'm piping the output of a curl request to gawk so that I can pull some data. The gawk code is already working if I'm searching through the html file which curl request produces. However, I was ...
3
votes
1answer
190 views

Problem with regular expression in gawk ('\<' not working)

I am trying to get into bash a little bit. I was going through this guide but the second example in this subsection on gawk doesn't seem to work. The problem is this command: ls -l | awk '/\<(a|x)...
3
votes
2answers
981 views

Why awk says “syntax error” for the comma I placed between the two patterns?

I run the following script under Linux (Mint on a USB key) with GNU Awk 4.0.1 installed by default. Error returned by awk is the following : awk: cmd. line:7: /^Observation/,/^@@@/ {...
3
votes
1answer
201 views

awk to split the input file into 2 files(one for duplicated values and one for unique values)

I have an input file like below. file.txt 14 2.692022 author1 0 14 2.57826 author2 1 229 2 author3 2 323 2 author4 3 514 2.0625 author5 4 I am trying to split the file with the duplicated values as ...
3
votes
1answer
1k views

How to find all matches to a regular expression in a string

In POSIX awk and Gawk respectively, how can we find all the matches to a regular expression in a string? More specifically, find all the matches that are substituted by gsub builtin function, in ...
3
votes
2answers
817 views

Combine two operations in one command - awk

I need to change - FROM Car Bus - TO Helicopter Airplane This two commands are sufficient. awk -i inplace '{sub(/Car/,"Helicopter")}1' file awk -i inplace '{sub(/Bus/,"Airplane")}1' file And ...
3
votes
1answer
1k views

Using awk to process a mapping file with dynamic number of columns

I am wondering how to process information as following: mapping.txt: 80 001 002 81 011 012 013 014 82 021 022 ... input.txt: 81 103823044 80 103823054 81 103823064 ... Desired output.txt: ...
3
votes
2answers
586 views

How to exclude lines that don't include a string from the output of an awk script?

Completely new to scripting in general, and have been working on a script to go through our logs which contain entries for a variety of different things. What I'm interested in, is the entries that ...
3
votes
0answers
112 views

Why does this gawk coprocess hang?

While having a go at Convert date in bash shell, I tried GNU awk's coprocess feature: gawk -F, -v cmd='date +"%Y-%m-%d %H:%M:%S" -f-' '{print $5 |& cmd; cmd |& getline d; $5 = d}1' foo This ...
2
votes
6answers
349 views

Find Maximum of all columns based on distinct first column

I am using Ubuntu and I have an input file like this ifile.dat 1 10 15 3 34 20 1 4 22 3 32 33 5 3 46 2 2 98 4 20 100 3 13 23 4 50 65 1 40 76 2 20 22 How do I ...
2
votes
3answers
275 views

AWK/GAWK adding character after pattern is matched

I have the following data structure in my test file: "111","222","AAABBB","333","444","555" I want to transform the third field so there is a '-' after the 3rd [A-Z] like so: "111","222","AAA-BBB","...