Questions tagged [gawk]

gawk is the GNU implementation of awk. The awk utility interprets a special-purpose programming language that makes it possible to handle simple data-reformatting jobs with just a few lines of code.

Filter by
Sorted by
Tagged with
1
vote
1answer
30 views

Transpose columns into rows with first column data repeating using awk

I've the input in test.txt as month,var1,var2 June,10,a Sep,20,b Dec,30,c I want the output as month,variable,value June,var1,10 June,var2,a Sep,var1,20 Sep,var2,b Dec,var1,30 Dec,var2,c Note ...
1
vote
3answers
73 views

awk — replace once after immediate pattern in same line!

We have a 1.5MB file. One particular line has about 160000 characters. In the line we need to change "false" into "true" only once immediate after PATTERN. Following awk only changes first ...
0
votes
1answer
119 views

awk: Extracting characters one by one from a string

Someone often writes like this when extracting characters one by one from the beginning to the end of a string. For example: function rev1(str, i, ret) { for (i = 1; i <= length(str); i++) ...
0
votes
1answer
48 views

How to dynamically pass values in awk

I'm trying to split a csv file with too many rows into chunks of user defined no. of rows, using the following script using awk but its not generating any output, the file is created but its blank. ...
3
votes
2answers
65 views

Why does gawk (sometimes?) think 2.0e-318 > 2.0?

I am trying to find the maximum value of a column of data using gawk: gawk 'BEGIN{max=0} {if($1>0+max) max=$1} END {print max}' dataset.dat where dataset.dat looks like this: 2.0 2.0e-318 ...
0
votes
1answer
30 views

replace strings in columns basing on another file values

input file: col01 20083d70-1bf0-40f4-9df9-6247d587c085 text text col02 d233d76d-e60d-442e-9b44-f0e1dc9ebefb text text col03 20083d70-1bf0-40f4-9df9-6247d587c085 text text col04 20083d70-1bf0-40f4-...
-1
votes
2answers
45 views

Need to add Spaces and commas in specific places

Need your expert advice Input: 2020-01-1713:57:5888823,aJ,MO,ZXC,adf 2020-01-1713:58:3878823,sJ,MO,AXC,Ldf 2020-01-1713:59:1898823,DJ,LO,AZC,NDf Desired Output 2020-01-17 13:57:58,88823,...
-3
votes
1answer
58 views

Need to only show lines of a certain length and characters

I have a text file I'm working with and I want to only have lines that: Are greater than 8 characters long, and that contain an uppercase,lowercase, number, and special character. How would I do ...
1
vote
1answer
36 views

Why is it legal to have white space before builtin function in awk?

I am trying to understand how busybox's awk works so I'm looking into the standard and hit weird thing which I do not fully understand why is legal. Standard ( https://pubs.opengroup.org/onlinepubs/...
5
votes
1answer
497 views

Why does awk's system() call behave this way?

I'm trying to perform a grep inside awk using system() which according to the manual should return the exit code of the command being run. $ cat foo.txt bar $ grep -q bar foo.txt; echo $? 0 $ awk '...
1
vote
2answers
56 views

Merge rows using common values in any column

I have a tab-delimited file like shown below, and would like to merge the rows based on matches in any of the columns. The number of columns are usually 2, but could vary in some cases and be 3. ...
0
votes
1answer
24 views

strftime - ignore empty fields

input sample: one two 1578678600 awk command: $ echo -e "one\ntwo 1578678600"|awk '{print $1,strftime("%Y-%m-%d %H:%M:%S", $2)}' one 1970-01-01 00:00:00 two 2020-01-10 17:50:00 How strftime could ...
2
votes
2answers
43 views

add Value in file with value of output generated with script

I have a txt/log file which has data- cat sample.log/(.txt) 500,3,2 there is script running which overwrite the value in sample.log. Instead of overwrite/or generating a new line I want the data ...
1
vote
0answers
26 views

Issue in printing column with gawk [duplicate]

This is my data stored as Exist_eutran_freqs.csv: 120,1775 2984,1800 1942,2850 65,150 206,3050 8,1825 45,125 when I want to print them by below gawk command and switch the col1 and col2, I'm getting ...
0
votes
2answers
61 views

How to join 2 files based one key and selected some specific column?

I have 2 files and want to inner join them using awk. This is written using sql server : SELECT [file1.column1],[file2.column2] FROM file1 INNER JOIN file2 on file2.column1 = file1.column5; This is ...
1
vote
3answers
127 views

Awk partial string match

So I'm trying to partially match strings in one field and then use that field with another field from a seperate file, Example of inputs - input1.txt: example/world example/forever input2.txt ...
8
votes
1answer
213 views

Where is awklib?

I am working my way through GAWK: Effective AWK Programming: A User’s Guide for GNU Awk, Edition 5.0; March, 2019. I installed gawk using the following commands: sudo apt-get update sudo apt-get ...
-1
votes
1answer
28 views

awk's range is reset when working with multiple files

Let's say I have files 1.txt, 2.txt and 3.txt, all with the following contents respectively: foo1 Notes foo2 bar Now, I want to print out lines, which are placed after the pattern Notes till the ...
0
votes
1answer
68 views

Awk : extract the actual value of a RegExp pattern match

In the following awk code part, file contains a file name with its full Linux path that may include a directory of the type backup-YYMMDD where YYMMDD is a date. I would like to assign YYMMDD to ...
1
vote
1answer
22 views

BASH: Renaming and organizing files based on information in the filenames

Suppose I have a bunch of files has information separated by underscores: nickname_username_realname_date_filename.txt I want to make a directory for each username and rename the file to get rid of ...
12
votes
5answers
2k views

How could a sequence of random dates be generated, given year interval?

What is needed here is a command that generates six dates given a range of years (1987 to 2017). For example: 12/10/1987 30/04/1998 22/02/2014 17/08/2017 19/07/2011 14/05/2004 How it could be done, ...
1
vote
2answers
45 views

Appending first row time stamp to list below

Trying to take an input that looks like this: 08/22/2019 12:00:58 Name Cans Bucks Puns Clyde 12 2 79 Sheila 32 16 42 Elmo 44 18 21 08/23/2019 19:00:22 Name Cans Bucks Puns Clyde 18 21 46 Sheila 37 2 ...
1
vote
2answers
56 views

How to print the content of one of the columns related to the same user just using awk?

I have columns like these: User ColumnA ColumnB ComumnC abc123 hi hello howdy? xyz123 Namaste que paso rowdy? abc123 hi hello bowdy? I want to write the gawk script which ...
0
votes
2answers
35 views

Counting the number of fields in a column that are consecutively same

My input file looks like this: bio, john chem, john lang, john math, dina math, john math, matt lang, matt I want to count rows where the name is same consecutively only once. For this input, I want ...
0
votes
0answers
21 views

How to join all consecutive lines starting with pattern to the first line using sed and/or awk

I have a file; Section1 Target: RANDOM WORDS RANDOM WORDS RANDOM WORDS Box Number: Section2: Target: MORE RANDOM WORDS RANDON WORDS RANDOM WORDS Offset: I want to look like this: Sections1: ...
1
vote
1answer
37 views

setting delimiter in awk script

I have data seperated by either "space" or "space followed by new line" or "new line". I am writing awk script to post-process it. How do I define FS in this case ? This is what I have got so far ...
0
votes
0answers
45 views

how to merge two files to skip duplicate data

I have two different large files (each one more than 300,000 lines) and I want to combine them in a specific way. Some rows of the two data measure the same thing; when columns 9, 14, 15, 16, 17 are ...
2
votes
2answers
859 views

Is there any way to use the tr command in place?

I want to remove all non-english alphanumerals from a file. tr -sc '[:alnum:][:punct:]' ' ' <file gawk and sed have the -i flag but I cannot find anything similar in the tr man pages. How can I ...
4
votes
3answers
359 views

Running a command on first column using system() function on awk

I have the following code which I'm running to extract some ip addresses and count the unique of them. For example ./my_program | awk '/^10./{a[$1 FS $2]++ } END { for(i in a) print i,a[i]}' > ...
1
vote
3answers
222 views

How to extract domain name from email address and add it to a new column in a pipe delimited file?

I have a pipe-delimited file with an email address in the third column. I need help to create the fourth column out of the domain name from the email address in the third column using awk or sed ...
1
vote
3answers
36 views

In a line containing multiple fields, create a new line for each field, and remove that field from the old line

I have a file that looks like this: elephant kangaroo, snake, zebra baboon parrot, eagle, owl ... ... I want to change this list so that each line has only one element in it elephant kangaroo snake ...
6
votes
3answers
1k views

Extract string from each line of a file

I have a file where each line contains a sentence where one word is found between the character > and <. For example: Martin went shopping at >Wallmart< and lost his wallet French food >...
1
vote
4answers
533 views

How to convert Syslog Date-Time [UTC] to Epoch?

I am currently piecing together a tool to work with Syslogs generated in my network, one of the requirements is to convert the DateTime from the format in which it is in syslog (%b %d %Y %T) to epoch. ...
2
votes
2answers
406 views

Merge multiple columns based on the first column values

I have multiple files (10+) that I want to merge/join into the one output file, for example: file 1 2000 0.0202094 2001 0.0225532 2002 0.02553 2003 0.0261099 2004 0.0280311 2005 0.028843 file 2 ...
1
vote
1answer
214 views

AWK if loop - If any value is less than one

I produce a huge multi-column output file running some analytical software on angle populations. Sadly, the angles are defines as -180° to 180° and I need 0°-360°. What I want do to is take this ...
-1
votes
1answer
70 views

awk compare array to array on previous line

I would appreciate your help. I am extracting information from data files that are in some cases more than a terabyte in size. Variables on each line are are separated by whitespace. Quantity of ...
1
vote
1answer
74 views

awk printing out an associative array with OFS

I have the following awk code. It first determines the field numbers of interest in the header line and then it prints the output of those fields. The issue is that I am using an associative array to ...
1
vote
3answers
68 views

I want to know which word is duplicated on a line and how many times it appears in a file

example: I have file a.txt that contains the following: aunt bear bear true I also have a file b.txt that contains the following: tru tru high the end result I want is : for a.txt : bear x 2 for ...
1
vote
1answer
76 views

How to replace substrings with values taken from a dictionary indexed by the substrings

I need to parse a big file replacing possibly occurring substrings, that are part of possibly occurring strings that match a regex, with a value taken from an array whose indexes are the substrings in ...
0
votes
2answers
42 views

AWK - Need to display total value of field and field number

I have data below that needed to count or total the value start with specific field. Sample below the needed field to compute is from Field5. For sample Field5 total is 42, field6 total is 107, etc.. ...
2
votes
3answers
322 views

AWK - How to count stored or index on array

I have data that need to be count how many occurrences. 1 2 1 2 3 1 And used the awk command below and work properly. awk -F'","' '{ a[$1]++ } END { for (n in a) print n "=" a[n] } ' text.txt And ...
1
vote
1answer
106 views

Improve performance when using “system” call (shell escape) processing large files in awk

I have an awk script that processes very large files that look something like this: K1353 SF3987.7PD833391.4 KARE K1353 SF3987.2KD832231.4 MEAKE K1332 IF4987.7RP832231.2 LEAOS K1329 SF2787.7KD362619....
0
votes
1answer
103 views

Gawk cross referencing

I'm having a issue with cross referencing data using gawk, when I use: gawk -F: 'FNR==NR{a[$1]=$2;next} $2 in a{print $1 FS a[$2]}' email.phone.txt name.email.txt > name.phone.txt Example of ...
1
vote
3answers
102 views

If line contains pattern, print variable to column

I have the file DEMLIR-GEO_OPT-1-distance-1.coordLog: REQUESTED STRUCTURE DATA Distance vector r(i,j) between the atom i and j in ANGSTROM r(1,5) = 0.944776 0.190651 1....
0
votes
0answers
20 views

Modifying text file using (g)awk [duplicate]

Is it possible to make a modification to a text file for example and save those modification to the same file using (g)awk script/command? All I see is only a modification to the printed output and ...
-1
votes
2answers
33 views

extract specific fields from a file using a character delimiter

I have a 1gb file and would like to get the third and fifth fields. The file is delimited by (,): input file: luis, jose, jesus atoche, mark, john 1,2,3,4,5 aa 1, bb 2, cc3, dd 4, ee5 output file: ...
0
votes
1answer
32 views

modify between (only) 1st occurrence of patterns (inclusive of patterns) and keep processing

There are several questions about matching between only the first occurrence of two patterns (e.g., this) — but they all seem to rely on exit, which is no good, as I want to keep processing the file. ...
9
votes
4answers
3k views

awk assign to multiple variables at once

I'm trying to pull two numerical values out of a string and assign them to variables using awk (gawk is what I'm using specifically). I want to pull the major and minor version numbers out of a tmux ...
4
votes
2answers
343 views

GNU awk ERRNO not set on command failure

I was trying run a few commands using getline() function of GNU awk and print the error number (errno) value returned. But for simple failure cases of non existent directory/file the variable doesn't ...
2
votes
1answer
200 views

Using NF with FPAT – what am I doing wrong?

When NF is used with FPAT regex, the comma is considered a field. I prefer using NF and FPAT: 1) NF – to limit the output to the actual number of fields for the record 2) FPAT – to handle an ...

1 2 3 4 5