Questions tagged [gawk]

gawk is the GNU implementation of awk. The awk utility interprets a special-purpose programming language that makes it possible to handle simple data-reformatting jobs with just a few lines of code.

Filter by
Sorted by
Tagged with
1
vote
1answer
31 views

How to extract domain name from email address and add it to a new column in a pipe delimited file?

I have a pipe-delimited file with an email address in the third column. I need help to create the fourth column out of the domain name from the email address in the third column using awk or sed ...
6
votes
3answers
479 views

Extract string from each line of a file

I have a file where each line contains a sentence where one word is found between the character > and <. For example: Martin went shopping at >Wallmart< and lost his wallet French food >...
1
vote
2answers
22 views

In a line containing multiple fields, create a new line for each field, and remove that field from the old line

I have a file that looks like this: elephant kangaroo, snake, zebra baboon parrot, eagle, owl ... ... I want to change this list so that each line has only one element in it elephant kangaroo snake ...
1
vote
4answers
97 views

How to convert Syslog Date-Time [UTC] to Epoch?

I am currently piecing together a tool to work with Syslogs generated in my network, one of the requirements is to convert the DateTime from the format in which it is in syslog (%b %d %Y %T) to epoch. ...
2
votes
2answers
61 views

Merge multiple columns based on the first column values

I have multiple files (10+) that I want to merge/join into the one output file, for example: file 1 2000 0.0202094 2001 0.0225532 2002 0.02553 2003 0.0261099 2004 0.0280311 2005 0.028843 file 2 ...
3
votes
1answer
135 views

Why does this gawk coprocess hang?

While having a go at Convert date in bash shell, I tried GNU awk's coprocess feature: gawk -F, -v cmd='date +"%Y-%m-%d %H:%M:%S" -f-' '{print $5 |& cmd; cmd |& getline d; $5 = d}1' foo This ...
1
vote
1answer
26 views

AWK if loop - If any value is less than one

I produce a huge multi-column output file running some analytical software on angle populations. Sadly, the angles are defines as -180° to 180° and I need 0°-360°. What I want do to is take this ...
-1
votes
1answer
28 views

awk compare array to array on previous line

I would appreciate your help. I am extracting information from data files that are in some cases more than a terabyte in size. Variables on each line are are separated by whitespace. Quantity of ...
1
vote
1answer
17 views

awk printing out an associative array with OFS

I have the following awk code. It first determines the field numbers of interest in the header line and then it prints the output of those fields. The issue is that I am using an associative array to ...
1
vote
3answers
63 views

I want to know which word is duplicated on a line and how many times it appears in a file

example: I have file a.txt that contains the following: aunt bear bear true I also have a file b.txt that contains the following: tru tru high the end result I want is : for a.txt : bear x 2 for ...
1
vote
1answer
43 views

How to replace substrings with values taken from a dictionary indexed by the substrings

I need to parse a big file replacing possibly occurring substrings, that are part of possibly occurring strings that match a regex, with a value taken from an array whose indexes are the substrings in ...
2
votes
1answer
367 views

Is there any other method to uncomment in awk?

This command works perfect for me to uncomment a line. awk -i inplace '/# MyLINE/ { sub ("^ *#","") } { print }' file Just for curiosity, I am wondering if there are any other methods for "awk" to ...
0
votes
2answers
41 views

AWK - Need to display total value of field and field number

I have data below that needed to count or total the value start with specific field. Sample below the needed field to compute is from Field5. For sample Field5 total is 42, field6 total is 107, etc.. ...
2
votes
5answers
5k views

How do I append/prepend a timestamp to grep output?

I am running a small linux server at home, and I am writing a script to log the temperature of the CPU cores every 5 seconds, but I need timestamps for it to be useful. So far I have something that ...
2
votes
3answers
82 views

AWK - How to count stored or index on array

I have data that need to be count how many occurrences. 1 2 1 2 3 1 And used the awk command below and work properly. awk -F'","' '{ a[$1]++ } END { for (n in a) print n "=" a[n] } ' text.txt And ...
1
vote
1answer
80 views

Improve performance when using “system” call (shell escape) processing large files in awk

I have an awk script that processes very large files that look something like this: K1353 SF3987.7PD833391.4 KARE K1353 SF3987.2KD832231.4 MEAKE K1332 IF4987.7RP832231.2 LEAOS K1329 SF2787.7KD362619....
-1
votes
2answers
199 views

awk - replace pattern

how do I search the following pattern by awk in a file. '\b' The pattern is one of the lines in a file. I have to add text next to the pattern. Like this: '\b' TEXT Any of these commands are not ...
0
votes
1answer
31 views

Gawk cross referencing

I'm having a issue with cross referencing data using gawk, when I use: gawk -F: 'FNR==NR{a[$1]=$2;next} $2 in a{print $1 FS a[$2]}' email.phone.txt name.email.txt > name.phone.txt Example of ...
1
vote
3answers
82 views

If line contains pattern, print variable to column

I have the file DEMLIR-GEO_OPT-1-distance-1.coordLog: REQUESTED STRUCTURE DATA Distance vector r(i,j) between the atom i and j in ANGSTROM r(1,5) = 0.944776 0.190651 1....
0
votes
1answer
1k views

how to get multiple command output as variable in awk

I am trying to execute 3 command and get the output as a variable which needs some percentage calculation and display the same as the example. can this be executed as script. First command output $ ...
0
votes
0answers
20 views

Modifying text file using (g)awk [duplicate]

Is it possible to make a modification to a text file for example and save those modification to the same file using (g)awk script/command? All I see is only a modification to the printed output and ...
-1
votes
2answers
26 views

extract specific fields from a file using a character delimiter

I have a 1gb file and would like to get the third and fifth fields. The file is delimited by (,): input file: luis, jose, jesus atoche, mark, john 1,2,3,4,5 aa 1, bb 2, cc3, dd 4, ee5 output file: ...
0
votes
1answer
26 views

modify between (only) 1st occurrence of patterns (inclusive of patterns) and keep processing

There are several questions about matching between only the first occurrence of two patterns (e.g., this) — but they all seem to rely on exit, which is no good, as I want to keep processing the file. ...
1
vote
1answer
100 views

Concerning AWK programming, what do the -f switch and ~ operator do? [closed]

I can't seem to find anything on these switches. I know the "-F" option can be used to change the delimiter but that's the extent of what I've found other than it looks like maybe a compiling option; ...
0
votes
2answers
73 views

gawk - Dynamic creation of text file using input file fields

I have a text file that looks like this: UICEX_0001 UICEX_0001_T1.bam UICEX_0001_C2.bam chr1:16946335 chr19:9064309 chr8:10480278 ...
9
votes
4answers
1k views

awk assign to multiple variables at once

I'm trying to pull two numerical values out of a string and assign them to variables using awk (gawk is what I'm using specifically). I want to pull the major and minor version numbers out of a tmux ...
4
votes
2answers
274 views

GNU awk ERRNO not set on command failure

I was trying run a few commands using getline() function of GNU awk and print the error number (errno) value returned. But for simple failure cases of non existent directory/file the variable doesn't ...
1
vote
1answer
174 views

Extracting trace information using gawk

I am using gawk to extract trace information from a mrt file in order to use it further for analysis. I have successfully been able to extract trace information from a pcap file format but cannot ...
1
vote
1answer
82 views

Using NF with FPAT – what am I doing wrong?

When NF is used with FPAT regex, the comma is considered a field. I prefer using NF and FPAT: 1) NF – to limit the output to the actual number of fields for the record 2) FPAT – to handle an ...
5
votes
1answer
99 views

Why does gawk treat `0123` as a decimal number when coming from the input data?

According to $ man gawk, the strtonum() function can convert a string into a number: strtonum(str) Examine str, and return its numeric value. If str begins ...
6
votes
4answers
465 views

Why does regex `“\.pdf”` match `/…/pdf…/…` in gawk, but not in mawk?

From How can I extract only the pid column and only the pathname column in the lsof output? awk '{ for (i=9; i<=NF; i++) { if ($i ~ "string" && $1 != "wineserv" && $5 == "REG" &...
2
votes
4answers
132 views

How to break line after matching case but needs to go back a couple of characters

I have the following sample: 1. TNT 00:00 2. Swing From The Gutters 7:34 3. Ten-Day Interval 13:29 4. I Set My Face To The Hillside 18:16 5. The Equator 24:26 6. A Simple Way To Go Faster Than Light ...
1
vote
1answer
33 views

Using GAWK to sum certain numbers line by line

My lines will look as such X 2.63766751981802 Y 1.68326762144808 Z 0 XX 2.63577810877244 YY 1.67475013544567 ZZ 0 Using GNU awk, how do I print (X-XX) + (Y-YY) on the end of every line? Just to ...
3
votes
2answers
921 views

Combine two operations in one command - awk

I need to change - FROM Car Bus - TO Helicopter Airplane This two commands are sufficient. awk -i inplace '{sub(/Car/,"Helicopter")}1' file awk -i inplace '{sub(/Bus/,"Airplane")}1' file And ...
1
vote
1answer
72 views

Word frequency gawk memory leak

I have the following bash script: #!/usr/bin/env bash grep -e '^[a-zA-Z]\{4,8\}$' data/words3.txt | tr '[:upper:]' '[:lower:]' | sort -u > data/passphrase-words.txt function wordfrequency() { ...
1
vote
3answers
64 views

Organizing data by a header

I have two csv files. File 1 contains header. File 2 contains data file 1 format : file1.csv id,abc,xyz,aaa,bbb,ccc file 2 format : file2.csv id,source,value 1,abc,100 1,xyz,...
2
votes
3answers
2k views

How to get the last occurrence of lines between two patterns from a file?

I have a log file which reports on the output of a process, I'd like to extract all lines from between the last occurrence of two patterns. The patterns will be along the lines of; Summary process ...
0
votes
2answers
29 views

Using awk how to compare 2 fields and print something out

Let say I have file1 with the following content: 1,2,3,4,12,6 1,2,3,5,333,6 1,2,3,12,5,6 1,2,3,44,101,6 While file2 has the following content: 22,4,111,1,6,7,12 22,4,56,55,6,7,12 22,4,12,12,6,7,12 ...
3
votes
4answers
3k views

display the missing icmp_seq count | ping command

I am looking for something which shows me the missing ICMP package when running ping command or which can display failed in the output. Example below Current output where we are missing the icmp_seq ...
1
vote
2answers
104 views

Shell-Script Bash

I have this loop in the shell script, but it seems I don't fully understand what it does: especially gawk -v for ((k=$1; $k<3 + 3 ; k=$k + $3)) do echo $k | gawk -v s=4 '{print $1*s}' done ...
1
vote
1answer
168 views

Unexpected split behavior

I read about Awk split behavior here: [...] the fs argument to the split function (see String Functions) shall be interpreted as extended regular expressions. These can be either ERE tokens or ...
0
votes
2answers
67 views

Centos gawk command line usage

I am trying to run the following command on Centos however I get an error? Unexpected token near '(' gawk { if($0=="id323"){print $0} } doc.txt
0
votes
1answer
85 views

AWK ending standalone script

I started to do this.. to swap fields in a csv file but I don't get to finish the script. What I want to do is to concatenate fields from 2 to 7 and group them by the first field. Fields 1, 8 and 9 ...
0
votes
2answers
104 views

Grep output concerns

Okay so I'm using grep on a external HDD example, M:/ grep -rhI "bananas" . > out.txt which would output any lines within " M:/ " containing " bananas " However I would like to output the ...
-2
votes
2answers
53 views

Data Manipulation

I have a data as mentioned below. host_name Server1.domain.com contacts DL - Desktop contact_groups ravi, raj, rahim host_name Server2.domain.com contact_groups DL-Server host_name Server3....
0
votes
1answer
66 views

Unix file edit for pattern substitution

I have a file with repetitive patterns like below: set m [ meas new $proj rajesh ] meas $m set_type lvf_timing abcde erft 1213 234 456 897 00000000000000 once upon a time set t [ table new $m ...
-2
votes
1answer
72 views

Can I use awk to perform some number or string operations, without specifying a file as input?

Awk has builtin functions which are operating on numbers and strings. That is exactly what a programming language does. That makes me wonder if I can use awk to perform some number or string ...
1
vote
4answers
382 views

How shall I perform multiline matching and substitution using awk?

In a text file, ignoring any trailing whitespace at the end of each line, I assume that if a line is not ended with a digit, then there is a line break between the line and the next line, and I would ...
-3
votes
1answer
226 views

Are `match()`, `gensub()`, `gsub()` and `sub()` only concerned about records not fields?

In gawk, is it correct that match(), gensub(), gsub() and sub() don't consider the fields, and are not affected by change to FS and OFS? In other words, are they only concerned about records not ...
0
votes
3answers
1k views

Rounding a number in awk

I’m trying to find a way to round up a few numbers. The topic I found: https://stackoverflow.com/questions/2395284/round-a-divided-number-in-bash I am using the following command in my bash script.. ...