Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [gawk]

gawk is the GNU implementation of awk. The awk utility interprets a special-purpose programming language that makes it possible to handle simple data-reformatting jobs with just a few lines of code.

0
votes
2answers
73 views

gawk - Dynamic creation of text file using input file fields

I have a text file that looks like this: UICEX_0001 UICEX_0001_T1.bam UICEX_0001_C2.bam chr1:16946335 chr19:9064309 chr8:10480278 ...
0
votes
1answer
598 views

How to convert awk one-liner to standalone script? [closed]

How can i rewrite this awk one-liner as a standalone script? I want to improve readability to help me understand how it works. awk 'BEGIN{FS=OFS=";"} NR==FNR{for(i=2;i<=7;i++) if($i!="\"\"") a[$1]...
0
votes
1answer
122 views

Merging and appending tables to create a master table of multiple tables

How to we can merge and append the multiple tables in the same format having same as well as different data point and merge all of the together by inputting zero to the null data points. Desired input ...
0
votes
2answers
271 views

Passing timestamp variable from shell to gawk [duplicate]

I want to take time difference of two time stamp but getting an error while passing timestamp variable from shell toawk. Shell code: FTIMESTAMP="2015-07-01 12:30:50" gawk -v FTIMESTAMP=$FTIMESTAMP -...
0
votes
2answers
29 views

Using awk how to compare 2 fields and print something out

Let say I have file1 with the following content: 1,2,3,4,12,6 1,2,3,5,333,6 1,2,3,12,5,6 1,2,3,44,101,6 While file2 has the following content: 22,4,111,1,6,7,12 22,4,56,55,6,7,12 22,4,12,12,6,7,12 ...
0
votes
1answer
62 views

gawk job getting killed, too large of input?

I can't seem to understand why a script I've been running fine up until now fails on a slightly larger input file. If I run the script on a file that is 820000x75 (lines by columns), the script works ...
0
votes
1answer
31 views

trying to execute base62 operation in unix with more than 25 digits in a row, but awk treating a default value

Below is my code: #!/usr/bin/ksh awk 'BEGIN {FS="|"; flag=0;} { x=$2 rem=int(x)%62 quo=int(x/62) flag=0 while(quo>62) { sub_rem=int(quo)%62 quo=int(quo/62) if(flag==0) { grp_rem=sub_rem } else { ...
0
votes
2answers
88 views

check if no line contains string in given field [closed]

I need to search for thousands of strings one by one if they exists on a second column and if not do action against each such string which was not found in second column of any line file: line a ...
0
votes
1answer
425 views

awk and windows paths backslash escape character

Here I found a nice way to extract basename from path with awk: awk 'BEGIN{ var="Z:\201708021541\file name with spaces.123"; n=split(var,a,/\//); print a[n]}' but for backslashes it does not work as ...
0
votes
3answers
1k views

awk searching strings from file

I'm currently using $ awk 'NR==FNR{a[$1];next} ($3 in a)' find.txt path_to_100_files/* which uses find.txt strings to search multiple files within path_to_100_files/ for matches. find.txt ...
0
votes
2answers
2k views

Read mysql column result in awk variable

I am unable to read single VALUE from mysql Query in awk.I am able to read it in bash but not in awk. I want to read the result variable in awk as I want to fire the query only based on some dynamic ...
0
votes
1answer
382 views

How can I pass a regular expression constant as an argument to a function?

From gawk manual Because some built-in functions (e.g. match, split, sub) accept regexp constants as arguments, confusion can arise when attempting to use regexp constants as arguments to user-...
0
votes
3answers
265 views

GAWK using parenthesis as FS

I have been struggling with the workings of gawk when setting a regexp to FS as space-open_parenthersis OR closing_parenthesis-coma-space I have tried multiple approaches none with the desired ...
0
votes
1answer
480 views

Find out max number from given column from csv file

I have space-separated file with 4 columns. I want to make use of awk to find out max from each column. My awk script should output that information at end of file. e.g if my file is like this: ...
0
votes
2answers
41 views

AWK - Need to display total value of field and field number

I have data below that needed to count or total the value start with specific field. Sample below the needed field to compute is from Field5. For sample Field5 total is 42, field6 total is 107, etc.. ...
0
votes
1answer
30 views

Gawk cross referencing

I'm having a issue with cross referencing data using gawk, when I use: gawk -F: 'FNR==NR{a[$1]=$2;next} $2 in a{print $1 FS a[$2]}' email.phone.txt name.email.txt > name.phone.txt Example of ...
0
votes
0answers
20 views

Modifying text file using (g)awk [duplicate]

Is it possible to make a modification to a text file for example and save those modification to the same file using (g)awk script/command? All I see is only a modification to the printed output and ...
0
votes
1answer
26 views

modify between (only) 1st occurrence of patterns (inclusive of patterns) and keep processing

There are several questions about matching between only the first occurrence of two patterns (e.g., this) — but they all seem to rely on exit, which is no good, as I want to keep processing the file. ...
0
votes
1answer
26 views

How to join two files using two columns in unix

I have file a and file lookup and the structure is below. vi a.txt empid|ename|sal_grade|MANAGER_ID 1|raj|A|202 2|MAN|B|203 3|JOS|C|204 vi lookup.txt Grade|sal|manager_id A|$100000|202 A|$1000|099 ...
0
votes
1answer
83 views

gawk process multiple files inside directory

I have multiple text files in a folder that I want to process through gawk, rather than process them each individually is it possible I can process them all at once without merging them into 1 .txt ...
0
votes
1answer
38 views

two input files data processing

I have two huge input files: file1 contains 3 tabulator separated fields: field1: UID field2: username field3: Real Name Example of file1: 644 u11496 Real name1 640 u1309 Real name2 ...
0
votes
2answers
67 views

awk column spliting using delimitter

I want to split the first column into two, so that the output look like 00400000 00401000 I have an input file like this 00400000-00401000 r-xp 00000000 08:05 2762331 ...
0
votes
1answer
969 views

how to get multiple command output as variable in awk

I am trying to execute 3 command and get the output as a variable which needs some percentage calculation and display the same as the example. can this be executed as script. First command output $ ...
0
votes
1answer
38 views

Using gawk to alter columns after a line identifier

I have multiple drawing files in the format detailed below. I need to increase the value of a number in the area of the file below after the line BO. All values (in the sample below) that are 14.00 ...
0
votes
1answer
587 views

Matching lines from one file to lines in another file

I have a rather big list(1million or so) and another huge list(17gb), I need to match the lines in list1 with the first part of a delimited file 2 as such: List1: 98433259@34 90345394@43 94335053@23 ...
0
votes
2answers
478 views

Find and replace dot with text in column1 using gawk or awk

I have a csv as follows 1.2kbps,link1,description 1.2mbps,link2,description I would like to Find and replace a)dot and kbps with 000 in column1 b)dot and mbps with 000000 in column1 using gawk ...
0
votes
1answer
144 views

gawk on multiple files in bash

I am running this command in a command prompt gawk -f comparefirstandsecond.awk First.out Second.out I want to do it in bash shell script and I am trying it like: compareAwkOut=$(cat $...
0
votes
2answers
51 views

Print N number of observations that are non-missing from specific column in delimited file

Say I have file.txt that is pipe-delimited, and I want to take a look at 10 non-missing observations from the 20th column to ensure they appear in the correct format. Would I use the awk command and ...
0
votes
1answer
66 views

Unix file edit for pattern substitution

I have a file with repetitive patterns like below: set m [ meas new $proj rajesh ] meas $m set_type lvf_timing abcde erft 1213 234 456 897 00000000000000 once upon a time set t [ table new $m ...
0
votes
1answer
41 views

gawk columns from multiple files and add to a single text file

I have 50 sets of files containing 9 columns (sample shown in the attached picture). The files are named as ( 1) inputfile_1.assoc.logistic (2) inputfile_2.assoc.logistic etc…. The values in ...
0
votes
1answer
120 views

Changing specific column and retaining original format of PDB using gawk

I have a *.pdb file, which looks like this :- CRYST1 78.428 78.428 78.428 90.00 90.00 90.00 P 1 1 ATOM 1 CI1 JPIPP 1 41.874 78.965 24.633 1.00 0.00 PI C ...
0
votes
1answer
109 views

GAWK script explained

Can anybody explain me what does this script in each line? I can not run this because I think the syntax is incorrect. #! /bin/bash v=`echo $@|gawk '{print $NF}'` if[-d $v];then for v2 in $@;do if ...
0
votes
1answer
84 views

AWK ending standalone script

I started to do this.. to swap fields in a csv file but I don't get to finish the script. What I want to do is to concatenate fields from 2 to 7 and group them by the first field. Fields 1, 8 and 9 ...
0
votes
1answer
920 views

Multiple input files and output files in awk

I have a folder full of text files named 000.txt to 181.txt. How can I process all of them with the same awk script (program.awk) and send them to their respective output files (output000.txt - ...
-1
votes
3answers
67 views

Remove a whole line beginning with a pattern occurring anywhere after another pattern

How can I remove any line beginning with --- occurring in any line number after a line containing # Match (the first occurrence in any line number)? EDIT: The relationship between #Match with the ...
-1
votes
1answer
24 views

awk compare array to array on previous line

I would appreciate your help. I am extracting information from data files that are in some cases more than a terabyte in size. Variables on each line are are separated by whitespace. Quantity of ...
-1
votes
2answers
26 views

extract specific fields from a file using a character delimiter

I have a 1gb file and would like to get the third and fifth fields. The file is delimited by (,): input file: luis, jose, jesus atoche, mark, john 1,2,3,4,5 aa 1, bb 2, cc3, dd 4, ee5 output file: ...
-1
votes
3answers
146 views

Filter and save in-place, lines with value less than or equal ≤‎X in a specific column and with header

I have simulation files that I want to filter one column with only numbers that are less than 100. My files look like this: Time Node AppId SeqNo Type DelayS DelayUS RetxCount ...
-1
votes
2answers
195 views

awk - replace pattern

how do I search the following pattern by awk in a file. '\b' The pattern is one of the lines in a file. I have to add text next to the pattern. Like this: '\b' TEXT Any of these commands are not ...
-2
votes
2answers
128 views

How to delete everything after the 5th line with awk? [duplicate]

How to delete everything in a file after 5th line using awk? I can do it by sed, but these days I am moving from sed to awk.
-2
votes
1answer
65 views

Can I use awk to perform some number or string operations, without specifying a file as input?

Awk has builtin functions which are operating on numbers and strings. That is exactly what a programming language does. That makes me wonder if I can use awk to perform some number or string ...
-2
votes
2answers
53 views

Data Manipulation

I have a data as mentioned below. host_name Server1.domain.com contacts DL - Desktop contact_groups ravi, raj, rahim host_name Server2.domain.com contact_groups DL-Server host_name Server3....
-2
votes
1answer
61 views

Help reg to write gawk script [closed]

Need sample gawk to join two records with unique key $2.$1 has different entity eventhough $2 is key thats why there are ther are two different lines for same key I/p FIle not in sorted order 01|...
-2
votes
1answer
2k views

awk - how to remove extra space? [duplicate]

Using awk, how we can remove extra spaces, where it finds more than one space? We just need to keep one space between words, as usual. The file size is 3kb.
-2
votes
1answer
115 views

ipvsadm output processing

I need to process ipvsadm output which is similiar to this: IP Virtual Server version 0.2.12 (size=16384) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ...
-3
votes
3answers
2k views

How to write an awk script?

I am trying to figure out how to write a standalone awk script file. I thought it would be similar to a standalone bash script file: #! /usr/bin/awk -f BEGIN{ for (i = 0; i < ...
-3
votes
2answers
434 views

How to sort the values in one column based on other column values?

I want to sort the given tab separated data. first level is column #1 and is sorted in increasing order second level is column #2 is also sorted in increasing order, (but conditional); condition: ...
-3
votes
1answer
205 views

Are `match()`, `gensub()`, `gsub()` and `sub()` only concerned about records not fields?

In gawk, is it correct that match(), gensub(), gsub() and sub() don't consider the fields, and are not affected by change to FS and OFS? In other words, are they only concerned about records not ...
-3
votes
1answer
109 views

How Do I compare text in 2 files and export to file 3? [duplicate]

I have 2 files 1.txt aab aac aad 2.txt aaa aab aac File 3 should contain aab aac
-4
votes
1answer
652 views

Sort data in order based on values in one columns conditional upon another column [closed]

I have been trying to sort the given input data. And these are the conditions for sorting the data. strictly sort the column #1 in increasing order. now, within column #1 sort the data in column #2 ...