awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
1answer
62 views

Awk that I am stuck on

I have a project I am working on for work. The script I am building initially takes two arguments: one is an unique Identifier and the other is for a log file (both strings). Now I have the first part ...
2
votes
3answers
103 views

Use awk results as parameters to another command

I'm extracting rows from a set of text files with awk. The files look like this: 1000 1 75 1000 2 76 1001 1 76 1001 2 80 I'm searching several directories of these with this ...
0
votes
3answers
115 views

Finding and replacing text inside awk block

I want to find and replace a value inside awk block. Here is a sample input file to my script P1 10,9:11/18013013582 ,10:1 ,167:0 ,487:5/E.164 11,9:15/310410532169026 ,10:60 ,167:0 ,487:4/IMSI ...
4
votes
3answers
553 views

awk filtering live capture with file as pattern list

What I'm trying to do is quite simple. I'm generating output from tshark and redirecting it to awk with a pipe |. Since tshark gets live data, I want that awk search in every output for a pattern ...
6
votes
8answers
2k views

Grep rest of line…after match

I have a file containing only two lines, with the following structure: $ cat /tmp/pwpower.log 000D6F0000D34227, -114.10 000D6F0001A405C4, -130.09 The values are power values of my solar plant. ...
3
votes
3answers
1k views

AWK: Passing shell variables to awk

I am trying to pass a variable number of arguments from the shell script to a pattern recognition subset of a table. Here is my attempt so far: The file 'infile': ID,GROUP 1,GROUP2 ...
4
votes
3answers
378 views

UNIX paste columns and insert zero's for all missing values

I would like to merge specific columns from two txt files containing varying number of rows, but same number of columns (as shown below): file1: xyz desc1 12 uvw desc2 55 pqr desc3 ...
1
vote
1answer
63 views

Move data row(s) to single column while retaining row header(s)

I have reports that are generated in the following tab-delimited format: UNIT TC CC PC TCP FTX FRX HOUSE 55 65 75 85 95 105 CAR 100 200 300 400 500 600 H2 ...
2
votes
3answers
7k views

awk for loop for each line in a file

I'm a newbie to bash scripting. I need help trying to figure out how to get loop through a text file and for every line take the fields of name , project #, and email, and replace them in a email ...
2
votes
2answers
190 views

How to print top five highest numbers from a column?

I have a text file with four columns. I need to read till end of file and print the highest number from column3 (top 5 values) along with column 1. input.txt xm|340034177|ref|RT_235820.1| 139697 ...
5
votes
2answers
281 views

How to use shell to derive an IPv6 address from a MAC address?

We know that we can use the MAC address to create an interface identifier, e.g. for a link-local IPv6 address which should be unique in the Network. The image shows the way to do this: My ...
-1
votes
3answers
123 views

AWK: Nested Conditional Subset of Rows without Headers [duplicate]

I have a pretty specific question, and have been able to find lots on conditional subsetting using awk but none that lends enough explicit code for me to generalize to my situation. I have a file ...
1
vote
2answers
309 views

printing special characters in awk output

I have a large file that contains lot MQ messages with RFH2 header. Each message in the file is separated by a blank line. Now I need to split this large file into small files each containing single ...
1
vote
1answer
160 views

AWK: Nested Conditional Subset of Rows

I have a pretty specific question, and have been able to find lots on conditional subsetting using awk but none that lends enough explicit code for me to generalize to my situation. I have a file ...
1
vote
5answers
461 views

Use SED or AWK to move string to new column and row

I have a .csv file that is tab delimited and I need to move the header row information to a "new" column at the beginning and a specified numbers of rows down. Please forgive any formatting errors for ...
3
votes
3answers
5k views

How to replace the content of specific column with awk?

Given there is 40 columns in a record. I want to replace 35th column so that the 35th column will be replace with the content of 35th column and a "$" symbol. What came up my mind is like: awk ...
2
votes
3answers
2k views

GREP / SED or AWK: Print entire paragraph in a file on pattern match

I have a file with hundreds of paragraphs of around 15 lines each. I need to search for a pattern, say Occurance: 1. If this pattern is found in the para, I need to print the entire paragraph. Note ...
0
votes
1answer
177 views

Search a string between two timestamps starting from bottom of file

I was trying to find a string Cannot proceed: the cube has no data in huge test.txt file only between timestamps yesterday 22:30 pm to today 00:30 am. Script: tac test.txt | awk -v today=$(date ...
1
vote
2answers
139 views

Compare set of values from two files and print the output in third file in awk

Hi friends i have problem with comparing two files.I'll try my best to explain.I have two file f1 and f2 which i have to compare on basis of two sets Parameter(p) and Value(v). Basically f1 contains ...
1
vote
3answers
51 views

Shell command to remove the first string that is enclosed with slashes

I need a shell command(sed or awk is preferred) to remove the string that is enclosed within the slashes. For example If my string is /some/dir/example/ then the output should be dir/example/
4
votes
3answers
2k views

awk command for parsing a file

I have a following text file. I am showing you the first 3 lines. chrom st end gene strand c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 c11 c12 c13 c14 chr6 3345 3543 geneA + 36 -23 ...
5
votes
2answers
134 views

How would I add a prefix to input I receive from a pipe via awk and color the prefix conditionally?

Is there a way to add a prefix to lines received (and filtered) via awk - yes I know that's possible so far - and then conditionally (e.g. based on the existence of the $PS1 variable of the ...
4
votes
2answers
5k views

Extract file name from path in awk program

I have an awk script and I have passed a CSV file to it. awk -f script.awk /home/abc/imp/asgd.csv What am I doing is to get FILENAME within script.awk. FILENAME gives me the whole path. As I am in ...
3
votes
2answers
175 views

how to parse a file that is changed continuously using awk?

for example if we have a file contains : Hello world Hello earth Hi everybody we can easily parse it to get the first and second columns of the lines contains the word Hello using awk: awk ...
0
votes
1answer
1k views

two .csv files compare using awk

I need to compare two files, File1.csv and File2.csv (Separated by ',') using Awk Logic: Column PID in both files are reference. If for same 'PID' in 'File1.csv' and 'File2.csv' and in both files ...
5
votes
4answers
544 views

how can we use awk to get multiple words after a significant word?

I want to connect to my DB only once to do multiple SELECT queries as follows: #!/bin/bash #Begin Code query=$(echo "SELECT COUNT(*) from USER_IPTable; SELECT User from USER_IPTable; ...
3
votes
3answers
433 views

How can I get the first column of a 300GB txt file?

let me first explain my problem in detail. It's actually pretty simple. I have a huge .txt file, 300GB to be more preciese, and I would like to put all the distinct strings from the first column, that ...
3
votes
3answers
263 views

How to keep record of previous operation results in awk?

I ran into another challenge that I am completely stumped on. I have a file filled with data in the following format. There are 3 lines that need to stay together and they have a predictable pattern: ...
4
votes
4answers
174 views

Combine fdisk and df in the same output

sudo fdisk -l gives : 255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size ...
5
votes
2answers
1k views

Use awk to insert a line after N output

I have a text file filled with a bunch of data. I need to pull out lines that contain specific strings. I accomplished this with awk using the following: awk '/pattern1|pattern2|pattern3/ {print;}' ...
3
votes
3answers
105 views

can I make awk cut its input according to '\n'?

we all know that we can get the columns of a line using awk, but the columns is split by white spaces: cat FileName | awk ' { print $1,$2,$3 .... }' but what if we want to split the files according ...
7
votes
3answers
5k views

What are the exact differences between awk and cut with grep? [closed]

We know that we can get the second column of the line we want from a file using these two techniques: awk '/WORD/ { print $2 }' filename or grep WORD filename| cut -f 2 -d ' ' My questions are: ...
1
vote
2answers
273 views

Compare two file and return a true or false value

I have two files with one column. If the first rows in the two files are same the print 1 otherwise print 0. file1: M A A M file2: M M A A output: 1 0 1 0 Can anybody tell me the awk script? ...
0
votes
3answers
81 views

Compare three file and print the highest value

I have three files. All of them having only one column. File1 M 0 M A File2 A 0 M A File3 M 0 A A compare each row in the three files, i.e, for first row (M,A,M) hence the maximum occurrence is ...
0
votes
4answers
149 views

compare columns in a one file and print the count of matches

I have a CSV file that contains two columns. I want to compare the two columns and print the number of matches. file1 M,M A,M A,A M,A Expected output: If the first columns in the first row is the ...
0
votes
4answers
618 views

Search for a Value only in one column w.o using awk, sed, or perl

I have a file that looks like this : Marketing Ranjit Singh FULLEagles Dean Johnson Marketing Ken Whillans FULLEagles Karen Thompson Sales Peter RobertsonPARTGolden ...
1
vote
3answers
84 views

awk one single line

hello i try to use a awk that compiles several lines into one, how can ia do that? i'm using the comand: awk < $pot/towhee.prod 'NR==411{print$5,$6}''NR==412{print$5,$6}''NR==430{print$7,$8}' ...
2
votes
1answer
69 views

Substite values from file1 to file2 awk

I have input file in xml FILE1 : <Sector sectorNumber="1"> <Cell cellNumber="1" cellCreated="YES" cellIdentity="" cellRange="35000" numberOfTxBranches="1" hsCodeResourceId="0"/> ...
2
votes
3answers
1k views

replace column of a csv file with column from another file

I have two sample files like this: $ cat file1 abc,sachin cat,kumar $ cat file2 xyz pressure $ cat file3 xyz,sachin pressure, kumar I want the first column of file1 to be replaced with file2. ...
3
votes
2answers
152 views

unix awk begin statement

I was trying to use a begin statement in awk, but somehow the begin statement is printed on top of every record instead of the very first record that I wanted. I am not very sure why, it would be ...
0
votes
1answer
129 views

Duplicate specific text in text file, using perl or grep

I need to duplicate part of a plain text file as follows: The text file consists of a sequence of "pages", of the form: <page [lots of lines of text] /page> <page [lots of ...
0
votes
4answers
94 views

Print text between tags (inclusive) if certain text is found

I've got a task to extract data from several Apache servers. The task is to print out: <Directory ...> ... </Directory> where +ExecCGI is located within. Let me give an example to ...
-3
votes
1answer
184 views

how to select specific range of line & count the specific occurence of the first charcter w.r.t. each unique second charcter? [closed]

Hi I have a file like this : #0 0:() 1b:cg*b 1c 0:cg xe #4 0:() 0b:cg*b xc 0:cg 1e #8 0:() 0b:cg*b xc 0:cg xe #12 1b:cg*b xc 0:cg 0e #16 xb:cg*b 1c xe #20 1:() xb:cg*b xc 1:cg 1e #24 x:() xb:cg*b xc ...
3
votes
2answers
89 views

how to grep complex hierarcy?

I have a file as:- start apple 1 a 2 b 3 c start orange 4 a 5 b start mango 1 a start a/b/c 5 z end 4 b end 6 c end start banana 3 c end 4 d 5 e end I want the output as :- 1 apple/a 2 apple/b 3 ...
4
votes
4answers
3k views

How to remove duplicate lines with awk whilst keeping empty lines?

Below awk command removes all duplicate lines as explained here: awk '!seen[$0]++' If the text contains empty lines, all but one empty line will be deleted. How can I keep all empty lines whilst ...
2
votes
3answers
2k views

How to copy or read the return value (exit status) from AWK script into the shell script (bash) to be used in if statement to compare

I want to read the error codes or the exit status of awk script into shell to use in if condition to compare for error handling my shell script looks something like: My script: #!/bin/sh awk -v ...
1
vote
3answers
3k views

Awk printf number in width and round it up

I need to printf a number out, but with given width and rounded (with awk!) %10s I have this and somehow I need to connect the %d but everything I do, ends up with too much parametres for awk ...
0
votes
2answers
182 views

Formatting numbers using awk / print

I found this very cool code to get upload/download speed: awk '{ if (l1) { print "↓"($2-l1)/1024"kB/s ","↑"($10-l2)/1024"kB/s" } else { l1=$2; l2=$10; } }' <(grep wlan0 ...
0
votes
1answer
69 views

change column with awk scriting

I am trying to make a script B that uses another script A to change a specific column in a specific line. The problem is that when I run the script A alone it runs perfectly but when I try to use the ...
3
votes
6answers
712 views

Add a symbol before any letter

I have a text file with each line having some defined number of fields but they can differ from line to line. All I want to do is to add "=" sign in front of every value in the field in that line. ...