awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

5
votes
2answers
101 views

How would I add a prefix to input I receive from a pipe via awk and color the prefix conditionally?

Is there a way to add a prefix to lines received (and filtered) via awk - yes I know that's possible so far - and then conditionally (e.g. based on the existence of the $PS1 variable of the ...
3
votes
2answers
353 views

Extract file name from path in awk program

I have an awk script and I have passed a CSV file to it. awk -f script.awk /home/abc/imp/asgd.csv What am I doing is to get FILENAME within script.awk. FILENAME gives me the whole path. As I am in ...
3
votes
2answers
121 views

how to parse a file that is changed continuously using awk?

for example if we have a file contains : Hello world Hello earth Hi everybody we can easily parse it to get the first and second columns of the lines contains the word Hello using awk: awk ...
0
votes
1answer
488 views

two .csv files compare using awk

I need to compare two files, File1.csv and File2.csv (Separated by ',') using Awk Logic: Column PID in both files are reference. If for same 'PID' in 'File1.csv' and 'File2.csv' and in both files ...
5
votes
4answers
327 views

how can we use awk to get multiple words after a significant word?

I want to connect to my DB only once to do multiple SELECT queries as follows: #!/bin/bash #Begin Code query=$(echo "SELECT COUNT(*) from USER_IPTable; SELECT User from USER_IPTable; ...
3
votes
3answers
132 views

How can I get the first column of a 300GB txt file?

let me first explain my problem in detail. It's actually pretty simple. I have a huge .txt file, 300GB to be more preciese, and I would like to put all the distinct strings from the first column, that ...
3
votes
3answers
101 views

How to keep record of previous operation results in awk?

I ran into another challenge that I am completely stumped on. I have a file filled with data in the following format. There are 3 lines that need to stay together and they have a predictable pattern: ...
4
votes
4answers
115 views

Combine fdisk and df in the same output

sudo fdisk -l gives : 255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size ...
5
votes
2answers
177 views

Use awk to insert a line after N output

I have a text file filled with a bunch of data. I need to pull out lines that contain specific strings. I accomplished this with awk using the following: awk '/pattern1|pattern2|pattern3/ {print;}' ...
3
votes
3answers
93 views

can I make awk cut its input according to '\n'?

we all know that we can get the columns of a line using awk, but the columns is split by white spaces: cat FileName | awk ' { print $1,$2,$3 .... }' but what if we want to split the files according ...
7
votes
3answers
813 views

What are the exact differences between awk and cut with grep? [closed]

We know that we can get the second column of the line we want from a file using these two techniques: awk '/WORD/ { print $2 }' filename or grep WORD filename| cut -f 2 -d ' ' My questions are: ...
1
vote
2answers
149 views

Compare two file and return a true or false value

I have two files with one column. If the first rows in the two files are same the print 1 otherwise print 0. file1: M A A M file2: M M A A output: 1 0 1 0 Can anybody tell me the awk script? ...
0
votes
3answers
64 views

Compare three file and print the highest value

I have three files. All of them having only one column. File1 M 0 M A File2 A 0 M A File3 M 0 A A compare each row in the three files, i.e, for first row (M,A,M) hence the maximum occurrence is ...
0
votes
4answers
54 views

compare columns in a one file and print the count of matches

I have a CSV file that contains two columns. I want to compare the two columns and print the number of matches. file1 M,M A,M A,A M,A Expected output: If the first columns in the first row is the ...
0
votes
4answers
192 views

Search for a Value only in one column w.o using awk, sed, or perl

I have a file that looks like this : Marketing Ranjit Singh FULLEagles Dean Johnson Marketing Ken Whillans FULLEagles Karen Thompson Sales Peter RobertsonPARTGolden ...
1
vote
3answers
57 views

awk one single line

hello i try to use a awk that compiles several lines into one, how can ia do that? i'm using the comand: awk < $pot/towhee.prod 'NR==411{print$5,$6}''NR==412{print$5,$6}''NR==430{print$7,$8}' ...
2
votes
1answer
61 views

Substite values from file1 to file2 awk

I have input file in xml FILE1 : <Sector sectorNumber="1"> <Cell cellNumber="1" cellCreated="YES" cellIdentity="" cellRange="35000" numberOfTxBranches="1" hsCodeResourceId="0"/> ...
2
votes
3answers
258 views

replace column of a csv file with column from another file

I have two sample files like this: $ cat file1 abc,sachin cat,kumar $ cat file2 xyz pressure $ cat file3 xyz,sachin pressure, kumar I want the first column of file1 to be replaced with file2. ...
3
votes
2answers
128 views

unix awk begin statement

I was trying to use a begin statement in awk, but somehow the begin statement is printed on top of every record instead of the very first record that I wanted. I am not very sure why, it would be ...
0
votes
1answer
85 views

Duplicate specific text in text file, using perl or grep

I need to duplicate part of a plain text file as follows: The text file consists of a sequence of "pages", of the form: <page [lots of lines of text] /page> <page [lots of ...
0
votes
4answers
69 views

Print text between tags (inclusive) if certain text is found

I've got a task to extract data from several Apache servers. The task is to print out: <Directory ...> ... </Directory> where +ExecCGI is located within. Let me give an example to ...
-3
votes
1answer
156 views

how to select specific range of line & count the specific occurence of the first charcter w.r.t. each unique second charcter? [closed]

Hi I have a file like this : #0 0:() 1b:cg*b 1c 0:cg xe #4 0:() 0b:cg*b xc 0:cg 1e #8 0:() 0b:cg*b xc 0:cg xe #12 1b:cg*b xc 0:cg 0e #16 xb:cg*b 1c xe #20 1:() xb:cg*b xc 1:cg 1e #24 x:() xb:cg*b xc ...
3
votes
2answers
80 views

how to grep complex hierarcy?

I have a file as:- start apple 1 a 2 b 3 c start orange 4 a 5 b start mango 1 a start a/b/c 5 z end 4 b end 6 c end start banana 3 c end 4 d 5 e end I want the output as :- 1 apple/a 2 apple/b 3 ...
4
votes
4answers
643 views

How to remove duplicate lines with awk whilst keeping empty lines?

Below awk command removes all duplicate lines as explained here: awk '!seen[$0]++' If the text contains empty lines, all but one empty line will be deleted. How can I keep all empty lines whilst ...
2
votes
3answers
594 views

How to copy or read the return value (exit status) from AWK script into the shell script (bash) to be used in if statement to compare

I want to read the error codes or the exit status of awk script into shell to use in if condition to compare for error handling my shell script looks something like: My script: #!/bin/sh awk -v ...
0
votes
2answers
199 views

Awk printf number in width and round it up

I need to printf a number out, but with given width and rounded (with awk!) %10s I have this and somehow I need to connect the %d but everything I do, ends up with too much parametres for awk ...
0
votes
2answers
76 views

Formatting numbers using awk / print

I found this very cool code to get upload/download speed: awk '{ if (l1) { print "↓"($2-l1)/1024"kB/s ","↑"($10-l2)/1024"kB/s" } else { l1=$2; l2=$10; } }' <(grep wlan0 ...
0
votes
1answer
53 views

change column with awk scriting

I am trying to make a script B that uses another script A to change a specific column in a specific line. The problem is that when I run the script A alone it runs perfectly but when I try to use the ...
3
votes
6answers
384 views

Add a symbol before any letter

I have a text file with each line having some defined number of fields but they can differ from line to line. All I want to do is to add "=" sign in front of every value in the field in that line. ...
1
vote
2answers
77 views

Renaming a whole folder content

I'm on OSX 10.9.2 (BASH 3.5) and I wanted to rename a whole folder content names. So I run a script and somehow ended with this: s01eDavid.Letterman.2014.02.27.Stupid.Human.Tricks.x264-SWOLLED.mp4 ...
1
vote
3answers
1k views

Delete lines from a file using awk or sed based on the variable from previous command

I have a CSV file (say temp.csv) with strings and numbers. I need to delete the lines which contain a particular pattern of string at the beginning of the lines in the file. For example, here is my ...
-2
votes
3answers
69 views

Can anyone please explain me below AWK script

format of flat file : ;metier_code ; ;-------------------------; (0 rows affected) ;CRDS_Ptf_No; ; ; ; ; ...
3
votes
1answer
211 views

Fastest `uniq` tool in linux

I have large text file (1.5 G), I want to know what the fastest and more reliable tool in linux is. I usualy use: awk '!x[$0]++' file.txt But when I use htop command I see my memory usage is ...
1
vote
1answer
39 views

Tile logfile into pieces and search for errors

I want to tile a logfile into pieces and then search in this pieces for errors. If there is an error, I want to put the relevant logfile-piece into another file. Little example: Logfile looks like ...
6
votes
6answers
90 views

How can I avoid escaped delimiter characters in a Unix command?

I need to take the sum of a field which is tilde (~) delimited. Problem I have is that my data is also having delimiter escaped. Example 1~CEO~ashok\~kumar~1000 As we see in 3rd field above we ...
1
vote
2answers
112 views

How can I format a flat file using awk?

Below is the format of some table data contained within a single file: ;NULL;ABCD;ABHJARS;;ABCD;ABCD;Y;;;;;;;;;A; ;NULL;XEU-ANKD;XEU-AJKD;;ABCD;ABCD;Y;;;;;;;;;A; . . ;11744;AMKDIONSKH;AMKDJ AN DJ ...
4
votes
4answers
563 views

remove blank lines in awk display

I was trying to figure out a solution for this question. I wanted to use awk for the solution. My input file is something like below. -bash-3.2$ cat file ramesh ramesh_venkat ramesh3_venkat3 ...
0
votes
1answer
71 views

Please explain the Awk script provided below

Can anyone please expain me step by step below AWK script written. I have the below code written in my script to format the flat file data. Just wanted to understand so that i can reuse -- I am not a ...
0
votes
2answers
72 views

Method to paste specific column when in terminal

I'd like to paste only the third column of data from within the terminal. I originally have something like: 4 SF0086 SF0086 12 SF0087 SF0087 17 TSF141 TSF141 29 TSF070 TSF070 30 TSF028 TSF028 31 ...
6
votes
3answers
520 views

Is there a perl or awk solution for this problem?

I have an input file (input.txt) like below. id1 id2 name weight 53723848 12651711 timburnes 1.36667 53530214 12651711 timburnes 1.51191 53723848 53530214 timburnes 1.94 764157 ...
-1
votes
1answer
56 views

Unix Awk group by

Can anyone shed me light how to group by based on the username and sum the values divide it by number of rows of that specific username. for example below. root 25 oracle 25 batch 30 griduser 20 ...
0
votes
2answers
134 views

what does getline do inside awk?

I have this following awk script that takes the following input file, input.txt and produces the output below. Can someone please take the time to break down how this awk script works? I've spent a ...
4
votes
7answers
323 views

How can I groups numbers in a file

I have a file with numbers in float format. I can review them via sort -rn numbers.txt | less I would like though to be able to "group" them. I.e. easily see how many are in the same range. To give an ...
1
vote
1answer
91 views

Using awk for data manipulation

I have an input file in the following format. #track chr11 61731756 61735132 FTH1 - chr12 6643584 6647537 GAPDH + chr11 18415935 18429765 LDHA + chr12 21788274 21810728 ...
2
votes
1answer
92 views

Pipe hex characters to awk

This command works as expected $ awk 'BEGIN {print "\x41"}' A As does this $ printf '\x41' | awk '{print}' A However when you rely on awk to recognize the hex characters, it fails $ echo '\x41' ...
1
vote
1answer
118 views

Parse/Manipulate in awk

I am new to awk programming. I have a question on manipulating text file, which is required to draw certain network based images in a visualization software (Circos http://circos.ca) I have input ...
10
votes
9answers
1k views

How can I print the longest number in a string?

I am looking for a method to print the longest number in a string. E.g.: If I have the string 212334123434test233 how can I print 212334123434 ? Note: I am looking for the longest continuous ...
1
vote
2answers
72 views

Parse,Manipulate Data In linux/Unix

I want to have exactly the same output in this post(http://unix.stackexchange.com/questions/126139/extract-data-in-linux-unix)as above,but I have a new input file,which is below: ABCB11 4 ACE 11 ...
1
vote
2answers
167 views

Extract data in linux/unix

I have input data which I want to parse and extract values using awk/grep/sed: group-2 9 10 text color=black,from=ABCB11,fromid=4,order=2,thickness=3,to=ACE,toid=11,use=1,z=1 group-3 0 1 text ...
-4
votes
1answer
115 views

Write a script file 'account.awk' that from a file 'orders' creates the formated printout of account

Write a script file account.awk that from a file orders creates the formatted printout of account. The file orders is as follows: http://pastebin.com/kPF5B0c0 Beside the column Cost the account ...