awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

3
votes
2answers
86 views

Check if conditional with awk for matches in logfile

I want to check the auth.log file for accepted sshd connections and execute an action if any are found. Here is my code: tail -f /var/log/auth.log | awk '{ if($0 ~ /sshd/ && $0 ~ /Accepted/) ...
0
votes
3answers
54 views

How do I pass a series of patterns into awk from a shell script?

I've got a series of patterns stored in foo.txt and I want to iterate over those patterns and see which match lines in file bar.txt. while read PWW do cat user/dump/bar.txt | awk '/${PWW}/{print ...
0
votes
1answer
36 views

How to integrate awk parameters in awk range pattern

I need to get something like this #Batch Job - AB1234 texts texts texts texts texts #-------- I used awk '/#Batch Job.*AB1234/,/#--.*/' filename and i went well. But then when I tried to ...
5
votes
3answers
154 views

Escape sequences passed as args to gawk not interpreted

I want to be able to pass an argument on the command line to gawk that is evaluated for escape sequences. The issue: $ gawk 'BEGIN { print ARGV[1]; }' '\t' \t Instead, I would like to get an ...
4
votes
2answers
41 views

How to preserve mixed linefeed formats in gawk?

I have an input file which contains both Unix (LF) and Windows (CR/LF) style newlines. (Specifically, it's XML from a Linux system, but it contains some raw HTTP headers, and HTTP prefers CRLF for ...
15
votes
2answers
1k views

How does awk '!a[$0]++' work?

This one-liner removes duplicate lines from text input without pre-sorting. For example: $ cat >f q w e w r $ awk '!a[$0]++' <f q w e r $ The original code I have found on the internets ...
3
votes
3answers
68 views

Output to stdout works, fails when redirected to a file or a fifo

I have a Wifi antenna and I want to optimise it's position. For that propose I'd like to visualise the current latency in real time. I wrote a one-line script to output the ping latency: lat.sh: ...
1
vote
1answer
41 views

Insert variable in specific place using sed

Please help me i'm trying to insert variable in a file with sed but has not yet successfully(( I have variable $time and i want insert it in index.html file in specific string(line number 53): the ...
15
votes
10answers
2k views

How can I quickly sum all numbers in a file?

Each line contains text and numbers in one column. I need to calculate the sum of the numbers in each row. How can I do that? Thx example.log contains: time=31sec time=192sec time=18sec time=543sec ...
4
votes
3answers
89 views

awk ifs and variables - cannot pass a variable from one line towards subsequent lines

First of all I'm new to awk so please excuse if it's something simple. I'm trying to generate a file that contains paths. I'm using for this an ls -LT listing as well as an awk script: This is an ...
0
votes
3answers
51 views

Extract value from one file and insert it into another file replacing some other value

I have two files, whose line numbers are not same. But in file1 I want to replace lines taking reference from file2 using sed command. In file1: tid.infno := 72 tid.setnr := 120 (This number 120 is ...
0
votes
1answer
52 views

Why doesn't this simple piping with awk work? [duplicate]

I'm trying to follow this answer but I can't make it work. I simplified it down as much as I could to try work out what's going on (just running this at the terminal for now): rsync --progress -iav ...
3
votes
6answers
176 views

grep with continuation lines

How can I grep/awk/sed a file looking for some pattern, and print the entire line (including continuation lines if the matched line ends with \? File foo.txt contains: something whatever thisXXX ...
2
votes
4answers
143 views

Using Grep -o or Sed/Awk to Grab Snippet from Middle of String

I am watching log traffic and see regular abuse of the various image search engines, Bing particularly. Example URL: ...
2
votes
3answers
54 views

Delete lines between 2 strings in Solaris

I want to delete all the line between two patterns using basic awk or sed. foo.txt : ---------------------------------------------------------------------- Cap in MB line 2 line 3 line 4 line 5 ...
5
votes
3answers
19k views

How to merge two files based on the matching of two columns?

I have file1 likes: 0 AFFX-SNP-000541 NA 0 AFFX-SNP-002255 NA 1 rs12103 0.6401 1 rs12103_1247494 0.696 1 rs12142199 0.7672 And a file2: 0 AFFX-SNP-000541 1 0 ...
14
votes
10answers
43k views

Show sum of file sizes in directory listing

The Windows dir directory listing command has a line at the end showing the total amount of space taken up by the files listed. For example, dir *.exe shows all the .exe files in the current ...
-3
votes
1answer
76 views

AWK sum every two rows

I want to sum two rows until end of file: 1 2 3 4 0 1 2 3 0 4 5 6 2 3 5 6 Output should be 1 3 5 7 2 7 10 12 I have tried the following, but it only works for the 1st row awk ...
0
votes
2answers
36 views

Issue on awk input parameter in ksh

I am running this script: INPUTNAME=ABC.XYZ FILENAME="Sample.xml" awk -v inputName=$INPUTNAME ' /<machine.*name=/ { f=1 ; m=0 ; res="" } f { res = res $0 ORS } f && ...
0
votes
1answer
45 views

How to implement awk range pattern in fetching an xml block when the input parameter is in the middle of the block

I am trying to fetch an xml block like this: <machine name="sample1" min="1" max="10" idleTime="300" backend="ABC,XYZ"> <handler className="com.abc.xyz.qwerty.foo.FooBar" ...
5
votes
6answers
358 views

sed command to swap characters

My input file layout is: mm/dd/yyyy,hh:mm,other fields I need to format it as: yyyy-mm-dd hh:mm:00,other fields sample input: 01/02/1998,09:30,0.4571,0.4613,0.4529,0.4592,6042175 ...
2
votes
1answer
47 views

How to output a comparison of values in separate files

With the following two files available with build names and version numbers available: server.txt Build2 200 Build3 100 Build1 400 Build7 200 local.txt Build1 300 Build2 200 Build3 100 In my ...
2
votes
2answers
205 views

Awk: match exact string in line

I have this file a deiauk Biking US 200 G b kespaul 202 A c deiauk NY 222 5 Z And I want to match the exact string 200 using awk. So my result should be a deiauk Biking US 200 G Here's my code ...
3
votes
7answers
145 views

Printing specific section of a line when a trigger value is present

I'm parsing data in the following format: prop1=value1:prop2=value2:prop3=value3+prop1=value4:prop2=value5 parts of the string are delimited by + properties can appear in any order the desired ...
2
votes
1answer
32 views

How is the order for which keys are stored and reteived determined in awk?

I am just starting to learn awk and was familiarizing myself with associative arrays only to discover that the order in which the keys were retrieved (or stored) didn't make a whole lot of sense. As ...
1
vote
3answers
133 views

Using AWK to convert numbers to 0 or 1

This might be a simple question but I can't figure it out for the life of me. I have a file called keyout.dat which looks like this: Title 1 Variable -999 -999 0.234 0.123 -999 -999 0.555 The file ...
2
votes
2answers
62 views

Awk: compare two files

I have a file1.txt USA Joe 123.123.123 Russia Marry 458.786.892 Canada Greg 151.844.165 Latvia Grace 125.895.688 and file2.txt 1 123.123.123 2 151.844.165 3 465.879.515 and I want to create a ...
2
votes
3answers
152 views

determining column number using string in awk

Suppose I have a pipe separated file like: |Sr|Fruits|Colors| |1 |apple |red| |2 |orange |orange |3 |grapes |purple| Here it is evident using awk that, $2 is Fruits and $3 is the colors column. ...
0
votes
3answers
100 views

Find all words beginning with Q

How do I search a text file and list all the words begining with Q? The file I am searching is Python source code. The closest I have found to a soulution is this question but that is not what I ...
1
vote
2answers
27 views

get specific variable using awk

I have this file.txt kespaul pts/14 2001:778:200:180 Fri May 29 11:33 - 16:40 (00:07) kespaul sshd 2001:778:200:180 Fri May 29 16:33 - 16:40 (00:07) algvai pts/2 ...
1
vote
1answer
43 views

finding lines in a file matching patterns defined in another file

I have two files file1.txt kepaul jack mike david and file2.txt 1 london smith 12564 2 miami jack 51616 3 la mike 165156 4 london kepaul 54654 5 miami grag 46546 and I need to print to a new ...
1
vote
2answers
47 views

Specify String Lenth in Awk Pattern Search

I'm not sure what I'm doing wrong. Given a test.log of: 123.123.123.123 321.321.321.321 GET /test1234/asdfasdfpioasdfjkhweris,9 123.123.123.123 321.321.321.321 GET ...
3
votes
2answers
132 views

how to print field between the double quote

I have the following file conf.txt: perl /home/site/gen.pl "GG - W C1 Test - sai1" "15072" && ssh 13.4.93.103 perl /home/site/gen.pl "GG - W C1 Test - sai2" "15072" && ssh ...
0
votes
0answers
34 views

awk search for string containing forward slash in variable [duplicate]

I'm searching for a string in a massive XML file. Snip: <sv:node sv:name="notfound"> <sv:property sv:name="jcr:primaryType" sv:type="Name"> ...
1
vote
1answer
129 views

How can I combine information of three files to create a fourth one?

I have three csv files, each of them containing the information I need to get a fourth one. I'm describing toy files but the actual ones contain much more columns and rows, still on the same models. ...
2
votes
2answers
57 views

Add a number in a huge ASCII file

Background: (1) Here is a screen capture of a part of my ascii file (over 600Mb): (1.1) Here is a part of the code: 0, 0, 0, 0, 0, 0, 0, 0, 3.043678e-05, 3.661498e-05, 2.070347e-05, ...
3
votes
3answers
86 views

Specific column level substitution

If I have a file with columnar entries, is it possible to substitute specific entries from a specific columns only and save changes to the same file?? Example a file abc.txt: SR|FRUITS|COLOR| ...
1
vote
2answers
5k views

Type URL and show HTTP status

I want to make a shell script where I type an url and it returns me the status code.So far I've tested wget is there any other cause it has the tendency to save whatever you run. Then I tried to some ...
0
votes
1answer
37 views

Move a range of words to the end of the string

I have the following string: "5 S comment=whatever Unspecific text fruit=apple animal=mouse sky=blue" I need to move to end of the string, all that begin with comment to the word text. The start ...
34
votes
13answers
53k views

Is there a way to get the min, max, median, and average of a list of numbers in a single command?

I have a list of numbers in a file, one per line. How can I get the minimum, maximum, median and average values? I want to use the results in a bash script. Although my immediate situation is for ...
2
votes
2answers
103 views

Delete blank columns

This must be a common problem, but I couldn't find a solution. I have several files which have different sets of columns which are completely blank (for all rows except header) , how can I delete ...
3
votes
1answer
491 views

What is an array in awk?

I'm very new to Unix and am trying to learn basic awk scripting - during which I keep coming across the word "array", and "associative arrays". I have no clue as to what this is; and don't tell me to ...
2
votes
4answers
218 views

Awk - Print row if number is between column 1 and column 2

I have a file with data formatted like this. Input file "1930","1946","FOO","BAR","GREEN" "1949","1962","FOO","BAR","BLUE" "1970","1980","FOO","BAR","YELLOW" Numbers are not continuous, with some ...
1
vote
3answers
64 views

Data extraction from text file

I have a text file in the following format:- $DATA1 SOURCE='HSPICE' VERSION='I-2013.12-SP2-1 32-BIT' .TITLE 'cmos inverter transfer characteristics' index t_r t_f ...
0
votes
2answers
59 views

Error in file comparison - awk [duplicate]

I have two files file1.txt, file2.txt. I want to select records which are common to both file1.txt and file2.txtbased on column 2 and column 3 and copy them to out.txt with additional column(column 4 ...
0
votes
1answer
53 views

How do I add a \n to the end of the record on file using AWK

I have a file that can contain 1 or many lines oracle1.txt oracle2.txt oracle3.txt How do I using AWK put a \n after the last line only so it will show oracle1.txt oracle2.txt oracle3.txt\n I ...
9
votes
9answers
22k views

How to print only last column?

echo -e 'one two three\nfour five six\nseven eight nine' one two three four five six seven eight nine how can I do some "MAGIC" do get this output?: three six nine UPDATE: I don't need it in this ...
0
votes
3answers
70 views

awk or sed or perl: remove only characters on specific position

I have multiple files to rename leng-1494-001 leng-1464-002 leng-2414-004 leng-7894-005 leng-1323-006 I want to rename it ferr-1494-001 ferr-1464-002 ferr-2414-004 ...
-1
votes
1answer
50 views

Webpage content - search for pattern & save to file

I have a web page, say https://test.salesforce.com. I would like to search for a pattern, e.g. "src" at the beginning of a line in the entire webpage content. If the pattern is found, I need to ...
0
votes
2answers
92 views

Print text between two patterns not containing a particular word

I wanted to print some text between two patterns which doesn't contain a particular word input text is HEADER asdf asd asd COW assd TAIL sdfsdfs HEADER asdf asd sdfsd DOG sdfsdfsdf TAIL sdfsdfs ...