awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

3
votes
1answer
188 views

Using awk to process a mapping file with dynamic number of columns

I'm a complete newbie to awk and am wondering how to process information as following: mapping.txt: 80 001 002 81 011 012 013 014 82 021 022 ... input.txt: 81 103823044 80 103823054 81 ...
3
votes
5answers
445 views

Case-insensitive count duplicate line, remove duplicate by choose case with highest duplicate

The duplicate is combination of different case text. I need to count number of duplicate (case-insensitive) and then I need to remove duplicate by choose case with highest duplicate. Below example: ...
3
votes
1answer
85 views

Sending signals to long-running processes (AWK, DD)

I'm familiar that DD command doesn't produce output while running. However, if I send a USR1 signal to it, it can print current state. Can I do the same with AWK? Prints its variables, register ...
3
votes
1answer
33 views

Match pattern, do calculation and print only 3 or 4 floating values max

I would like to convert below to GB and print only 3 floating values v copvol2 fsgen ENABLED 204800 - ACTIVE - - v copvol3 fsgen ENABLED 204800 - ...
0
votes
2answers
49 views

Unable to perform a simple multiplication in script

I am havingf trouble performing a simple multiplication in my script. while read A B C do tmp=$A\*$C/100 echo $tmp >> out1.txt done < foo.txt foo.txt: 13721725 99 100 ...
2
votes
4answers
222 views

Change text in tag, but only tag is contained within a certain XML block

Using Git Bash, I'm trying to conditionally replace what's in the yrot tag in hundreds of files, but only if it belongs with a part name tag that pertains to wheel. // YES, change <part ...
2
votes
5answers
163 views

Adjust gap between 2 columns to make them look straight

file1.txt: hi wonderful amazing sorry superman superhumanwith loss file2.txt : 1 2 3 4 5 6 7 When i try to combine using paste -d" " file1.txt file2.txt > actualout.txt actualout.txt : hi 1 ...
2
votes
1answer
101 views

Unset all ENV variables matching _PROXY

I'm trying to unset all environment variables that match _PROXY: env | grep -i _proxy | column -t -s '=' | awk '{ print $1 }' | grep -iv 'no_proxy' | xargs -0 -I variable unset variable but it's ...
0
votes
2answers
71 views

How to find and replace only a particular word (which is at the end of the file) using shell command?

File1: Dms 01.01.2015 feeder1 6kv close at 04:30 Updated Dms 01.01.2015 feeder1 6kv open at 06:20 Updated Dms 04.02.2015 feeder10 6kv close at 17:23 Updated Dms 04.02.2015 feeder12 6kv ...
4
votes
2answers
360 views

how to extract parts of file into separate files in linux using awk or grep or sed commands

I have text file as shown below: Input file1.txt ....................... cha21 1 3 5 nar cha21 2 3 6 piy cha23 2 3 5 ram cha23 3 ...
6
votes
8answers
7k views

How do I delete the first n lines and last line of a file using shell commands?

I have a file named Element_query containing the result of a query : SQL> select count (*) from element; [Output of the query which I want to keep in my file] SQL> spool off; I want to ...
1
vote
1answer
100 views

How to write data one by one from tmp files to the final output file using for loop?

I have two tmp files named tmp1 and tmp2 which contains some lines. tmp1 file, 1c\ datafile no. 23 2c\ datafile is ok tmp2 file, 3c\ datafile no. 24 4c\ datafile is ok I have a ...
0
votes
1answer
44 views

How to remove certain number of lines every time particular string is found in file

I want to remove 1 previous line and 10 following lines every time a string was found in the file. foo.txt: } Name: john Line 3 Line 4 Line 5 Line 6 Line 7 Line 8 Line 9 Line 10 Line 11 Line 12 ...
1
vote
2answers
107 views

How to print few lines from middle of a file using unix shell commands? [closed]

My file containing the following lines, $BQ { VOL @home } database daba relation tcdeatid opendb clear .lruno := 72 <-- (This line lruno. := 72 has 10 times ...
2
votes
3answers
88 views

Truncating the precision of a date

I'm working with a bash script which polls build progress from several different systems. If a build is complete, it stats a log file to find out the last time it was edited, then feeds that time into ...
1
vote
2answers
32 views

Addition of values

foo.txt: Lillypaul_sg 204803 204803 204803 Ammy_sg 3 ramaswamy_sg 3 33 tommy_sg 3 3 137374 100 Desired Out.txt : Lillypaul_sg 614409 Ammy_sg 3 ramaswamy_sg 36 tommy_sg 137480 The output is a sum ...
1
vote
3answers
650 views

Sed Replace at a specific group of positions

I have a sample input file as below: Apple Orange Gold Silver Spoon Apple Orange Gold Apple Apple Orange Apple Orange Mango Gold ...
3
votes
2answers
148 views

shell script to extract between characters

I would like to extract a user name from the file between the keywords "/" and "NET" like below. How can I do so with programs like awk, sed or cut. from: audit: command=true ...
3
votes
4answers
754 views

Extract paragraph separated with *** using AWK

I have a file like below: blablabla blablabla *** thingsIwantToRead1 thingsIwantToRead2 thingsIwantToRead3 blablabla blablabla I want to extract the paragraph with thingsIwantToRead. When I had to ...
1
vote
2answers
132 views

Count frequency of specific numbers in a text file of scientific notations

Background I have a text file, named blood_conc.txt as shown: 0, 0, 0, 0, 0, 0, 0, 1.32006590271e-05, 1.990014992001e-05, 1.504668143682e-05, 2.176900659261e-06, 7.673488970859e-06, ...
3
votes
2answers
1k views

Split log file by time range

I have a log file and would like to cut it into copies of 15min. In the log file the lines start with: dd mmm yyyy hh:mm:ss,xxx e.g. 12 Feb 2013 16:05:02,xxx log text... and the file rotates ...
-2
votes
4answers
110 views

How to print last few contents in a line?

I have a file that has lines shown below, ta=/*92/*39/*1/*649/*12 ta=/*92/*12/*4/*342/*90 ta=/*92/*33/*9/*455/*3 ta=/*92/*12/*3/*649/*25 ta=/*92/*99/*7/*649/*7 and so on. I want a file that will ...
3
votes
2answers
171 views

Check if conditional with awk for matches in logfile

I want to check the auth.log file for accepted sshd connections and execute an action if any are found. Here is my code: tail -f /var/log/auth.log | awk '{ if($0 ~ /sshd/ && $0 ~ /Accepted/) ...
0
votes
3answers
110 views

How do I pass a series of patterns into awk from a shell script?

I've got a series of patterns stored in foo.txt and I want to iterate over those patterns and see which match lines in file bar.txt. while read PWW do cat user/dump/bar.txt | awk '/${PWW}/{print ...
0
votes
1answer
44 views

How to integrate awk parameters in awk range pattern

I need to get something like this #Batch Job - AB1234 texts texts texts texts texts #-------- I used awk '/#Batch Job.*AB1234/,/#--.*/' filename and i went well. But then when I tried to ...
5
votes
3answers
222 views

Escape sequences passed as args to gawk not interpreted

I want to be able to pass an argument on the command line to gawk that is evaluated for escape sequences. The issue: $ gawk 'BEGIN { print ARGV[1]; }' '\t' \t Instead, I would like to get an ...
4
votes
2answers
98 views

How to preserve mixed linefeed formats in gawk?

I have an input file which contains both Unix (LF) and Windows (CR/LF) style newlines. (Specifically, it's XML from a Linux system, but it contains some raw HTTP headers, and HTTP prefers CRLF for ...
16
votes
2answers
3k views

How does awk '!a[$0]++' work?

This one-liner removes duplicate lines from text input without pre-sorting. For example: $ cat >f q w e w r $ awk '!a[$0]++' <f q w e r $ The original code I have found on the internets ...
3
votes
3answers
225 views

Output to stdout works, fails when redirected to a file or a fifo

I have a Wifi antenna and I want to optimise it's position. For that propose I'd like to visualise the current latency in real time. I wrote a one-line script to output the ping latency: lat.sh: ...
1
vote
1answer
135 views

Insert variable in specific place using sed

Please help me i'm trying to insert variable in a file with sed but has not yet successfully(( I have variable $time and i want insert it in index.html file in specific string(line number 53): the ...
4
votes
3answers
165 views

awk ifs and variables - cannot pass a variable from one line towards subsequent lines

First of all I'm new to awk so please excuse if it's something simple. I'm trying to generate a file that contains paths. I'm using for this an ls -LT listing as well as an awk script: This is an ...
0
votes
3answers
153 views

Extract value from one file and insert it into another file replacing some other value

I have two files, whose line numbers are not same. But in file1 I want to replace lines taking reference from file2 using sed command. In file1: tid.infno := 72 tid.setnr := 120 (This number 120 is ...
-1
votes
1answer
108 views

Why doesn't this simple piping with awk work? [duplicate]

I'm trying to follow this answer but I can't make it work. I simplified it down as much as I could to try work out what's going on (just running this at the terminal for now): rsync --progress -iav ...
2
votes
4answers
1k views

Using Grep -o or Sed/Awk to Grab Snippet from Middle of String

I am watching log traffic and see regular abuse of the various image search engines, Bing particularly. Example URL: ...
2
votes
3answers
102 views

Delete lines between 2 strings in Solaris

I want to delete all the line between two patterns using basic awk or sed. foo.txt : ---------------------------------------------------------------------- Cap in MB line 2 line 3 line 4 line 5 ...
-2
votes
1answer
359 views

AWK sum every two rows

I want to sum two rows until end of file: 1 2 3 4 0 1 2 3 0 4 5 6 2 3 5 6 Output should be 1 3 5 7 2 7 10 12 I have tried the following, but it only works for the 1st row awk ...
0
votes
2answers
45 views

Issue on awk input parameter in ksh

I am running this script: INPUTNAME=ABC.XYZ FILENAME="Sample.xml" awk -v inputName=$INPUTNAME ' /<machine.*name=/ { f=1 ; m=0 ; res="" } f { res = res $0 ORS } f && ...
0
votes
1answer
152 views

How to implement awk range pattern in fetching an xml block when the input parameter is in the middle of the block

I am trying to fetch an xml block like this: <machine name="sample1" min="1" max="10" idleTime="300" backend="ABC,XYZ"> <handler className="com.abc.xyz.qwerty.foo.FooBar" ...
5
votes
6answers
564 views

sed command to swap characters

My input file layout is: mm/dd/yyyy,hh:mm,other fields I need to format it as: yyyy-mm-dd hh:mm:00,other fields sample input: 01/02/1998,09:30,0.4571,0.4613,0.4529,0.4592,6042175 ...
3
votes
1answer
55 views

How to output a comparison of values in separate files

With the following two files available with build names and version numbers available: server.txt Build2 200 Build3 100 Build1 400 Build7 200 local.txt Build1 300 Build2 200 Build3 100 In my ...
3
votes
7answers
164 views

Printing specific section of a line when a trigger value is present

I'm parsing data in the following format: prop1=value1:prop2=value2:prop3=value3+prop1=value4:prop2=value5 parts of the string are delimited by + properties can appear in any order the desired ...
2
votes
1answer
45 views

How is the order for which keys are stored and reteived determined in awk?

I am just starting to learn awk and was familiarizing myself with associative arrays only to discover that the order in which the keys were retrieved (or stored) didn't make a whole lot of sense. As ...
1
vote
3answers
183 views

Using AWK to convert numbers to 0 or 1

This might be a simple question but I can't figure it out for the life of me. I have a file called keyout.dat which looks like this: Title 1 Variable -999 -999 0.234 0.123 -999 -999 0.555 The file ...
2
votes
3answers
3k views

determining column number using string in awk

Suppose I have a pipe separated file like: |Sr|Fruits|Colors| |1 |apple |red| |2 |orange |orange |3 |grapes |purple| Here it is evident using awk that, $2 is Fruits and $3 is the colors column. ...
0
votes
3answers
2k views

Find all words beginning with Q

How do I search a text file and list all the words begining with Q? The file I am searching is Python source code. The closest I have found to a soulution is this question but that is not what I ...
1
vote
2answers
30 views

get specific variable using awk

I have this file.txt kespaul pts/14 2001:778:200:180 Fri May 29 11:33 - 16:40 (00:07) kespaul sshd 2001:778:200:180 Fri May 29 16:33 - 16:40 (00:07) algvai pts/2 ...
1
vote
1answer
157 views

finding lines in a file matching patterns defined in another file

I have two files file1.txt kepaul jack mike david and file2.txt 1 london smith 12564 2 miami jack 51616 3 la mike 165156 4 london kepaul 54654 5 miami grag 46546 and I need to print to a new ...
1
vote
2answers
62 views

Specify String Lenth in Awk Pattern Search

I'm not sure what I'm doing wrong. Given a test.log of: 123.123.123.123 321.321.321.321 GET /test1234/asdfasdfpioasdfjkhweris,9 123.123.123.123 321.321.321.321 GET ...
2
votes
2answers
142 views

how to print field between the double quote

I have the following file conf.txt: perl /home/site/gen.pl "GG - W C1 Test - sai1" "15072" && ssh 13.4.93.103 perl /home/site/gen.pl "GG - W C1 Test - sai2" "15072" && ssh ...
0
votes
0answers
34 views

awk search for string containing forward slash in variable [duplicate]

I'm searching for a string in a massive XML file. Snip: <sv:node sv:name="notfound"> <sv:property sv:name="jcr:primaryType" sv:type="Name"> ...