awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
2answers
68 views

Using multiple awk commands within single line

Here is what am trying to do: My data is in sets of rows within same file (the variables vaa vbb are looping over it and am able to control it as required) What i need is to add an extra column before ...
1
vote
1answer
20 views

calculate numbers on end and merge to one line

So in a file i have lines like these: thisdata:thisother:1337 thisdata:thisother:1800 thisdata:thisother:1500 thisdata:thisother:46984 I would like to output: thisdata:thisother:51621 for all ...
-1
votes
1answer
17 views

Matching 2 different files by first column

So I have 2 different files. file1.txt is: abcdefjeiireiiioe:data:otherdata rijirjeiwrieeoooe:datamore:otherdatamore file2.txt is: abcdefjeiireiiioe:data:otherotherdata ...
1
vote
1answer
33 views

Obtaining unique lines with matching strings [duplicate]

I have a JSON like text file of records with duplicate ids that look like: {"ID":"93" , "ST":[42,77,51]} {"ID":"21" , "ST":[43,4]} {"ID":"94" , "ST":[65,11,4]} {"ID":"93" , "ST":[42,77,51,29,63]} ...
2
votes
4answers
45 views

Sum of alternate values in a column using either sed or nawk

foo.txt: 1 10 11 2 20 22 3 30 32 4 40 42 5 50 52 6 60 62 7 70 72 8 80 82 9 90 92 10 100 110 Desired Out.txt : 1 10 11 2 20 22 3 ...
1
vote
2answers
38 views

Concatenate multiple fields separately based on one (key) column

I have a tab-separated test data like this: A 1 2 B 3 4 B 5 6 C 7 8 I want to concatenate (aggregate) both column#2 and #3 based on duplicate values ...
0
votes
1answer
32 views

Convert particular lines in a text file to bold [closed]

I have a text file with 100 lines in it. I want to convert particular lines, lets say line1, line6 , line27, line 48, line 86, line99 to bold. I am using solaris 5.10, with KSH. Is there a way I can ...
2
votes
2answers
37 views

Awk - keep word “example” and “EXAMPLE” with one input

I am attempting to to only keep customers that contain certain words. Here is an example < newname.txt awk 'BEGIN {FS=","} $7~/USA|United States/' > usa.txt The problem I am having is if they ...
-1
votes
2answers
50 views

can i do this or how can i do this awk + sort?

cat file_outunique.out | awk '{ print $1, $2}' | sort $2 -u > ledernier.out I want to output $1 and $2 for the 1st iteration of $2 Do you have any suggestions about this? I've tried to change ...
1
vote
3answers
77 views

awk manipulation of fasta file

I have a file like this >chr1 ACGTGGC TGCCGTT ATCCTTG >chr2 ACTTTTA CTCATAA I want to convert the seq into 1 string. This should be the output: >chr1 ACGTGGCTGCCGTTATCCTTG >chr2 ...
4
votes
3answers
80 views

How to insert multiple blank lines after every line

I want to insert 5 blank lines after every line in my input file. foo.txt: line 1 line 2 line 3 out.txt: line 1 line 2 line 3 ... Solaris 5.10, nawk or sed.
80
votes
2answers
170k views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
4
votes
4answers
484 views

How to insert all lines from one file to another file?

File1: .tid.setnr := 1123 .tid.setnr := 3345 .tid.setnr := 5431 .tid.setnr := 89323 File2: .tid.info := 12 .tid.info := 3 .tid.info := 44 .tid.info := 60 Output file: .tid.info := 12 ...
3
votes
1answer
41 views

Using awk to process a mapping file with dynamic number of columns

I'm a complete newbie to awk and am wondering how to process information as following: mapping.txt: 80 001 002 81 011 012 013 014 82 021 022 ... input.txt: 81 103823044 80 103823054 81 ...
3
votes
6answers
120 views

Case-insensitive count duplicate line, remove duplicate by choose case with highest duplicate

The duplicate is combination of different case text. I need to count number of duplicate (case-insensitive) and then I need to remove duplicate by choose case with highest duplicate. Below example: ...
3
votes
1answer
37 views

Sending signals to long-running processes (AWK, DD)

I'm familiar that DD command doesn't produce output while running. However, if I send a USR1 signal to it, it can print current state. Can I do the same with AWK? Prints its variables, register ...
3
votes
1answer
31 views

Match pattern, do calculation and print only 3 or 4 floating values max

I would like to convert below to GB and print only 3 floating values v copvol2 fsgen ENABLED 204800 - ACTIVE - - v copvol3 fsgen ENABLED 204800 - ...
0
votes
2answers
25 views

Unable to perform a simple multiplication in script

I am havingf trouble performing a simple multiplication in my script. while read A B C do tmp=$A\*$C/100 echo $tmp >> out1.txt done < foo.txt foo.txt: 13721725 99 100 ...
2
votes
4answers
74 views

Change text in tag, but only tag is contained within a certain XML block

Using Git Bash, I'm trying to conditionally replace what's in the yrot tag in hundreds of files, but only if it belongs with a part name tag that pertains to wheel. // YES, change <part ...
2
votes
5answers
115 views

Adjust gap between 2 columns to make them look straight

file1.txt: hi wonderful amazing sorry superman superhumanwith loss file2.txt : 1 2 3 4 5 6 7 When i try to combine using paste -d" " file1.txt file2.txt > actualout.txt actualout.txt : hi 1 ...
2
votes
1answer
24 views

Unset all ENV variables matching _PROXY

I'm trying to unset all environment variables that match _PROXY: env | grep -i _proxy | column -t -s '=' | awk '{ print $1 }' | grep -iv 'no_proxy' | xargs -0 -I variable unset variable but it's ...
0
votes
2answers
41 views

How to find and replace only a particular word (which is at the end of the file) using shell command?

File1: Dms 01.01.2015 feeder1 6kv close at 04:30 Updated Dms 01.01.2015 feeder1 6kv open at 06:20 Updated Dms 04.02.2015 feeder10 6kv close at 17:23 Updated Dms 04.02.2015 feeder12 6kv ...
4
votes
2answers
167 views

how to extract parts of file into separate files in linux using awk or grep or sed commands

I have text file as shown below: Input file1.txt ....................... cha21 1 3 5 nar cha21 2 3 6 piy cha23 2 3 5 ram cha23 3 ...
5
votes
8answers
462 views

How do I delete the first n lines and last line of a file using shell commands?

I have a file named Element_query containing the result of a query : SQL> select count (*) from element; [Output of the query which I want to keep in my file] SQL> spool off; I want to ...
1
vote
1answer
54 views

How to write data one by one from tmp files to the final output file using for loop?

I have two tmp files named tmp1 and tmp2 which contains some lines. tmp1 file, 1c\ datafile no. 23 2c\ datafile is ok tmp2 file, 3c\ datafile no. 24 4c\ datafile is ok I have a ...
0
votes
1answer
35 views

How to remove certain number of lines every time particular string is found in file

I want to remove 1 previous line and 10 following lines every time a string was found in the file. foo.txt: } Name: john Line 3 Line 4 Line 5 Line 6 Line 7 Line 8 Line 9 Line 10 Line 11 Line 12 ...
1
vote
2answers
42 views

How to print few lines from middle of a file using unix shell commands? [closed]

My file containing the following lines, $BQ { VOL @home } database daba relation tcdeatid opendb clear .lruno := 72 <-- (This line lruno. := 72 has 10 times ...
2
votes
3answers
31 views

Truncating the precision of a date

I'm working with a bash script which polls build progress from several different systems. If a build is complete, it stats a log file to find out the last time it was edited, then feeds that time into ...
1
vote
2answers
26 views

Addition of values

foo.txt: Lillypaul_sg 204803 204803 204803 Ammy_sg 3 ramaswamy_sg 3 33 tommy_sg 3 3 137374 100 Desired Out.txt : Lillypaul_sg 614409 Ammy_sg 3 ramaswamy_sg 36 tommy_sg 137480 The output is a sum ...
1
vote
3answers
59 views

Sed Replace at a specific group of positions

I have a sample input file as below: Apple Orange Gold Silver Spoon Apple Orange Gold Apple Apple Orange Apple Orange Mango Gold ...
3
votes
2answers
72 views

shell script to extract between characters

I would like to extract a user name from the file between the keywords "/" and "NET" like below. How can I do so with programs like awk, sed or cut. from: audit: command=true ...
3
votes
4answers
206 views

Extract paragraph separated with *** using AWK

I have a file like below: blablabla blablabla *** thingsIwantToRead1 thingsIwantToRead2 thingsIwantToRead3 blablabla blablabla I want to extract the paragraph with thingsIwantToRead. When I had to ...
0
votes
2answers
48 views

Count frequency of specific numbers in a text file of scientific notations

Background I have a text file, named blood_conc.txt as shown: 0, 0, 0, 0, 0, 0, 0, 1.32006590271e-05, 1.990014992001e-05, 1.504668143682e-05, 2.176900659261e-06, 7.673488970859e-06, ...
3
votes
2answers
877 views

Split log file by time range

I have a log file and would like to cut it into copies of 15min. In the log file the lines start with: dd mmm yyyy hh:mm:ss,xxx e.g. 12 Feb 2013 16:05:02,xxx log text... and the file rotates ...
-2
votes
4answers
57 views

How to print last few contents in a line?

I have a file that has lines shown below, ta=/*92/*39/*1/*649/*12 ta=/*92/*12/*4/*342/*90 ta=/*92/*33/*9/*455/*3 ta=/*92/*12/*3/*649/*25 ta=/*92/*99/*7/*649/*7 and so on. I want a file that will ...
2
votes
2answers
75 views

Check if conditional with awk for matches in logfile

I want to check the auth.log file for accepted sshd connections and execute an action if any are found. Here is my code: tail -f /var/log/auth.log | awk '{ if($0 ~ /sshd/ && $0 ~ /Accepted/) ...
0
votes
3answers
52 views

How do I pass a series of patterns into awk from a shell script?

I've got a series of patterns stored in foo.txt and I want to iterate over those patterns and see which match lines in file bar.txt. while read PWW do cat user/dump/bar.txt | awk '/${PWW}/{print ...
0
votes
1answer
32 views

How to integrate awk parameters in awk range pattern

I need to get something like this #Batch Job - AB1234 texts texts texts texts texts #-------- I used awk '/#Batch Job.*AB1234/,/#--.*/' filename and i went well. But then when I tried to ...
5
votes
3answers
145 views

Escape sequences passed as args to gawk not interpreted

I want to be able to pass an argument on the command line to gawk that is evaluated for escape sequences. The issue: $ gawk 'BEGIN { print ARGV[1]; }' '\t' \t Instead, I would like to get an ...
4
votes
2answers
41 views

How to preserve mixed linefeed formats in gawk?

I have an input file which contains both Unix (LF) and Windows (CR/LF) style newlines. (Specifically, it's XML from a Linux system, but it contains some raw HTTP headers, and HTTP prefers CRLF for ...
15
votes
2answers
1k views

How does awk '!a[$0]++' work?

This one-liner removes duplicate lines from text input without pre-sorting. For example: $ cat >f q w e w r $ awk '!a[$0]++' <f q w e r $ The original code I have found on the internets ...
3
votes
3answers
55 views

Output to stdout works, fails when redirected to a file or a fifo

I have a Wifi antenna and I want to optimise it's position. For that propose I'd like to visualise the current latency in real time. I wrote a one-line script to output the ping latency: lat.sh: ...
1
vote
1answer
36 views

Insert variable in specific place using sed

Please help me i'm trying to insert variable in a file with sed but has not yet successfully(( I have variable $time and i want insert it in index.html file in specific string(line number 53): the ...
15
votes
10answers
2k views

How can I quickly sum all numbers in a file?

Each line contains text and numbers in one column. I need to calculate the sum of the numbers in each row. How can I do that? Thx example.log contains: time=31sec time=192sec time=18sec time=543sec ...
4
votes
3answers
84 views

awk ifs and variables - cannot pass a variable from one line towards subsequent lines

First of all I'm new to awk so please excuse if it's something simple. I'm trying to generate a file that contains paths. I'm using for this an ls -LT listing as well as an awk script: This is an ...
0
votes
3answers
47 views

Extract value from one file and insert it into another file replacing some other value

I have two files, whose line numbers are not same. But in file1 I want to replace lines taking reference from file2 using sed command. In file1: tid.infno := 72 tid.setnr := 120 (This number 120 is ...
0
votes
1answer
50 views

Why doesn't this simple piping with awk work? [duplicate]

I'm trying to follow this answer but I can't make it work. I simplified it down as much as I could to try work out what's going on (just running this at the terminal for now): rsync --progress -iav ...
2
votes
6answers
161 views

grep with continuation lines

How can I grep/awk/sed a file looking for some pattern, and print the entire line (including continuation lines if the matched line ends with \? File foo.txt contains: something whatever thisXXX ...
2
votes
4answers
102 views

Using Grep -o or Sed/Awk to Grab Snippet from Middle of String

I am watching log traffic and see regular abuse of the various image search engines, Bing particularly. Example URL: ...
2
votes
3answers
51 views

Delete lines between 2 strings in Solaris

I want to delete all the line between two patterns using basic awk or sed. foo.txt : ---------------------------------------------------------------------- Cap in MB line 2 line 3 line 4 line 5 ...