awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
2answers
552 views

gawk: Summing floats formatted with comma separator

I'm trying to sum floats which use the comma as separator, and gawk does integer calculations. Here's the script: BEGIN { FS=";" OFS=";" CONVFMT = "%2.2f" } { print $1 print $2 print ...
4
votes
2answers
342 views

Merge fields in a file

I have a file with 7 columns, a GFF file having chromosomal regions.I want to collapse the rows where REGION ="exon" to only one row in the file.The row has to be collapsed on the basis of regions ...
8
votes
4answers
14k views

Pass shell variable as a /pattern/ to awk

Having the following in one of my shell functions: function _process () { awk -v l="$line" ' BEGIN {p=0} /'"$1"'/ {p=1} END{ if(p) print l >> "outfile.txt" } ' } , so when called as ...
2
votes
5answers
424 views

Print text between (and including) two delimiters on a line

What's the best way to print the text between (and including) the first pattern ("SELECT" or "SSELECT") and a second pattern ("[") using a command-line took (e.g. cut, sed, awk)? For example, given ...
1
vote
3answers
255 views

How can I delete the first dot before the first number?

How to remove the first dot before the first number? For example I have the following file more file Version …………2.465.76.8.332 I want to get only the version number - 2.465768.332
7
votes
4answers
4k views

BEGIN and END with the awk command

According to the awk manual, BEGIN and END are not used to match input, but rather to provide start-up and clean-up information to the awk script. Here is the example given: ls -l | \ awk 'BEGIN { ...
2
votes
5answers
292 views

identifying words in file 1 and matching them to second file and printing them

I have two files as follows File 1 has list of Ids ram Tarah taikah jhdfklj File 2 has id=ram;*between*hkjhdswklf kjsdklhglshglkhgklsdgkhdfg id=taikah;jhdjfkhg**kjdfkjksfdjk ...
3
votes
5answers
227 views

Merging Two Files columns in order

I want to merge two files. Files A.txt 001;abc;def;ghi;jkl;pqr 002;abc;def;ghi;jkl;pqr 003;abc;def;ghi;jkl;pqr 004;abc;def;ghi;jkl;pqr 005;abc;def;ghi;jkl;pqr . Second File B.txt 001;mno 002;mno ...
0
votes
1answer
69 views

How can I use gsub with a pattern? And how to tell it which file to awk?

I've read here that I can replace the values of all columns containing a particular text using awk -F'\t' -vOFS='\t' '{ gsub("CC", "C", $1) ; gsub("AA", "A", $1) ; print }' But where do I tell awk ...
3
votes
3answers
2k views

awk remove the rows based on another column value

I have a file like below which I have sorted based on the username field. UserID score UserName 1234 200 Jack 5678 150 Jill 8543 200 Jill 5678 100 John I am trying to remove ...
1
vote
3answers
895 views

Print the last-but-one field

How to print the word before the last word in line (with ksh or awk or sed or perl one liner) Example 1: echo one two three will print "two" Example 2: echo 1 2 3 4 5 6 will print "5"
2
votes
1answer
91 views

awk to split the input file into 2 files(one for duplicated values and one for unique values)

I have an input file like below. file.txt 14 2.692022 author1 0 14 2.57826 author2 1 229 2 author3 2 323 2 author4 3 514 2.0625 author5 4 I am trying to split the file with the duplicated values ...
2
votes
6answers
116 views

Printing specific section of a line when a trigger value is present

I'm parsing data in the following format: prop1=value1:prop2=value2:prop3=value3+prop1=value4:prop2=value5 parts of the string are delimited by + properties can appear in any order the desired ...
0
votes
1answer
466 views

Confused with awk! Getting: 'cannot open “” for output'

Hi I have 2 files in my working directory. One is a simple file given by: $ cat file2 START Unix Linux START Solaris Aix SCO And I have a 6000 line xml file called dblp_1_76007.xml. When I ...
2
votes
4answers
296 views

How to multiply a data file with another index data file?

I want your help in multiplying my data file with another index data file. Both data files have the same number of rows. But the data file have more than 1000 columns while the index data has only one ...
1
vote
2answers
96 views

What's the difference between print and head?

I know head gives you the beginning section of a file. But in this case, I'm trying to go through my source control and find every file that has a shebang line. And I'm failing to get results when I ...
3
votes
4answers
7k views

Compare two text files and find matching lines

I have two files A and B. A looks like this (4 to 6 lines): GAGA CAGA GGGT TATT file B is a really big file with thousands of lines, here is a short example: AAATGTCAAGAGACAGAAATGTCAAGAGGGT ...
1
vote
1answer
758 views

syntax error in conditional expression while executing the shell script?

I am working on a project in which I need to make a url call to one of my server from the bash shell script.. http://hostname.domain.com:8080/beat After hitting the above url, I will be getting the ...
3
votes
2answers
777 views

Only print rows that have 2 or more fields?

I've got a text file and I'm trying to figure out how to filter out any lines with data in only 1 field, leaving lines with 2 fields. I came across the following code: awk '$NF > 1' but I can't ...
0
votes
2answers
136 views

search for first pattern and copy the string inside the ( ) into the 3rd field in lines where the second pattern found

Here are my spec: first search for "#firstpattern(" copy the content inside the ( ) to the 3rd field inside the ( ) of all lines where the "secondpattern" found. remove the entire line with the ...
6
votes
2answers
3k views

Counting number of times each IP address appears in log file

I have a file in the format as follows: $ cat file.txt 27.33.65.2 27.33.65.2 58.161.137.7 121.50.198.5 184.173.187.1 184.173.187.1 184.173.187.1 What's the best way to parse the file file.txt into ...
2
votes
1answer
427 views

solaris awk problems with date and if statements

I have daily files that come in via FTP with incorrect dates in the first column of the file. I have figured out how to deduct one day to derive the correct date and print this to a new file. The main ...
1
vote
1answer
400 views

awk date identification from input file

I currently have daily files that come in via FTP with incorrect dates in the first column of the file. I have figured out how to deduct one day to derive the correct date and print this to a new ...
1
vote
1answer
166 views

replace a pattern found with a number plus 1 on the same line

I have a file where each line contains a keyword (HELLO) and one or more numerical fields designated A1, A2, etc. The value in the numerical field is enclosed in brackets after the field name and is ...
6
votes
3answers
3k views

How to subtract rows (lines) with AWK

I'm trying to figure out how I can use AWK to subtract lines. For example, imagine the input file is: 30 20 The output would be: 10 Now, as a test I am trying to calculate the "Used" memory ...
-3
votes
2answers
80 views

How to get a device details only using awk commands

Friends, I have fdisk -l output in one file called test1.txt. Now I want to get the device name only using awk. How to get a device name? I want output like this: /dev/sda1 /dev/sda2 /dev/sda3 ...
2
votes
5answers
269 views

Can the string “abcedf” be matched to string “bafcde” in a single line command?

I am planning to implement an indexing structure in my program. For example, if I have 100 rows in the table, I will number these rows from 1 to 100 in another column by appending an _ to the end of ...
6
votes
3answers
12k views

Adding a Column of values in a tab delimited file

How can I add a Column of values in a file which has a certain number of rows. I have a input file like this: Input file: SPATA17 1 217947738 LYPLAL1 1 219383905 FAM47E 4 77192838 SHROOM3 4 ...
1
vote
4answers
248 views

How can I list the second to last word found in a sequence of statements in awk

My file is written like: apples,oranges,berries,cherries beer,wine,shots chocolate,icecream,cake,muffins cherries,guacamole,lychee I need to output the second to last word in lines that contain ...
1
vote
4answers
1k views

Pattern Matching and Delete the whole line

I want to delete all the Lines of File 1, if Column1 of File1 matches exactly with Column 1 File2. File 1: r001:21:10 21 AAAAAATTTGC * = XM:21 r002:21:10 21 YAAAATTTGC * ...
0
votes
2answers
159 views

How to extract submatches from a string and write those to a file?

I want to parse a string for specific submatches and output those - one submatch per line - into a file. Essentially I would like to extract the value of each href-attribute: $ cat Fin <a ...
2
votes
3answers
4k views

How can I remove the text from a line after a certain character with awk

How can I use awk to remove all text after a certain character ";" that appears on every line of my text file? (I then need to run for loops on the text) Jenny,Sarah,John;North Dakota ...
2
votes
3answers
257 views

List all word pairs that appear in a line together in a file

How can I use awk to run through a file written as: Sarah,Masha,Helen Connor,Jessica,Jennifer,Candace Betty,Sarah,Helen John,Harold,Frank,Daisy and put out all pairs that appeared in a line ...
4
votes
2answers
155 views

How can I delete all unordered lines from a text file?

Overview Consider an ordered list interspersed with unordered elements, e.g.: Alligator Ant Falcon <-- Baboon Badger Armadillo <-- Caiman Cat How can this list be processed so that all ...
5
votes
2answers
1k views

Replace string with sequential index

Can someone suggest an elegant way to accomplish this? Input: test instant () test instant () ... test instant () //total 1000 lines output should be: test instant1 () test ...
0
votes
3answers
603 views

removing spaces from first column

My input file has positions in first column with different number of spaces (or no space) 16504 16516 1650811 16520 1651 16524 16516111 16528 165204 16532 ...
2
votes
1answer
493 views

Awk regular expression syntax with number of repetition - different handling between gawk 3 and gawk 4

I am having trouble understanding or explaining why the following expression fails in gawk 3.1.x and yet works in gawk 4.1.x: (Minimum working example) echo ";#ADCDE#" | awk '/#.{5}#$/' -> produces ...
2
votes
2answers
160 views

/etc/shadow & gawk

I ran this small script on /etc/shadow: awk '{ FS = ":" } { print $1, $2 }' /etc/shadow It does what it should on all lines except for the root's record I.e. it just prints the entire record. Know ...
2
votes
3answers
106 views

How to optimize this oneliner? (if second column empty, put the line in another file)

I have a little "oneliner": while read ONELINE; do if [[ -n "`echo $ONELINE | awk '{print $2}'`" ]]; then echo "$ONELINE" >> LIST-ok.txt else echo "$ONELINE" >> ...
2
votes
1answer
612 views

Awk field separator bug?

This is the expected way for the awk field separator to work: $ echo 'fooXbar' | awk 'BEGIN {FS="X"} {print $1}' foo $ echo 'fooXbar' | awk 'BEGIN {FS="X"} {print $2}' bar $ but if the FS is ...
1
vote
3answers
103 views

parsing file from ls -laLR

ls -laLR /home/tools > all_site1.txt I need the state of all folders and files in subfolders of /home/tools. With this command I put all in this file. What I need now is a solution how to parse ...
5
votes
3answers
266 views

Increment numbers inside strings by an offset

I want to search a text file for strings like these. [task0] [task1] ... [task1613] and increment each by a given offset, in this case by 3640. So, with the example I gave above, I'd get the ...
2
votes
3answers
2k views

How to show CPU time for processes via top without 'root' procs

Okay, So I've been attempting this for about three hours now without success. How do you search/display/use the top command (or ps command if it works...) to output a list of all procs, sorted by CPU ...
1
vote
2answers
4k views

How do I multiply and sum column data using awk and or sed?

I have several columns of data. I always have the same number of rows (say 5). In the 2nd column, I want to multipy the first value by 5, then the second value by 4, the third value by 3, etc. I then ...
3
votes
2answers
138 views

Command in variable one are affecting output of other variable when displayed together in UNIX

I am having two file naming Extra_info and Class_info Class_info contains data : stuId stuName stuClass S01 jagdeep First S02 harsh First S03 Roop Second S04 Kali Third ...
2
votes
4answers
7k views

awk + print awk output in one line

When I run the following hasys command with awk , I get output with machines names as the following: hasys -display|grep Shutdown | awk '{print $1}' machine1a machine1b please advice how to ...
4
votes
4answers
2k views

Print line after nth occurrence of a match

I am looking to display the line 4598 in the following file. Effectively I want to display the line AFTER the nth occurrence of a match. In this case, the line after the 3rd occurrence of <Car>. ...
2
votes
1answer
391 views

What is an array in awk?

I'm very new to unix and am trying to learn basic awk scripting - during which I keep coming across the word "array", and "associative arrays". I have no clue as to what this is, and don't tell me to ...
4
votes
5answers
139 views

Reformat each two-line sequence into two columns

I would like to tab my printout: 1 2 3 4 5 6 7 8 to 1 2 5 6 3 4 7 8 How to adjust it using simple command line (awk or sed)??
2
votes
4answers
435 views

Count rows with specific integer in a column

I have 6 columns, each with multiple rows. I want to count the number of rows which have the integer 4 or 5 in the fifth column. A1 jhfj jdhfjkhd kdkfjjh 5 jhsdjkfh A2 ujhf jhdfhsd dsfkks 4 ...