awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

2
votes
2answers
844 views

print a file in hex using awk

I have a file name x $ cat x 1A34532112345 I wants to print in hex using awk I written a script (with help from web) fold -1 /home/cscape/Desktop/x | gawk '{ printf("%s , %X\n",$0, int($0) )}' ...
3
votes
1answer
368 views

Split file based on a pattern with leading zeros

I have a book in text format. I would like to split the book into several files where each file contains a single chapter. Therefore I'm using the following command: awk '/Chapter/{i++}{print > ...
0
votes
1answer
291 views

globbing, sed, or awk html files

I wanted to create a script that I could check .xhtml files for images and add alt tags if needed. While searching I found regular expressions and did man on glob but I'm unsure on where or what to ...
1
vote
2answers
114 views

Extra % in output of awk script

I have the following script in awk that sums all columns in a file that I pipe to it: #sum_all.awk { for (i=1; i<=NF; i++) { sum[i]+= $i } } END { for (i=1; i<=NF; i++ ) { printf "%d ", ...
2
votes
3answers
873 views

Merging content of two files with comparing content of column

I have a problem trying to merge content of two files : file 1 : 873 1.1.1.1 1033 99.99.99.10 1108 78.2.204.174 1257 78.2.8.181 file 2: test1 78.2.8.181 p6 test2 99.99.99.10 p9 test7 1.1.1.1 ...
2
votes
3answers
474 views

Script matching literal pattern over multiple lines?

I have a multi-line string in the variable $PAT. $PAT must be searched for within the file $FILE. If $PAT is in $FILE, it needs to print the file with $PAT removed. If $PAT is not found, then print ...
3
votes
5answers
186 views

grep device name and look for next value :

I have this output from a find command: abc,10.11.13.14,def,1.2.3.4,geh,6.7.54.23 where abc,def and geh are device names and could be of any length and others are IP address belong to devices. ...
7
votes
10answers
1k views

What's a good way to filter a text file to remove empty lines?

I have a .csv file (on a mac) that has a bunch of empty lines, e.g.: "1", "2", "lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum ...
4
votes
1answer
1k views

awk high precision arithmetic

I am looking for a way to tell awk to do high-precision arithmetic in a substitution operation. This involves, reading a field from a file and substituting it with a 1% increment on that value. ...
0
votes
1answer
53 views

how does this work in awk even without print?

bash-3.00# cat > b1.txt AAA,Apples,123 BBB,Bananas,124 CCC,Carrot,125 bash-3.00# cat > b2.txt Store1|AAA|123|11 Store2|BBB|124|23 Store3|CCC|125|57 Store4|DDD|126|38 bash-3.00# awk -F"[,|]" ...
1
vote
1answer
339 views

How do associative arrays work?

bash-3.00# cat iplogs.txt 180607 093423 123.12.23.122 133 180607 121234 125.25.45.221 153 190607 084849 202.178.23.4 44 190607 084859 164.78.22.64 12 200607 012312 202.188.3.2 13 ...
2
votes
1answer
390 views

Can i update my commands( like awk,sed) to the latest release in solaris?

bash-3.00# awk '{print NR}' f1.txt 1 2 3 4 but FNR does not print anything. awk '{print FNR}' f1.txt the above line does not gives me any output,seems that the awk that i am having in my ...
1
vote
2answers
411 views

Deleting specific lines and duplicates from a 11gb wordlist text file

I have an 11gb wordlist file which is already sorted as each word is on its own line. I need to remove duplicates and lines starting from 077. I guess I need to run sed and sort -u together but I ...
0
votes
1answer
97 views

Sort Column with matching string

I have tab-demlimted file, I want to sort them only for red_ball as follows input.txt id tagid name HH556 EN76 red_ball JH678 EN45 green_ball MH678 CV67 red_ball TY896 ...
1
vote
2answers
179 views

Missing float function in awk on RHEL 5.8

Why is float missing in awk on my RHEL 5.8? Was it replaced by some other function? On Solaris: echo "Foampile=123" | awk -F"=" '{ print float($2) "<->" $1 }' returns 123<->Foampile ...
2
votes
2answers
2k views

How can I use awk to extract URL's from a HTML file?

I have an HTML file with javascript and CSS in the source. Listed in the JS is a series of URLs' embedded with other meta-data. I want to use awk to extract the URLs (all enclosed in double quotes ...
2
votes
2answers
2k views

Diff, show modified line from right file only

I have file a and b and I would like to output lines of b that changed since it was cloned from a. Just the modified lines, no surrounding context, no diff offset marks. How can I do that using shell ...
1
vote
3answers
3k views

Print lines where first field has only four characters using regex in awk?

John Goldenrod:(916) 348-4278:250:100:175 Chet Main:(510) 548-5258:50:95:135 Tom Savage:(408) 926-3456:250:168:200 Elizabeth Stachelin:(916) 440-1763:175:75:300 output should contain the lines ...
2
votes
1answer
405 views

Replace semicolons to commas between the quotes only [duplicate]

Possible Duplicate: Remove comma between the quotes only I'm trying to use awk function to replace the semicolons between quotes to comma and it is 98% ok for me, because I want to keep the ...
1
vote
1answer
125 views

renaming diverse files

I store files named like this: word-word2-word3-word4.txt . The number of "words" varies from file to file. I want to replace certain of the dashes with pluses e.g word+word2-word3+word4.txt I want ...
1
vote
2answers
1k views

external variable in awk [duplicate]

Possible Duplicate: Use a script parameter in awk I have a script, in which a script snippet is x=3 awk '$2=$x{print $1}' infile the external variable is x, but it prompts an error in ...
10
votes
4answers
12k views

Remove line containing certain string and the following line

I use this cat foo.txt | sed '/bar/d' to remove lines containing the string 'bar' in the file. I would like however to remove those lines and the line directly after it. Preferably in sed, awk or ...
0
votes
2answers
550 views

put rows of numbers into a column with shell script

I have a file with many lines, like 1 jfkdajfd 1 2 3 5 2 fkldfjld 3 fdkfloaf 9 10 4 fldfldkf 5 fdskf;ak 12 1 4 I want to get all the numbers and put them in a column in a file, like 1 ...
2
votes
4answers
978 views

Replace all but a set of characters in a file with newline

how to delete all characters in file except numbers and "." , each word (numbers/dot) should be in new line in file see example2 the solution can be with sed or awk or ksh syntax remark - ...
0
votes
2answers
266 views

Easiest way to find/replace in a file using a list?

I have a file, foo.txt, and a regexp I want to find in that file. Each time I find the regexp, I want to take a line from another file, bar.txt, and substitute it in for the regexp match I found in ...
1
vote
1answer
1k views

awk - get input from both file and STDIN?

Is it possible to get input from both file and the terminal? I wish to ask the user something in the END portion of the awk script. That input should be typed in by the user and not read from the ...
2
votes
3answers
2k views

How to cat a file from awk?

END { file = "a.txt" system(cat file) } I wish to do something like that? (open a file whose name is in an awk variable). How is it done?
4
votes
5answers
3k views

Convert formatted dates to seconds since the epoch

I have a file: pablo tty8 Thu Nov 1 12:51:21 2012 still logged in (unknown tty8 Thu Nov 1 12:50:57 2012 - Thu Nov 1 12:51:21 2012 (00:00) pablo tty2 Thu Nov 1 12:50:39 2012 still logged in pablo ...
0
votes
3answers
72 views

Fixing header and print

I have header starting with '>' and I want fix the header by keeping first word and removing other shown as in output.txt and print it input.txt >AGAJ01065549.1 ...
0
votes
3answers
891 views

Split and print

I have tab-demlimted text file with 50,000lines. I want to split the first column and print as shown in output.txt Input.txt rt|371443144|mb|MN556661.1| 2200443 A rt|371443344|mb|MN556645.1| ...
5
votes
2answers
2k views

Replacing missing value blank space with zero

I have input.txt tab-delimited text file around 30K lines, I would like to check each row (s1..s30K lines) for missing value (i.e blank space) and fill the missing value with zero value.See out.txt ...
2
votes
1answer
213 views

Count occurrences where two fields are equal

Is it possible to write an awk one-liner which will compare two fields on the same line to see if they are equal, and output the number of lines where this is true? I tried: awk '$1==$2 { print($1 ...
6
votes
5answers
1k views

Find images by size: find / file / awk

I've been trying to find png image files a certain height (over 500px). I know that file will return image dimensions. Example: $ file TestImg1a.png TestImg1a.png: PNG image data, 764 x 200, ...
1
vote
2answers
334 views

Print range of files based on time range using awk?

I have done a few simple things with awk but I cannot find a way to do this! I have a text file containing entries named with year, julian date and time as such: blah_2012251_130500_blah ...
1
vote
3answers
2k views

Find common elements in a given column from two files and output the column values from each file

I have two files with tab-seperated values that look like this: file1: A 1 B 3 C 1 D 4 file2: E 1 B 3 C 2 A 9 I would like to find rows between files 1 and 2 where the ...
1
vote
4answers
650 views

Replace text between brackets

I'm using awk '{ gsub(/BAR|WIBBLE/, "FOO"); print }' to replace text in data like: SOMETHING [BAR, WIBBLE] SOMETHING [BAR] This gives the desired result of: SOMETHING [FOO, FOO] SOMETHING [FOO] ...
2
votes
3answers
525 views

Regex/pattern matching problem to extract data

What is the best way to remove distances in the following lines and output cdv format. Can this be done reasonably easily on the command line? I'd like to avoid having to write a full-blow script. ...
3
votes
3answers
137 views

Print only lines that are completely numeric

I'd like to filter through a text file and only print the lines where each column is a valid floating point number. For example: 3 6 2 -4.2 21.2 3 x 4.2 21.2 3 2 2.2.2 Only the first line would ...
1
vote
1answer
215 views

Combining files with 4 columns each, into 4 files for each column [closed]

So I have a bunch of text files which look like this: T1 r1 a b c d r2 d e f g ... T2 r1 h i j k r2 l m n o ... And I want to make FOUR different text files in the following ...
1
vote
2answers
132 views

Extracting a multiline regex without leading tabs

I've been trying to hack a little code extraction script together, but I can't get it to work. My goal is to examine all .txt files in a directory. If it contains a line which doesn't start with a ...
1
vote
5answers
215 views

frequency of occurence

I have a tab-delimited text file, that is around 3K lines long. I want to calculate frequency of data appearing in it, in 10,000 ranges. Input.txt Price 500 1500 10001 15000 17000 ...
2
votes
4answers
196 views

Output some sections of a text file [closed]

I have input like the following: >gi|481443199|gb|MH447771.1| 49 0 0 0 1 50 0 0 1 0 >gi|987078004|gb|RTYJ01067061.1| 412 0 0 1 ...
1
vote
2answers
327 views

List info about files/directories with spaces

I am trying to use ls to get information about files and directories. My current command fails to properly get the name whenever a file/directory has a space. I am using this to list all ...
2
votes
4answers
761 views

Change currency format to generic number using sed

I am working on a large csv file with all coma separated entries. The entire document is composed of plain numbers, except for a few columns reflecting currencies that are formatted as text. For ...
5
votes
2answers
499 views

Use a script parameter in awk

Here is my script (to find the files that contain a specified pattern) : find . -type f -exec awk -v vawk="$1" '/'"$vawk"'/ {c++} c>0 {print ARGV[1] ; exit 0 } END { if (! c) {exit 1}}' \{\} \; ...
1
vote
3answers
3k views

bash merge 2 lists removing duplicated

I have an embedded linux system using Busybox (OpenWRT) - so commands are limited. I have two files that look like: first file aaaaaa bbbbbb cccccc mmmmmm nnnnnn second file mmmmmm nnnnnn ...
3
votes
3answers
1k views

Average rows with same first column

Given a file with two columns: Id ht 510 69 510 67 510 65 5l0 62 510 59 601 29 601 26 601 21 601 20 I need a way to coalesce all rows with the same ID into one that has an average height. In this ...
4
votes
4answers
3k views

AWK - how to reduce the greed of a regular expression?

I have an example: echo "@article{gjn2010jucs, Author = {Grzegorz J. Nalepa}, " | awk '{ sub(/@.*,/,""); print }' Is it possible to write a regular expression that selects the shorter pattern? ...
1
vote
1answer
1k views

Variable Substitution in Awk Print Statement -v

I've spent over an hour poking at this. It can't be this hard... I want to print a column of data from one file to another file. The column I want to print is dependent on what is passed in and stuff ...
2
votes
1answer
111 views

Extracting some files from a folder and creating a list of these files

I am just a beginner with Unix and so my skills in Unix commands is limited. I am currently dealing with a folder containing about 1000 files and I have to extract some filenames from this folder and ...