awk - pattern-directed scanning and processing language
4
votes
4answers
1k views
Text between two tags
I wanna retrieve whatever is between these two tags – <tr> </tr> – from an html doc.
Now I don't have any specific html requirements that would warrant for an html parser. I just plain ...
10
votes
4answers
10k views
How to manipulate a CSV file with sed or awk?
How can I do the following to a CSV file using sed or awk?
Delete a column
Duplicate a column
Move a column
I have a big table with over 200 rows, and I'm not that familiar with sed.
3
votes
3answers
823 views
Show lines matching a pattern and the 4 lines before each
For example, from this file:
CREATE SYNONYM I801XS07 FOR I8010.I801XT07
*
ERROR at line 1:
ORA-00955: name is already used by an existing object
CREATE SYNONYM I801XS07 FOR ...
7
votes
5answers
3k views
Remove comma between the quotes only in a comma delimited file
I have a input file delimited with commas (,). There are some fields enclosed in double quotes that are having a comma in them. Here is the sample row
123,"ABC, DEV 23",345,534.202,NAME
I need to ...
0
votes
2answers
319 views
external variable in awk [duplicate]
Possible Duplicate:
Use a script parameter in awk
I have a script, in which a script snippet is
x=3
awk '$2=$x{print $1}' infile
the external variable is x,
but it prompts an error in ...
11
votes
11answers
6k views
Grep huge number of patterns from huge file
I have a file that's growing about 200,000 lines a day, and it is all formed with blocks of three lines as such:
1358726575123 # key
Joseph Brunner # name
carpenter # job
...
15
votes
9answers
6k views
Tool in UNIX to subtract dates
Is there any tool in Solaris UNIX (so no GNU tool available) to subtract dates? I know that in Linux we have gawk that can subtract one date from another. But in Solaris the maximum we have is nawk ...
5
votes
1answer
353 views
Why are capital letters included in a range of lower-case letters in an awk regex?
$ echo ABC | awk '$0 ~ /^[a-b]/'
ABC
$ echo ABC | awk '$0 ~ /^[a-a]/'
$ echo ABC | awk '$0 ~ /^a/'
$
You see. /[a-b]/ captures A, but /[a-a]/ or /a/ doesn't. Why?
5
votes
2answers
179 views
Use a script parameter in awk
Here is my script (to find the files that contain a specified pattern) :
find . -type f -exec awk -v vawk="$1" '/'"$vawk"'/ {c++} c>0 {print ARGV[1] ; exit 0 } END { if (! c) {exit 1}}' \{\} \;
...
5
votes
3answers
13k views
How to use regex with AWK for string replacement?
Suppose there is some text from a file:
(bookmarks
("Chapter 1 Introduction 1" "#1"
("1.1 Problem Statement and Basic Definitions 23" "#2")
("Exercises 31" "#30")
("Notes and References 42" "#34"))
)
...
1
vote
4answers
347 views
Replace text between brackets
I'm using awk '{ gsub(/BAR|WIBBLE/, "FOO"); print }' to replace text in data like:
SOMETHING [BAR, WIBBLE]
SOMETHING [BAR]
This gives the desired result of:
SOMETHING [FOO, FOO]
SOMETHING [FOO]
...
11
votes
6answers
15k views
Is there a way to get the min, max, median, and average of a list of numbers in a single command?
I have a list of numbers in a file, one per line. How can I get the minimum, maximum, median and average values? I want to use the results in a bash script.
Although my immediate situation is for ...
7
votes
6answers
1k views
Is there any alternative to grep's -A -B -C switches (to print few lines before and after )?
grep -A 2 -B 3
prints 2 lines after the grep string and prints 3 lines before.
grep -C 3
prints 3 Lines before and 3 lines after
Unfortunately, the grep I'm using does not support these ...
12
votes
9answers
2k views
There must be a better way to replace single newlines only?
I am in the habit of writing one line per sentence because I typically compile things to LaTex, or am writing in some other format where line breaks get ignored. I use a blank line to indicate the ...
9
votes
3answers
425 views
Process last line first using awk
I have a data file that I want to normalize using awk, based on the last datapoint. Therefor, I would like to access the last data point first, to normalize the data, then process normally.
The ...
5
votes
4answers
189 views
Show sum of file sizes in directory listing
The Windows dir directory listing command has a line at the end showing the total amount of space taken up by the files listed. For example, dir *.exe shows all the .exe files in the current ...
5
votes
4answers
2k views
How to print certain columns by name?
I have the following file:
id name age
1 ed 50
2 joe 70
I want to print just the id and age columns. Right now I just use awk:
cat file.tsv | awk '{ print $1, $3 }'
However, this ...
3
votes
2answers
1k views
My awk program to change spaces into tabs doesn't work
How to write a shell script which uses awk to read in the data file students.txt and output the data in the tabbed format as shown:
Surname Forename MSc Stream Date of Birth
Smith John IT 15.01.1986
...
2
votes
4answers
662 views
gawk join two TSV by columns (a'la sql join)
How to join, to tsv files, examples:
a.tsv
c 7 r z
d 6 s w
f 1 f f
b 8 p y
a 9 q x
b.tsv
a q a
c r ccc
b p bb
0 0 0
d s dddd
Here I'd like to ...
8
votes
2answers
143 views
Are there 2 ways to set awk vars via command line?
I noticed an O'Reilly awk example (1997) which assigned an awk variable by setting it on the command line after the program-text. It does work, but I can't find this syntax in man / info awk. Have I ...
6
votes
4answers
332 views
Find images by size: find / file / awk
I've been trying to find png image files a certain height (over 500px). I know that file will return image dimensions. Example:
$ file TestImg1a.png
TestImg1a.png: PNG image data, 764 x 200, ...
5
votes
2answers
108 views
Sort fields inline
I'm trying to sort within a line of input over an unknown number of fields:
Input:
ab bc
bc ab
cd ef bc
bc cd ef
cd bc ab
ef ab bc cd gh
Output:
ab bc
ab bc
bc cd ef
bc cd ef
ab cb cd
ab bc cd ...
5
votes
2answers
249 views
How would I sort these directory names numerically?
I'm aware that I can somehow sort this output numerically (so cpu1/ follows cpu0/) ... I could probably get something to work eventually by splitting up the string various ways with awk, but is there ...
3
votes
1answer
158 views
Pass colors from ls through pipe to awk 'print' statement
This is a follow-up to my question from yesterday, Show sum of file sizes in directory listing.
Thanks to Zero Piraeus and a point in the right direction by Mauritz Hansen, I now have
function ...
3
votes
2answers
361 views
What is the best way to find a list of several strings within a large text file
The short, general question is: In Unix/Linux, what is the best way to find a list of several (about 150) strings within a large text file?
I am asking this to all Unix/Linux experts as a general ...
3
votes
1answer
194 views
grep list of names and information from bigger file
I have two files: one with list of names (500 entries) and other having some more information for each entry in A.txt and extra entries too.
File A.txt (each line is starting with > (fasta format) ...
2
votes
1answer
1k views
Time/Date grep for 15 mins
I need to grep a specific type of line in a specific time of the log, do you have any ideas?
Here's my working script:
cat *.log |grep -E '2011-06-30 (1[0-1]:[0-1][0-5]|10:16)'| grep -ach '0110 ...
1
vote
1answer
252 views
Reading two files into an IFS while loop — Is there a way to get a zero diff result in this case?
I have a text file full of several hundred lines of sequences like this:
b 29.
b 52.
c 84.
c 83.
c 94.
c 93.
c 61.
b 38.
c 81.
c 92.
c 28.
c 37.
...
