awk - pattern-directed scanning and processing language
5
votes
3answers
672 views
killing processes automatically
I need to kill all processs in a certain shell excluding certain processes.
Like sh which is my shell. And the comand.
This is what currently in my shell right now.
rcihp146 :/home/msingh2> ps
...
2
votes
1answer
213 views
How to parse the file from end in awk
#!/bin/sh
if [ $# -ne 8 ]; then
echo "Usage: sh `basename $0` YYYYMMDD hour min masterid account destination filename forward|backward";
exit 1;
fi
day=${1}
hour=${2}
min=${3}
...
4
votes
6answers
2k views
How can I convert tab delimited data to comma delimited data?
I'm requesting a list of ec2 snapshots via amazon's ec2 command line tool:
ec2-describe-snapshots -H --hide-tags > snapshots.csv
The data looks something like this:
SnapshotId VolumeId ...
3
votes
2answers
857 views
use awk to replace parentheses?
How can I use awk to replace the parentheses in these 2 scenarios?
Change empty parens to the word "None"
192.168.0.24,Up,()
Remove parens to only host.domain.com is left
...
0
votes
2answers
589 views
nmap script needed to output text into csv format
ultimately I am looking to insert some info into a database. So I'm trying to output a file from something (probably nmap) that is formatted like this:
10.1.150.1,Up,1/3/2012,host.domain.lan
...
2
votes
1answer
697 views
Diff the output of two `awk` commands
I'm trying to compute the difference between the output of two awk commands but my simple attempts at it seem to be failing. Here is what I'm trying:
diff $(awk '{print $3}' f1.txt | sort -u) $(awk ...
10
votes
4answers
10k views
How to manipulate a CSV file with sed or awk?
How can I do the following to a CSV file using sed or awk?
Delete a column
Duplicate a column
Move a column
I have a big table with over 200 rows, and I'm not that familiar with sed.
1
vote
1answer
241 views
Reading two files into an IFS while loop — Is there a way to get a zero diff result in this case?
I have a text file full of several hundred lines of sequences like this:
b 29.
b 52.
c 84.
c 83.
c 94.
c 93.
c 61.
b 38.
c 81.
c 92.
c 28.
c 37.
...
1
vote
2answers
204 views
How can I match this text in awk, printing line numbers for matches?
I have a text file thousands (roughly 148,000 lines long) that consists of a lot of sequences like this:
b 29.
b 52.
c 84.
c 83.
c 94.
c 93.
c 61.
b 38.
c 81.
c 92.
c 28.
c 37.
c 27.
... and since ...
3
votes
6answers
738 views
Grep a directory and return list with line numbers
I'm currently trying to learn more about bash scripting and all of that fun stuff, and I pieced together this little command:
find $path | xargs grep -n $pattern | awk '{print $1}'
While this DOES ...
2
votes
1answer
275 views
Getting data section with maximum element
I have some data stored in JSON-like format.
{
{
value1: 14,
value2: 12,
value3: 1
},
{
value1: 4,
value3: -1
}
}
How can I get subsection ...
8
votes
3answers
2k views
When using awk /pattern/ { print “text”} /patern/ {print “”} is there an ELSE pattern?
Let's say I have text file like:
R1 12 324 3453 36 457 4 7 8
R2 34 2342 2525 25 25 26 26 2 2
R3 23 2342 32 52 54 543 643 63
R4 25 234 2342 4 234242
I want to use awk to process these lines ...
8
votes
6answers
734 views
Delete lines beginning with #
How do I delete lines beginning with a #, given that there can be whitespace on the left and right of the #?
# Master socket provides access to userdb information. It's typically
2
votes
4answers
362 views
Clean up concatenated XML files
I'm trying to fix my Delicious export script to work with the new "trick" of only serving 1000 bookmarks at a time. The script concatenates results (XML) from multiple calls in a single file, and now ...
2
votes
4answers
773 views
Trouble getting awk output in for loop
I'm trying to create a script that will check a website for a word. I have a few to check so I'm trying to input them via another file.
The file is called "testurls". In the file I list the keyword ...
5
votes
4answers
2k views
How to print certain columns by name?
I have the following file:
id name age
1 ed 50
2 joe 70
I want to print just the id and age columns. Right now I just use awk:
cat file.tsv | awk '{ print $1, $3 }'
However, this ...
3
votes
0answers
130 views
How to use regrex with AWK for string replacement in this example? [duplicate]
Possible Duplicate:
How to use regrex with AWK for string replacement in this example?
Suppose there is some text from a file:
(bookmarks
("Chapter 1 Introduction 1" "#1"
("1.1 Problem ...
5
votes
3answers
12k views
How to use regrex with AWK for string replacement in this example?
Suppose there is some text from a file:
(bookmarks
("Chapter 1 Introduction 1" "#1"
("1.1 Problem Statement and Basic Definitions 23" "#2")
("Exercises 31" "#30")
("Notes and References 42" "#34"))
)
...
1
vote
2answers
278 views
pattern search and display the last occuring pattern?
I have a log file containing startup and shutdown times for everday.
I want to see the last pattern pattern for startup and shutdown(which is located at the end of the file being updated everyday).
...
10
votes
6answers
2k views
How to print the longest line in a file?
I'm looking for the simplest method to print the longest line in a file. I did some googling and surprisingly couldn't seem to find an answer. I frequently print the length of the longest line in a ...
2
votes
2answers
363 views
Adding 2 lines in text file stanza
I have a text file divided in stanzas.
Each stanza starts with a [keyword] and ends with a blank line. Something like this:
[keyword1]
param1 = value1
param2 = value2
param3 = value3
[keyword2]
...
1
vote
3answers
386 views
To list out duplicate filenames in a listing?
bash-3.00$ cat f.txt
-rw-r--r-- 1 mukesh other 102 Nov 5 18:32 f1.txt
-rw-r--r-- 1 mukesh other 19 Nov 5 18:32 f2.txt
-rw-r--r-- 1 mukesh other 204 Nov 5 18:32 ...
3
votes
4answers
269 views
Find a string (like grep -q) within only a section of a file
I want to write some Bash that can verify that a string exists in a configuration file. I can't change the file format, it belongs to a different application.
The file is subdivided into groups named ...
-2
votes
2answers
228 views
How to delete the second column from this input?
INPUT
bix.hu. 42433 IN A 193.239.149.1
bix.hu. 42433 IN MX 10 deneb.iszt.hu.
bix.hu. 42433 IN NS ns.iszt.hu.
bix.hu. 42433 IN NS ns.iszt.hu.
bix.hu. ...
5
votes
2answers
230 views
How would I sort these directory names numerically?
I'm aware that I can somehow sort this output numerically (so cpu1/ follows cpu0/) ... I could probably get something to work eventually by splitting up the string various ways with awk, but is there ...
2
votes
2answers
580 views
Concatenating columns horizontally. Printing only rows that intersect on 1st column
Say I have two different commands that output contents by columns, e.g.:
$ command_1
7049857 abc fdg hsi
5409858 xxx fyy hsi
540958 abc zzz hsi
54230956 rbc sss hsi
$ command_2
7049857 0 fdg ...
2
votes
4answers
646 views
gawk join two TSV by columns (a'la sql join)
How to join, to tsv files, examples:
a.tsv
c 7 r z
d 6 s w
f 1 f f
b 8 p y
a 9 q x
b.tsv
a q a
c r ccc
b p bb
0 0 0
d s dddd
Here I'd like to ...
1
vote
4answers
940 views
Increment number in bash variable string
VERSION="4.0beta12"
NEXT_VERSION=?????
Writing an automatic release script, I have the current version in a variable $VERSION, and I would like to calculate the next version as $NEXT_VERSION.
Rule: ...
2
votes
1answer
439 views
How to read a hexdump byte by byte in bash or awk?
This is the hexadecimal output of an IPv6 TCP packet captured with tcpdump:
6000 0000 0018 0620 0000 0000
0000 0000 0000 0000 0000 0001 0000 0000
0000 0000 0000 0000 0000 0002 *0026 0026
0000 ...
0
votes
6answers
1k views
How do use awk along with a command to show the process ID with the 'ps' command
I'm able to to show the PID (2nd) column with the below command, but instead I want to show pwdx <PID> and also the COMMAND(8th) column. Thanks in advance, Chirag
ps -ef | grep runner | awk ...
3
votes
1answer
205 views
Relative line offsets in awk (/pattern/+1 or /pattern/-1)
When I want to print lines between two patterns, excluding the lines with those patterns, I can do it in ex using +1 and -1 after the pattern. That is awesome. Is this possible in awk? Right now, I ...
3
votes
3answers
801 views
Show lines matching a pattern and the 4 lines before each
For example, from this file:
CREATE SYNONYM I801XS07 FOR I8010.I801XT07
*
ERROR at line 1:
ORA-00955: name is already used by an existing object
CREATE SYNONYM I801XS07 FOR ...
1
vote
3answers
488 views
How let `sort | uniq -c` separate the number of occurrences by a tab charactor?
uniq -c separate the occurrence number by spaces, which is hard for to cut or awk to separate it out later.
1000_A1\tB1\n
___1_A2\tB2\n
I can solve this problem using sed -r 's/^ *([0-9]+)/\1\t/' ...
5
votes
2answers
826 views
How to use regex as field separator in awk?
I'm trying to use regex as a field seperator in awk. From my reading this seems possible but I can't get the syntax right.
rpm -qa | awk '{ 'FS == [0-9]' ; print $1 }'
awk: cmd. line:1: { FS
awk: ...
1
vote
1answer
141 views
What does this awk script do
echo 1 | awk '{ srand(); print int (rand()*67) }'
I get a different o/p everytime I run it. However if I omit the 1 I get same o/p ..why?
5
votes
1answer
90 views
Inderect references aka value of a value in awk
I'm searching for a way to solve this problem in awk. Input:
X 1
Y 2
Z 3
X 4
Y 5
The output should look like this:
X 5
Y 7
Z 3
I see two problems here: the first one is the indirect reference in ...
2
votes
3answers
173 views
How to substitute awk argument?
I want simplify awk command for common usage so instead of writing awk '{print "rm -r"$4 }' each time I want to write myawk "rm -r"$4.
I've tried to write such function
myawk() { awk '{ print $1 }' ...
3
votes
3answers
165 views
What does . match?
In working with regular expressions, I have been told that a dot character . will match everything.
Except for newlines \n.
Are there any other exceptions? What about the NUL character \0, or the ...
1
vote
1answer
369 views
How to manipulate awk variable NF in zsh?
I have a simple command line:
uptime | awk '{print $(NF - 2)}'
In a bash shell, this works without a hitch, but when I run it in zsh, I get the following error:
title:5: command not found: NF
As ...
3
votes
1answer
141 views
Help improving my AWK skills
This program works as I intended but I feel like I used some clumsy methods to get the out put I desired(especially with my use of print commands and variable declarations)
Could some one improve ...
1
vote
2answers
139 views
Parsing giant files into Ruby
I have a CSV that is 40 000 lines long what is the simplest tool to read from line x to line y. I am sure there are more professional ways to do it but I would just like a bash script that works in ...
1
vote
3answers
562 views
How to awk the “df -Pm” command to get FS names that have more then 90% the FS usage?
I'm trying to get the names of all disks with at least 90% usage, using this oneliner:
df -Pm | awk '+$5 >= 90 {print}'
But it doesn't show the "100% usage" case, I need to grep for "100%". Why ...
3
votes
3answers
427 views
Using sed to edit lines in a file with a variable
I need to edit lines in a file using sed. Now the problem is I am replacing a particular pattern with a combination of text and number. This number is a variable which keeps on incrementing for every ...
5
votes
1answer
344 views
Why are capital letters included in a range of lower-case letters in an awk regex?
$ echo ABC | awk '$0 ~ /^[a-b]/'
ABC
$ echo ABC | awk '$0 ~ /^[a-a]/'
$ echo ABC | awk '$0 ~ /^a/'
$
You see. /[a-b]/ captures A, but /[a-a]/ or /a/ doesn't. Why?
4
votes
2answers
394 views
How to test if a function is defined in awk?
I am writing an awk program and I want it to run it both with mawk and gawk.
I need a sorting function. I have written my own, but I want to use the builtin asort() function in gawk if available.
How ...
4
votes
4answers
527 views
How to echo an escaped string
How can I echo an escaped string that contains $ in Bourne Shell?
user@server:~$ cat test.sh
#!/bin/sh
echo $1
user@server:~$ ./test.sh \$sad\$test
$sad$test
I want it to return an escaped ...
2
votes
1answer
199 views
Awk : Range of PIDs
I am learning awk from this tutorial. It's quite basic.
I have a list of processes in a file which I got by doing ps aux > processes Now according to the tutorial, doing awk '$2 ~ 14022, $2 ~ ...
5
votes
7answers
4k views
How to print only last column?
echo -e 'one two three\nfour five six\nseven eight nine'
one two three
four five six
seven eight nine
how can I do some "MAGIC" do get this output?:
three
six
nine
UPDATE:
I don't need it in this ...
3
votes
3answers
960 views
In awk, how to print a range of fields with the original $0 delimiters intact?
How to extract a substring of $0, in awk, when that substring's position and length is based on the positions of fields in the original $0?
It effectively equates to removing the first n.lead fields ...
2
votes
2answers
702 views
How would I extract the user agent strings from a log file?
Currently I am running a command like this, to get the most requested content:
grep "17\/Jul\/2011" other_vhosts_access.log | awk '{print $8}' | sort | uniq -c | sort -nr
I want to now see the user ...
