awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

3
votes
2answers
2k views

use awk to replace parentheses?

How can I use awk to replace the parentheses in these 2 scenarios? Change empty parens to the word "None" 192.168.0.24,Up,() Remove parens to only host.domain.com is left ...
0
votes
2answers
1k views

nmap script needed to output text into csv format

ultimately I am looking to insert some info into a database. So I'm trying to output a file from something (probably nmap) that is formatted like this: 10.1.150.1,Up,1/3/2012,host.domain.lan ...
2
votes
1answer
1k views

Diff the output of two `awk` commands

I'm trying to compute the difference between the output of two awk commands but my simple attempts at it seem to be failing. Here is what I'm trying: diff $(awk '{print $3}' f1.txt | sort -u) $(awk ...
14
votes
4answers
20k views

How to manipulate a CSV file with sed or awk?

How can I do the following to a CSV file using sed or awk? Delete a column Duplicate a column Move a column I have a big table with over 200 rows, and I'm not that familiar with sed.
1
vote
1answer
438 views

Reading two files into an IFS while loop — Is there a way to get a zero diff result in this case?

I have a text file full of several hundred lines of sequences like this: b 29. b 52. c 84. c 83. c 94. c 93. c 61. b 38. c 81. c 92. c 28. c 37. ...
1
vote
2answers
264 views

How can I match this text in awk, printing line numbers for matches?

I have a text file thousands (roughly 148,000 lines long) that consists of a lot of sequences like this: b 29. b 52. c 84. c 83. c 94. c 93. c 61. b 38. c 81. c 92. c 28. c 37. c 27. ... and since ...
4
votes
6answers
1k views

Grep a directory and return list with line numbers

I'm currently trying to learn more about bash scripting and all of that fun stuff, and I pieced together this little command: find $path | xargs grep -n $pattern | awk '{print $1}' While this DOES ...
2
votes
1answer
383 views

Getting data section with maximum element

I have some data stored in JSON-like format. { { value1: 14, value2: 12, value3: 1 }, { value1: 4, value3: -1 } } How can I get subsection ...
10
votes
3answers
4k views

When using awk /pattern/ { print “text”} /patern/ {print “”} is there an ELSE pattern?

Let's say I have text file like: R1 12 324 3453 36 457 4 7 8 R2 34 2342 2525 25 25 26 26 2 2 R3 23 2342 32 52 54 543 643 63 R4 25 234 2342 4 234242 I want to use awk to process these lines ...
8
votes
6answers
2k views

Delete lines beginning with #

How do I delete lines beginning with a #, given that there can be whitespace on the left and right of the #? # Master socket provides access to userdb information. It's typically
2
votes
3answers
733 views

Clean up concatenated XML files

I'm trying to fix my Delicious export script to work with the new "trick" of only serving 1000 bookmarks at a time. The script concatenates results (XML) from multiple calls in a single file, and now ...
2
votes
3answers
2k views

Trouble getting awk output in for loop

I'm trying to create a script that will check a website for a word. I have a few to check so I'm trying to input them via another file. The file is called "testurls". In the file I list the keyword ...
6
votes
5answers
6k views

How to print certain columns by name?

I have the following file: id name age 1 ed 50 2 joe 70 I want to print just the id and age columns. Right now I just use awk: cat file.tsv | awk '{ print $1, $3 }' However, this ...
3
votes
0answers
131 views

How to use regrex with AWK for string replacement in this example? [duplicate]

Possible Duplicate: How to use regrex with AWK for string replacement in this example? Suppose there is some text from a file: (bookmarks ("Chapter 1 Introduction 1" "#1" ("1.1 Problem ...
6
votes
3answers
26k views

How to use regex with AWK for string replacement?

Suppose there is some text from a file: (bookmarks ("Chapter 1 Introduction 1" "#1" ("1.1 Problem Statement and Basic Definitions 23" "#2") ("Exercises 31" "#30") ("Notes and References 42" "#34")) ) ...
1
vote
2answers
474 views

pattern search and display the last occuring pattern?

I have a log file containing startup and shutdown times for everday. I want to see the last pattern pattern for startup and shutdown(which is located at the end of the file being updated everyday). ...
11
votes
7answers
6k views

How to print the longest line in a file?

I'm looking for the simplest method to print the longest line in a file. I did some googling and surprisingly couldn't seem to find an answer. I frequently print the length of the longest line in a ...
2
votes
2answers
711 views

Adding 2 lines in text file stanza

I have a text file divided in stanzas. Each stanza starts with a [keyword] and ends with a blank line. Something like this: [keyword1] param1 = value1 param2 = value2 param3 = value3 [keyword2] ...
1
vote
3answers
654 views

To list out duplicate filenames in a listing?

bash-3.00$ cat f.txt -rw-r--r-- 1 mukesh other 102 Nov 5 18:32 f1.txt -rw-r--r-- 1 mukesh other 19 Nov 5 18:32 f2.txt -rw-r--r-- 1 mukesh other 204 Nov 5 18:32 ...
3
votes
4answers
536 views

Find a string (like grep -q) within only a section of a file

I want to write some Bash that can verify that a string exists in a configuration file. I can't change the file format, it belongs to a different application. The file is subdivided into groups named ...
-2
votes
2answers
297 views

How to delete the second column from this input?

INPUT bix.hu. 42433 IN A 193.239.149.1 bix.hu. 42433 IN MX 10 deneb.iszt.hu. bix.hu. 42433 IN NS ns.iszt.hu. bix.hu. 42433 IN NS ns.iszt.hu. bix.hu. ...
6
votes
2answers
558 views

How would I sort these directory names numerically?

I'm aware that I can somehow sort this output numerically (so cpu1/ follows cpu0/) ... I could probably get something to work eventually by splitting up the string various ways with awk, but is there ...
2
votes
2answers
906 views

Concatenating columns horizontally. Printing only rows that intersect on 1st column

Say I have two different commands that output contents by columns, e.g.: $ command_1 7049857 abc fdg hsi 5409858 xxx fyy hsi 540958 abc zzz hsi 54230956 rbc sss hsi $ command_2 7049857 0 fdg ...
2
votes
4answers
985 views

gawk join two TSV by columns (a'la sql join)

How to join, to tsv files, examples: a.tsv c 7 r z d 6 s w f 1 f f b 8 p y a 9 q x b.tsv a q a c r ccc b p bb 0 0 0 d s dddd Here I'd like to ...
2
votes
4answers
3k views

Increment number in bash variable string

VERSION="4.0beta12" NEXT_VERSION=????? Writing an automatic release script, I have the current version in a variable $VERSION, and I would like to calculate the next version as $NEXT_VERSION. Rule: ...
2
votes
1answer
610 views

How to read a hexdump byte by byte in bash or awk?

This is the hexadecimal output of an IPv6 TCP packet captured with tcpdump: 6000 0000 0018 0620 0000 0000 0000 0000 0000 0000 0000 0001 0000 0000 0000 0000 0000 0000 0000 0002 *0026 0026 0000 ...
1
vote
6answers
4k views

How do use awk along with a command to show the process ID with the 'ps' command

I'm able to to show the PID (2nd) column with the below command, but instead I want to show pwdx <PID> and also the COMMAND(8th) column. Thanks in advance, Chirag ps -ef | grep runner | awk ...
3
votes
1answer
346 views

Relative line offsets in awk (/pattern/+1 or /pattern/-1)

When I want to print lines between two patterns, excluding the lines with those patterns, I can do it in ex using +1 and -1 after the pattern. That is awesome. Is this possible in awk? Right now, I ...
4
votes
3answers
1k views

Show lines matching a pattern and the 4 lines before each

For example, from this file: CREATE SYNONYM I801XS07 FOR I8010.I801XT07 * ERROR at line 1: ORA-00955: name is already used by an existing object CREATE SYNONYM I801XS07 FOR ...
1
vote
3answers
1k views

How let `sort | uniq -c` separate the number of occurrences by a tab charactor?

uniq -c separate the occurrence number by spaces, which is hard for to cut or awk to separate it out later. 1000_A1\tB1\n ___1_A2\tB2\n I can solve this problem using sed -r 's/^ *([0-9]+)/\1\t/' ...
6
votes
2answers
2k views

How to use regex as field separator in awk?

I'm trying to use regex as a field seperator in awk. From my reading this seems possible but I can't get the syntax right. rpm -qa | awk '{ 'FS == [0-9]' ; print $1 }' awk: cmd. line:1: { FS awk: ...
1
vote
1answer
246 views

What does this awk script do

echo 1 | awk '{ srand(); print int (rand()*67) }' I get a different o/p everytime I run it. However if I omit the 1 I get same o/p ..why?
5
votes
1answer
120 views

Inderect references aka value of a value in awk

I'm searching for a way to solve this problem in awk. Input: X 1 Y 2 Z 3 X 4 Y 5 The output should look like this: X 5 Y 7 Z 3 I see two problems here: the first one is the indirect reference in ...
2
votes
3answers
292 views

How to substitute awk argument?

I want simplify awk command for common usage so instead of writing awk '{print "rm -r"$4 }' each time I want to write myawk "rm -r"$4. I've tried to write such function myawk() { awk '{ print $1 }' ...
3
votes
3answers
192 views

What does . match?

In working with regular expressions, I have been told that a dot character . will match everything. Except for newlines \n. Are there any other exceptions? What about the NUL character \0, or the ...
1
vote
1answer
527 views

How to manipulate awk variable NF in zsh?

I have a simple command line: uptime | awk '{print $(NF - 2)}' In a bash shell, this works without a hitch, but when I run it in zsh, I get the following error: title:5: command not found: NF As ...
3
votes
1answer
168 views

Help improving my AWK skills

This program works as I intended but I feel like I used some clumsy methods to get the out put I desired(especially with my use of print commands and variable declarations) Could some one improve ...
1
vote
2answers
194 views

Splitting giant files into a specific number of lines

I have a CSV that is 40 000 lines long. What is the simplest tool to read from line x to line y? I am sure there are more professional ways to do it but I would just like a bash script that works in ...
1
vote
3answers
1k views

How to awk the “df -Pm” command to get FS names that have more then 90% the FS usage?

I'm trying to get the names of all disks with at least 90% usage, using this oneliner: df -Pm | awk '+$5 >= 90 {print}' But it doesn't show the "100% usage" case, I need to grep for "100%". Why ...
3
votes
3answers
2k views

Using sed to edit lines in a file with a variable

I need to edit lines in a file using sed. Now the problem is I am replacing a particular pattern with a combination of text and number. This number is a variable which keeps on incrementing for every ...
6
votes
1answer
541 views

Why are capital letters included in a range of lower-case letters in an awk regex?

$ echo ABC | awk '$0 ~ /^[a-b]/' ABC $ echo ABC | awk '$0 ~ /^[a-a]/' $ echo ABC | awk '$0 ~ /^a/' $ You see. /[a-b]/ captures A, but /[a-a]/ or /a/ doesn't. Why?
4
votes
2answers
704 views

How to test if a function is defined in awk?

I am writing an awk program and I want it to run it both with mawk and gawk. I need a sorting function. I have written my own, but I want to use the builtin asort() function in gawk if available. How ...
4
votes
4answers
2k views

How to echo an escaped string

How can I echo an escaped string that contains $ in Bourne Shell? user@server:~$ cat test.sh #!/bin/sh echo $1 user@server:~$ ./test.sh \$sad\$test $sad$test I want it to return an escaped ...
2
votes
1answer
258 views

Awk : Range of PIDs

I am learning awk from this tutorial. It's quite basic. I have a list of processes in a file which I got by doing ps aux > processes Now according to the tutorial, doing awk '$2 ~ 14022, $2 ~ ...
6
votes
7answers
13k views

How to print only last column?

echo -e 'one two three\nfour five six\nseven eight nine' one two three four five six seven eight nine how can I do some "MAGIC" do get this output?: three six nine UPDATE: I don't need it in this ...
3
votes
3answers
1k views

In awk, how to print a range of fields with the original $0 delimiters intact?

How to extract a substring of $0, in awk, when that substring's position and length is based on the positions of fields in the original $0? It effectively equates to removing the first n.lead fields ...
3
votes
2answers
2k views

How would I extract the user agent strings from a log file?

Currently I am running a command like this, to get the most requested content: grep "17\/Jul\/2011" other_vhosts_access.log | awk '{print $8}' | sort | uniq -c | sort -nr I want to now see the user ...
2
votes
2answers
382 views

maximum of column with matching id

I have a file with the following fields NM001 rp1 210 NM003 rp1 220 NM005 rag 200 NM004 rag 100 I would like to have an output that for every change in column2, find maximum in column4 but also ...
1
vote
1answer
85 views

awk expand window to 500% for two numbers from column file

I've got a file with two columns corresponding to start-end ranges like this: awk '{print $2"-"$3}' file.txt 91082802-91082990 94018177-94018321 146945732-146945949 88701397-88701594 ...
4
votes
1answer
2k views

awk: forcing a return status?

This is a followup to my earlier question. I am validating the number of fields in /etc/passwd using this handy snippit. In the following example, the users 'fieldcount1' and 'fieldcount2' have the ...