Questions tagged [awk]

A pattern-directed scanning and processing language.

Filter by
Sorted by
Tagged with
2
votes
2answers
270 views

How to make a list and count LaTeX keywords in a source file?

I am trying to make lists (and eventually count and make statistics) the LaTeX keywords appearing in some documents. Basically a LaTeX "valid" keyword can be identified by: The fact that it does ...
2
votes
3answers
1k views

How can I turn ugly output into pretty, useful information?

How can I turn this ugly output into pretty, useful data? The output: /* ---------- TA#box#AbC_p ---------- */ insert_job: TA#box#AbC_p job_type: a #owner: bob permission: gx date_conditions: 1 ...
1
vote
2answers
180 views

How to parse fields out of a text file and write them to another file

I have executed a database command and have generated an output file of the results. Each field name starts with dbt_xxxxxx Each field value is the value after the equal sign. how can I parse the file ...
4
votes
3answers
981 views

How to parse a file to extract 3 digits numbers kept in a “group number”

I am writing a shell script to parse a text file that is extracted from a standardization pdf file. I would like for each test group (identified by Group 0, Group 1... to get the list of test numbers,...
1
vote
1answer
2k views

AWK - change the file name (special characters)

I have two files: ~/Pulpit/kot$ find . -name "*jpg" -printf "%f\n" 1.jpg `[~!($%^_&*){.}\___"`]`1.jpg I want to change their name to: _home_pic_Pulpit_kot_1.jpg _home_pic_Pulpit_kot_`[~!($%^_&...
9
votes
1answer
10k views

escaping a single dot with double backslash - awk

"effective awk programming" book has an example on Field-Splitting. here is the example: If you want fields to be separated by a literal period followed by any single character, use ‘FS = "\\.."’....
13
votes
3answers
21k views

How to sed only that lines that contains given string?

INPUT: Select ASDF 325 sdfg sdflk lk Select TRG 46sdg rasdftz fsgs 45 Select ASDF 6ffg sdfg 4456 sdrg OUTPUT: Select ASDF 325 XXXX sdflk lk Select TRG 46sdg rasdftz fsgs 45 Select ASDF 6ffg XXXX ...
9
votes
4answers
3k views

Sort fields inline

I'm trying to sort within a line of input over an unknown number of fields: Input: ab bc bc ab cd ef bc bc cd ef cd bc ab ef ab bc cd gh Output: ab bc ab bc bc cd ef bc cd ef ab cb cd ab bc cd ef ...
1
vote
3answers
6k views

Using awk to process ls output with spaces in filenames/paths

All, I have a script in which I use find command to filter out all png files in a given folder and list them along with their size. I want the output in the following format: someFile.png => 1.2K ...
4
votes
3answers
20k views

How to delete files filtered out by awk

I have the following files in a directory: -rw-r--r-- 1 smsc sys 46 Apr 22 12:09 bills.50.1.3G.MO.X.20120422120453.Z -rw-r--r-- 1 smsc sys 28 Apr 22 12:15 bills.50.1.3G.MO.X.20120422120953.Z -rw-r--...
2
votes
1answer
3k views

file, awk and iconv to automatically adapt tex-file encoding

I have a set of tex-files with mixed encodings, e.g. (subset of output of file -i *.tex) f1.tex: text/plain; charset=utf-8 f2.tex: text/plain; charset=utf-8 f3.tex: text/x-tex; charset=us-ascii f4....
3
votes
3answers
530 views

swapping fields in columns by fixed pattern (yes, no, no, yes)

I want to produce a following list of pages (for imposition work); (number of pages is variable, but I make a sample of my needs using a list of 16 pages, logic is the same for longer lists) 1,16 8,9 ...
5
votes
4answers
9k views

How to list all unique ip address currently connected to a specific port?

Say I want to know how many unique clients are connected to port 5222 on a server. Can you find a way better/faster/stronger than this? netstat -nt | grep ':5222.*ESTABLISHED' | awk '{ print $5 }' \ ...
0
votes
1answer
1k views

Using Awk To Parse Individual Lines

Here is the output from qacct -o Total : OWNER WALLCLOCK UTIME STIME CPU MEMORY IO IOW =========================================================================================================...
2
votes
2answers
4k views

How can I delete lines that has an empty first column?

INPUT: CC2352345|m,safnlasndfmnyxdcvyxcvyxcv |klasjdf |3|lasdjflasdf| |2345567356 CC3543353|asdfasdffghntz |klasjdf |3|lsajdfl | ...
4
votes
2answers
1k views

Why does this awk command not play as well with find as sed does?

I'm creating a Bash shell script to do some file manipulation on some HTML files. Among the many actions I do, I have this sed command which works great: find $DIR -type f -name '*.html' -exec sed -...
2
votes
3answers
10k views

Combining Awk with Curl

How would I set my curl statement so that it puts the value of each $3 in for the username, and the value of each $4 for the wallclock information? Some further information I am trying to take the ...
2
votes
1answer
9k views

Use AWK add new line if match exists

I have a file with the following format: aaa@ bbb bbb@ ccc& ddddd@ aaaf ccc@ ddd& eee: abcd& dassda The output should be: aaaa@ bbb bbb@ ccc& ddddd@ aaaf ccc@ ddd& eee@ abcd&...
2
votes
2answers
1k views

Turn a bash and php script into a distributable rpm

I'm semi unix knowledgeable and I want to be able to call down an rpm from my server, run it with something like rpm -i , and have access to the script anywhere. The script is a combination awk and ...
3
votes
2answers
2k views

simple script rotating table

I'm trying to write a script, that just rotates csv table. I mean I have some file: head1;head2;head3 field11;field12;field13 and so on. All I want, just to make my file head1;field11;field21 head2;...
2
votes
2answers
4k views

How to divide a list of values by a number in command line?

I am trying to translate a simple program to the command line using unix utilities. For example, if I have a frequency list (after piping through uniq and sort) 5 x 4 y 1 z I want to print out, ...
10
votes
2answers
1k views

Are there 2 ways to set awk vars via command line?

I noticed an O'Reilly awk example (1997) which assigned an awk variable by setting it on the command line after the program-text. It does work, but I can't find this syntax in man / info awk. Have I ...
1
vote
2answers
506 views

Why does awk print on non-null strings and positive numbers

I've noticed some awk examples which use 1 instead of print to print $0 (eg. To conserve space, I normally use '1'. and on this site). Is this a documented / safe practice, or is is it subject to ...
23
votes
2answers
17k views

Why does awk do full buffering when reading from a pipe

I'm reading from a serial port connected to a gps device sending nmea strings. A simplified invocation to illustrate my point: $ awk '{ print $0 }' /dev/ttyPSC9 GPGGA,073651.000,6310.1043,N,...
2
votes
2answers
2k views

splitting the array and send the values to hashset in awk

I am getting var as argument from shell script to awk awk -v var = "blah1|blah2|blah3" -f awk_script.awk I need to put each blah's into a hashset, currently, I am doing it like this. I am getting ...
2
votes
1answer
2k views

remove trailing zeros in awk not working. syntax error

regex = "\\.*0+$"; subst = ""; system("echo "id "| awk '{sub(\\.*0+$," subst"); print}'"); It is giving the following error: awk: cmd. line:1: {sub(\.*0+$,); print} awk: cmd. line:1: ^ ...
2
votes
3answers
8k views

Faster awk script to get the substring / string we wanted

ORDER EVENT .........[] [] ... so many other tags... [Account<25106>=ACCT1] [Destination...] .. so many other tags. I am currently trying to get the account like this. I tried using match in ...
39
votes
10answers
64k views

How to insert the content of a file into another file before a pattern (marker)?

File1 contents: line1-file1 "1" line2-file1 "2" line3-file1 "3" line4-file1 "4" File2 contents: line1-file2 "25" line2-file2 "24" Pointer-file2 "23" line4-...
0
votes
1answer
708 views

Shell script to check server state

I am writing a script that would ssh into the VMWare server and check if the VMs are up or not. There's some error in the server=awk "/$i/{print $2}" /home/dharmit/scripts/vmids line. I am getting ...
8
votes
2answers
6k views

A shell script for joining two files

I want to write a shell script that get two files A and B, and get a result like this: File A: user_a tel_a addr_a user_b tel_b addr_b File B: process_1 user_a process_2 user_a process_3 user_b ...
2
votes
3answers
2k views

Sorting columns from two files with awk

I have two input files 28a39a289906c01159f999a68996091a tom@dong.com 274d1d2c7e931fb55ac0c91dd41f2be7 go@far.com 44d25d3b1b70b240d5058f1be1cef576 person@domain.com 2227a768f6d253b7bf81bb4ecb15b52d ...
3
votes
5answers
1k views

changing pattern of a text file

A text file has contents something like chair table pen desk Now i want it to be changed and stored in a variable say var as below ('chair','chair'),('table','table'),('pen','pen'),('desk','desk') ...
3
votes
3answers
12k views

Awk: check for length of field

In awk. I am working on Solaris 10, so it's probably an old(er) version of awk. I came up with this rudimentary one-liner that works, at least for my particular input. awk -F\; '$3 ~ /[ ]*...............
8
votes
3answers
15k views

How to apply the same awk action to different files?

I am new in awk and I do not know if it is possible to write an awk script that does this: I have hundred of data files that I have to sort. For each one I use the following one-liner: awk 'ORS=NR%3?...
3
votes
1answer
1k views

merging files and getting column values based on id field

bash-3.2$ cat sample.log sample.log.1 sample.log.2 ID COL1 COL2 COL4 1 col1 col2 col4 2 c1 c2 c4 3 co1 co2 co4 ID COL3 COL1 1 col3 col1 2 c3 c1 3 co3 co1 ID COL1 COL2 COL3 1 col1 ...
43
votes
5answers
206k views

How to run grep on a single column?

I want to grep the output of my ls -l command: -rw-r--r-- 1 root root 1866 Feb 14 07:47 rahmu.file -rw-r--r-- 1 rahmu user 95653 Feb 14 07:47 foo.file -rw-r--r-- 1 rahmu user ...
1
vote
2answers
660 views

$2 (field reference) in awk BEGIN is not working

In the following snippet, $2 in awk is returning empty. What am I doing wrong? I am trying to find the difference between MAX and MIN. #!/bin/ksh if [ $# -ne 1 ]; then echo "Usage: sh `...
5
votes
3answers
2k views

Is there a more elegant way to count words and assign that count to variables?

I have a script: #!/bin/bash /root/xiotech status > xiostatus.tmp SyncCount=$(grep -c Sync xiostatus.tmp) PauseCount=$(grep -c paused xiostatus.tmp) CopyingCount=$(grep -c Copying xiostatus.tmp) ...
2
votes
6answers
5k views

awk separate columns of 3

I have a file that looks like the following (the columns are separated by tabs and might contain spaces): 0637 apple 8528 1392 orange 1390 8528 unknown fruit 7537 1039 ...
4
votes
1answer
705 views

Efficient way of comparing in awk

#!/bin/awk BEGIN { while(getline var < compareTo > 0) { orderIds[var]=var; } } { if(orderIds[$0] == "") { print $0; ...
2
votes
2answers
1k views

xargs grep suggestion

grep -v "\<Swap" instruments.log | awk '{ idx=index($0, "MasterId="); masterId=substr($0, idx+length("MasterId=")+1); masterId=substr(masterId,1,index(masterId,"L")-3); print masterId; }' | xargs ...
5
votes
2answers
2k views

Replace one line with STDIN in a Makefile

One Makefile target is supposed to be in charge of: grabbing the output of $ perl Markdown.pl src/index.md (markdown to html) using that string to replace one line, something like CONTENT originating ...
2
votes
3answers
3k views

How to truncate the second column from a dot (.)?

INPUT: 10.175.192.16 vgxyxgf.de.foo.net asdf 34efg 10.175.192.17 sdtds.de.foo.net xyyxv e5432 10.175.243.20 asdfycfb.us.foo.net dfvasf 34525 OUTPUT: 10.175.192.16 vgxyxgf 10.175....
4
votes
5answers
626 views

Align a hex text file at 9 bytes each line

I want to align the selected portion of a text file at 9 bytes each column. For example suppose my text file looks like below. 00 2f c6 b8 29 fd 02 37 11 00 9f 74 34 0b 60 72 38 20 00 9e 61 33 8e ...
6
votes
3answers
11k views

Get lines with maximum values in the column using awk, uniq and sort

I have a file with next format 2011-12-01 user1 access1 2011-12-01 user1 access2 2011-12-01 user2 access2 2011-12-01 user4 access2 2011-12-02 user1 access1 2012-01-01 user3 access1 2012-01-01 user4 ...
28
votes
5answers
45k views

Delete range of lines above pattern with sed (or awk)

I have the following code that will remove lines with the pattern banana and 2 lines after it: sed '/banana/I,+2 d' file So far, so good! But I need it to remove 2 lines before banana, but I can't ...
30
votes
12answers
32k views

How can I write to the second line of a file from the command line?

I have an external program that produces an output file (largish, 20K lines possible). I need to insert a new line between the existing line 1 and line 2. I've been looking at awk and sed - I use ...
18
votes
11answers
26k views

Grep huge number of patterns from huge file

I have a file that's growing about 200,000 lines a day, and it is all formed with blocks of three lines as such: 1358726575123 # key Joseph Muller # name carpenter # job ...
54
votes
1answer
35k views

Difference between gawk vs. awk

Trying to understand the differences between the two functions gawk vs. awk? When would one use gawk vs awk? Or are they the same in terms of usage? Also, could one provide an example?
4
votes
2answers
2k views

Moving average on a log file with awk or other unix utilities?

Scenario: I have a log file that has a few number of entry "classes", like this: R0 dx=0.00500 rb=0.00000 sn=1 3145.88 2.59 0.08 se=21315 id=16190 R0 dx=0.00300 rb=-1.00000 sn=1 3150.40 2.38 0.05 ...