awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
1answer
63 views

AWK, Sum of category

I have tons of CSV files with similar content. The values are usually comma separated and they look like this. product_a, domestic, 500 product_a, abroad, 15 product_b, domestic, 313 ...
0
votes
1answer
33 views

subtract previous line based on another column

I'm trying to figure out how to subtract previous line based on another column using AWK. Here is the input: site year count 040130013 1997 34 040130013 1998 55 040130019 2006 79 040130019 2007 135 ...
-3
votes
2answers
47 views

linux + convert file format columns to one column

I have the following file (Linux.txt) linux-test01 linuxER LinuxCV LinuxWE LinuxPL LinuxDF LinuxXC LinuxSD LinuxAQ how to print using awk or perl one-liner to produce the following ...
1
vote
4answers
84 views

Impose character between repeated characters

Given a string of characters ,,a,,,b,,,, I want to impose the character - at the beginning, if no character is preceded comma, and between every comma that has no character. Meaning the result is -,-,...
1
vote
2answers
59 views

Process several files and print filename without extension and the value of a specific key

I have ASCII file and I want to output the file name and a specific field from this file beside each other. For example: The contents of the ASCII file "data1.txt" is as follow: Name: John Age: 90 ...
3
votes
1answer
32 views

Awk Search - Combined Character Classes in Same String

Good Evening, I am attempting to utilize multiple character classes at the same time without success. Given a column that contains the URI of HTTP traffic, I want the REGEX to identify lines that ...
3
votes
1answer
38 views

Splitting a large file by column with values in the header as file names

I have a large tab-delimited text file which contains a header and data. I want to split the file into smaller files by column, with the header excluded, and the files named as the column name. pos ...
0
votes
2answers
44 views

Awk: Print last N columns, where N is passed through a variable

My input variable $dirPath contains directory path. The action is to be able to retrieve the last N values in the directory path, where N is passed through a variable $depth. For a fixed N value (say ...
0
votes
1answer
55 views

Compare strings with specialized charcters

If I have a lookup table cat tmp1 [//rtwttwtr*fgg]\\\erw``~ 1 ^774574574565665f[[[//]\] 2 ()42543^[[D^[[D^[[D^[[D^[[D353535345****@3242- 3 and a number of strings to be looked up from the first ...
0
votes
1answer
57 views

Redirect output of: yum repolist all

If you run the command: yum repolist all you will see there are basically four columns of data. repo id, repo name, status, install count. If however your run something like: yum repolist all > ...
2
votes
4answers
359 views

How can I split a line into two lines if the length is greater than 7 using awk? [closed]

For example I just wanted to print something like this in the command line. Let say I have a file called file.txt. What is life? how are you? hi whatup this is more than And I want to print it ...
0
votes
2answers
57 views

How to determine what IP addresses are available in a hosts table?

I'm working on a script that will read from the /etc/hosts file and determine what IPs are available in the hosts file. The way the hosts file will look is as followed: 172.17.164.2 System1 172.17....
3
votes
2answers
51 views

Simple awk wrapper function or alias

I had seen someone use a bash alias or function based on awk, which I liked for its simplicity, but can't remember how to implement it (in bash). Lets say you have a large formatted file, from which ...
0
votes
3answers
69 views

Right syntax for awk usage in combination with other command inside xargs sh –c

How to make this command work: ls * | xargs -I {} sh -c 'echo {}; awk '{print $1}' {} | uniq' It should do the simple thing: print for each file in the folder its name and uniq values in the first ...
1
vote
1answer
48 views

Filter out duplicate lines of tail -f [duplicate]

I often need to tail -f apache access logs for websites to troubleshoot issues- one thing that makes it annoying is that anyone loading a page once may cause 12+ lines to get written to the log, and ...
2
votes
2answers
77 views

Justifying text using awk

I am trying to format my output file so that text in lines is spreaded evenly, to align it both left and right. To do left and right justifying is trivial, but how can one do both left and right with ...
0
votes
1answer
18 views

Log from a specific line to another specific line [duplicate]

I am doing a data monitoring and I am doing a cron job to get the data everyday. But I am receiving many junk data with my actual data that makes the file huge. For example my data is, eg.hh.jk.ll.mm ...
0
votes
0answers
41 views

find awk log print error

I'm trying to find one year old directories named in yyyymmdd format and print it in the log file placed at different location. find ${LOG} -maxdepth 1 -type d -mtime +${DAY} -name '20[0-9][0-9][0-9][...
1
vote
0answers
29 views

Paging with follow less +F and colorize with awk

I would like to paginate a file with follow, i.e. $less +F and also colorize the output on the screen. I want to use less because I can leave the follow mode, browse the file and then re-enter the ...
0
votes
1answer
25 views

sed - match on IP octet and remove line

I am in need of a solution, preferably sed, that can match on an IP octet that contains any number combination and remove the line. For example: My file: 192.168.100.12 192.168.200.12 10.10.20....
0
votes
1answer
25 views

Linux red hat + replace PATH or word on scripts

Under folder: /var/userTN/applo folder I have around ~100 shell scripts that include the following PATH: GLOBAL_PATH=/export/pop/PLW5 I want to replace the PATH on all the scripts under folder: ...
0
votes
1answer
21 views

Using awk to change an entire column of both positive and negative values, to just negative? [duplicate]

I use awk alot to manipulate xyz text files (lat,long,elevation). I have a specific file that has both positive and negative values in the z column, I would like all of these to be negative. So {$3*-1}...
1
vote
4answers
125 views

How to add a new text line at the first line of a file? [duplicate]

File: TABLE1 1234 9555 87676 2344 Expected output: Description of the following table: TABLE1 1234 9555 87676 2344
2
votes
2answers
140 views

How do I double each line of piped output? [duplicate]

How do I double each line of input piped in? Example: echo "foobar" | MYSTERY_COMMAND foobar foobar
-1
votes
2answers
34 views

errors using alias for an awk command

I'm attempting to add this alias to my .zshrc but run into errors. I've tried escaping the ==, ", and && values and still no luck. What am I missing? alias broke="ssh -t server@5.555.555.555 ...
2
votes
1answer
59 views

Can I determine type of an awk variable?

I have the gawk version of awk. In this part of gawk manual, it is stated that awk variables have "attributes", which are used to determine how to treat them in various operations. For example, a ...
0
votes
0answers
36 views

awk + how to manipulate awk output

The following awk command prints the results from services file # awk '{print $2}' /etc/services 48002/tcp 48002/udp 48003/tcp How can I manipulate the output so it prints the "service file" ...
-1
votes
1answer
37 views

Unix/Linux - File Formating

We have a source file which needs to be formatted into a fixed width file. Source File Format: Hdr1 Dtl1.1 Dtl1.2 Dtl1.3 Dtl1.4 Trailer0004 Hdr2 Dtl2.1 Dtl2.2 Dtl2.3 Dtl2.4 Dtl2.5 Trailer0005 . . . ...
-4
votes
2answers
35 views

How to process only certain columns and ignore header and footer in delimited text file using awk?

I have an input text file called text.txt: Header Column1|Column2|Column3 C11|C12|C13 C21|C22|C23 C31|C32|C33 Footer My output should be: Column1|Column3 C11|C13 C21|C23 C31|C33 My command that ...
3
votes
1answer
33 views

Replace a word after matched word with incrementing values

I have a file like this - ! interface csa2549abA xxxx yyyy zzzz ! interface aso23apwoeAe aaa bbb ccc ! interface FastEthernet0/1 ppp qqq rrr ! and so ...
2
votes
3answers
47 views

linux + ssh + perform two tasks in one ssh and print to CSV

the following command print the two values in one ssh shut ssh test01 "hostname && cat /etc/sysconfig/rhn/rhnsd " test01 INTERVAL=240 but how to change the syntax in order to print the ...
1
vote
1answer
17 views

How do I interpolate a text file of 3D coordinates (X,Y,Z) onto a regular grid?

I have a 3D surface in a text file which I need to plot on a regular X/Y grid. However, the values for X and Y are not regularly spaced and are not necessarily in ascending order. I need to regularly ...
4
votes
1answer
391 views

How do I sort words by syllable count?

How do I perform an ascending sort of a word list, based upon how many syllables each word contains? Example Input: something important dog cat bookshelf Example Output: dog cat something ...
1
vote
3answers
67 views

Delete the nth line from a matched string (which only occurs once in the file)

I have a question related to deletion of strings that appear at the n+2 position after the matched string which is at position n using awk for multiple files. I am able to print it using the command: ...
-1
votes
2answers
36 views

How to convert rows to columns based on cloumn value in awk in unix

I have data like below dn: uid=032096,OU=Customers,OU=People,O=B2C objectClass: top objectClass: person objectClass: organizationalPerson objectClass: user objectClass: inetOrgPerson objectClass: ...
2
votes
3answers
86 views

Replace values in file

I have a files: $cat file1.txt 1234|W 1345|S 8427|D 2132|C 3243|V and my sql file is : $cat select.sql SELECT * FROM CUSTOMERS WHERE ID IN (FLAG); and I have a shell with the next ...
0
votes
3answers
32 views

split lines and print it to new line for certain character

I need to split lines and print it to next line when pattern match. Like I have: ABC123xxx:: 2345 ABC345yyy:: 5678 ABC986zzz:: 7955 And I want it to print to new line when ABC pattern will come: ...
1
vote
4answers
65 views

Replace the specified string to its appearance time using sed or awk

I want to replace oooo to 1, 2, 3, ..., like a example below: From > cat demo.snippet snippet anova.cca "anova.cca fun" anova.cca( ,${oooo:object} ${oooo:,...} ,permutations = ${...
-1
votes
4answers
49 views

How to filter the table names from .txt file [duplicate]

select * from emp; select * from dept; selection end; I want to obtain this output: select * from emp; select * from dept; I tried by using: awk '/select/{a=1} a; /;/{a=0}' ...
1
vote
3answers
50 views

Elegant way of counting how many times patterns from a file occur in another file

Imagine that we have two for example files. The first file is filled with unique names of employees created by combining the first two characters of the first name and the last 2 characters of the ...
-4
votes
3answers
53 views

I need a command to filter the statements from a .txt file

I have a file with the following contents: select * from emp; select * from dept; selection end; I want to obtain this output: select * from emp; select * from dept; I tried by ...
2
votes
2answers
76 views

Find (and strip) common initial whitespace from file/pipe

I am looking for a way to find and remove the common initial whitespace from a text stream or file. I do not want to remove all leading whitespace (that would be a simple sed 's/^[[:space:]]*//'). ...
0
votes
2answers
50 views

Binding Timestamp with Hexdump?

I am running a script which is given below, sudo hexdump -v -e '1/1 "%02x\n"' /dev/ttyUSB0 | Many Pipes | gawk '{ print strftime(":: %s"), $0 }' Here the data is being hex dumped with timestamp ...
3
votes
1answer
80 views

Use awk as a stream editor

awk is the swiss army knife of text manipulation. If, however, I need to change small bits in a text, I reach out to sed generally. While it might be the best tool for the job, it is worth it to ...
0
votes
0answers
41 views

awk, how to split on special characters

Background Using awk {GNU Awk 3.1.7} Must split log file with awk on special characters: (id= To illustrate my problem This works: echo "aa_bc" | awk '{{split($0,var,"_")} {print substr(var[1],1)}}...
-2
votes
2answers
72 views

standard deviation using awk [closed]

I am using below command to get standard deviation of file A names 1 2 3 avg 23.3107 20.0372 21.7236 21.6905 awk '{x[NR]=$0 ;} END{a=$4; for (i in x){ss += (x[...
0
votes
6answers
100 views

how to do formatting of a file in unix

I have a file file.txt. which have below three records. <ResultDescxmlns="http://www.mpaet.com/boe/interface/common">Operation successful. 220095 9251000014 Now my desired format should be as ...
4
votes
1answer
64 views

Why does Gawk in compatibility mode split records in the same way as when GNU extensions are enabled?

I'm not sure if I found a bug in Gawk or if I'm misreading / missing something in gawk's man page. According to gawk's man page, when in compatibility mode, if RS contains multiple characters, only ...
1
vote
2answers
20 views

Deleting lines by matching 3rd and 4th character only

I am running a script that fetches me the data as given below, 1400d2b8 1f4d1e2507 1200781e As you can see the 3rd and 4th character is 0 in line 1 and line 3. I am trying to write a single line ...
1
vote
1answer
33 views

How to output multiple lines using a single condition?

I have a file with the pattern: n0 n1 n2 ... ni ------------------------------- N0 N1 <empty line> if N0 is less than a specific number, I want: 2 lines above the N1's line N1's line Empty ...