Tagged Questions

awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

6
votes
7answers
635 views

Slick one-liner to convert a list like “1: 2, 3, 4, 5” to “1.2, 1.3, 1.4, 1.5”

Let's say I have a file that looks something like this: 23: a, b, c, d 24: b, d, f 25: c, g and I want to get output like this: 23.a 23.b 23.c 23.d 24.b 24.d 24.f 25.c 25.g Of course it's not ...
3
votes
1answer
43 views

How to edit files in-place while performing operation on the values matching the searched pattern?

Given a list files containing random text and many datetimes in ISO format in it (e.g "2012-07-02T10:47:24+02:00"), how can I find all these datetimes and add 2 hours to each one ? Ideally a solution ...
2
votes
3answers
202 views

Bash commands/script to remove a line from CSV with duplicate in column

I have a lot CSV files that I have combined. However, there are duplicates, but the entire line is not duplicated. I do have a column that I want to use as the criteria to search for a duplicate. ...
1
vote
3answers
224 views

Newbie struggles with grep, sed, awk to filter html

I am a beginner linux user, trying to teach myself how to use linux tools on a cygwin install. I decided to make up a project to try to teach myself the basics of shell scripting and simultaneously ...
3
votes
4answers
137 views

Replacing pattern after nth match is found on each line?

I have a file containing lines: india;austria;japan;chile china;US;nigeria;mexico;russia I want to replace all the occurences of semicolon on each line with e.g. ;NEW;, but starting from the 2nd ...
0
votes
1answer
221 views

How can I find and filter a specific column in a .csv file? [duplicate]

I have .csv files with the following structure: cat,dog,mouse,pig,bird,cow,... 21,34,54,566,78,1,... 23,10,12,569,56,4,... 32,20,13,123,56,3,... 34,30,44,322,66,2,... I want to filter the column ...
0
votes
0answers
54 views

how to parse this data and count the matching patterns? [closed]

Gi1/0/12 Gi1/0/13 Gi1/0/14 Gi1/0/15 Gi1/0/16 Gi1/0/17 Gi1/0/18 Gi1/0/19 Gi1/0/20 Gi1/0/21 Gi1/0/22 Gi1/0/23 Fa2/0/13 Fa2/0/14 Fa2/0/15 Fa2/0/16 Fa2/0/17 Fa2/0/18 Fa2/0/19 Fa2/0/20 Fa2/0/21 Fa2/0/22 ...
1
vote
1answer
73 views

How to parse this output and put it into an array?

i have expect script whose output looks like : I would like to extract all words like Gi/02/12 etc in an array or a list. VLAN Name Status Ports ---- ...
1
vote
3answers
101 views

Is it mandatory to have a variable in awk's first statement?

I am having some hard time with awk today. If I try to do: df|awk '{print $2; $some=$2; print $some}' It works as expected and I get the size of the disks twice but if I do: df|awk '{$some=$2; ...
1
vote
2answers
66 views

Match ids and print original file

I have two files Original: (5000 entries) Chr Position chr1 879108 chr1 881918 chr1 896874 … and a file with allele frequencies (2000 entries) Chr Position MAF chr1 881918 0.007 ...
0
votes
1answer
91 views

Search low values in mainframe files

I am working with mainframe files and it seems that the low values (x'00') exists in the records in the file which throws off my import for fixed block. Does anyone know how to eliminate those ...
3
votes
2answers
122 views

Extract Data between comma

I have an output file which contains different numbers seperated by commas. But the issue is I do not know the number of commas. I would in turn like this data to be grep on another file. For ...
2
votes
2answers
105 views

changing pattern by removing spaces?

file : hi hello,new york, , ,brazil site ,brazil january,month is feb , , ,indiana jones task: delete all horizontal space (spaces and tab) between any two comas (like , ,) if ...
1
vote
1answer
332 views

AWK how to add all columns beginning with specific row

I have this output: Count Process 238 0 1 1124027 1 1124065 1 1124066 1 1124069 1 1124072 1 1124073 1 1547365 1 1547366 I wanted to add all Count ...
2
votes
1answer
143 views

shell variable in awk is not being passed to all the lines but just for the very first line of input?

inputfile:is2.txt 10.39.5.41,A1,B1 10.39.5.41,A2,B2 10.39.5.41,A3,B3 10.39.5.41,A4,B4 10.39.5.41,A5,B5 10.39.5.41,A6,B6 script : #!/bin/bash second_column="OOOOOOO" # OOOOOOO will be added to ...
1
vote
1answer
122 views

reading from a file and changing its pattern into an array?

I have a CSV file containging entries like this : ipaddress,VLAN,VLANid 10.192.168.1,vlan-xyz,3 10.192.168.1,vlan-abc,8 10.192.168.1,vlan-mnp,11 10.192.163.24,vlan-llz,3 10.192.163.24,vlan-bbz,5 ...
1
vote
4answers
68 views

How to find what device a file is on (and use that in a script)?

I want to find out what device my file is on so that I can use it in a script. I can get this far: $ df . Filesystem 512-blocks Used Available Capacity Mounted on /dev/disk0s2 498438976 ...
1
vote
2answers
109 views

sed pattern matching with deletion over a line

I want to delete only some spaces(say 4) after the line$#*425 with space I am not able to complete it please suggest echo "line$#*425 with space " | sed 's/ /\1 \2/g' i am thinking of ...
1
vote
2answers
90 views

Print fields from file only if line starts with a specific word

I have the following file: … LINK=dummy 172.17.100 => 10.218.11 [PATH=/etc] FILE=hosts 172.17.100 => 10.218.11 [PATH=/etc] FILE=network 172.17.100 => 10.218.11 ...
1
vote
1answer
67 views

Regex to Cisco Interface

I would like to know if have some sed, grep or awk regex to parse Cisco interface section and exclude if interface is down (shutdown), like example below. Example: ! interface FastEthernet0/1 ip ...
1
vote
0answers
146 views

Processing shell script options with awk

I'm looking for way to process shell script arguments that is cleaner and more "self documenting" than getopt/getopts. It would need to provide... Full support of long options with or without a ...
1
vote
0answers
120 views

How to awk printf additional string with backslash

I have a csv that I want to add a preceding column to and a trailing column. The trailing column is a date (manual string right now). Every time I try to get that date, it ends up replacing some text ...
1
vote
1answer
293 views

How to calculate average requests per seconds using awk or sed on Apache logs

I am trying to calculate the average requests per seconds for a specific period of time of my Apache logs. I have been able to narrow the results down to the period I am interested in, as well as ...
1
vote
1answer
233 views

Split log file by time range

I have a log file and would like to cut it into copies of 15min. In the log file the lines start with: dd mmm yyyy hh:mm:ss,xxx e.g. 12 Feb 2013 16:05:02,xxx log text... and the file rotates ...
4
votes
4answers
1k views

Text between two tags

I wanna retrieve whatever is between these two tags – <tr> </tr> – from an html doc. Now I don't have any specific html requirements that would warrant for an html parser. I just plain ...
3
votes
2answers
159 views

AWK for data smoothing

I need some help to smooth some data with awk olny of column 4 and next. This is a data example: Date;time;Time_ms;A;B;C;D 23.11.2012;15:03:00;41236627696;1;2;2;3 ...
2
votes
2answers
256 views

System calls, AWK, and Bringing in external inputs

awk '{ TEMPVAR="/usr/bin"; printf("%s", system("ls $TEMPVAR")); }' empty In this example I'm trying to bring in the variable TEMPVAR into the system call. How would I do this? What I'm aiming to ...
3
votes
2answers
149 views

AWK : Converting Row Formated Data into Columnar

I have a dataset as such: Name: Jim Bean Vice: Dice ID: AFDSDFDSFDSFASFA LoginTime: 12343314 Name: Bob Dylon Vice: Trumpets ID: AFD232SFDSFASFA LoginTime: 12343314 ...
5
votes
2answers
177 views

what is the meaning of 1 at the end of awk script

I was reading this awk script awk -F"=" '{OFS="=";gsub(",",";",$2)}1' I want to know what is the function of 1 at the end of it
2
votes
1answer
312 views

How can I use awk to (sometimes) remove the first character of a column?

I run a command and pipe it through awk as below : mycommand | awk '{print $1,tolower($3),$4} This gives me the following output: fred.o: t .abcdefg fred.o: u .rstuvwxy fred.o: t .defghi ...
0
votes
0answers
59 views

Co Record Verification

I have a task requiring some awk to verify a few records quickly. Let' say: A1,A2 B1,B2 C1,C2 C2,C1 A1,C1 A1,B1 B1,A1 Which would be the best way to check for reciprocity only between the A# the ...
2
votes
1answer
145 views

Parse csv using awk [duplicate]

Possible Duplicate: Is there a robust command line tool for processing csv files? I am trying to parse the second line in a file (which contains commas within double quotes) "100 Watt ...
2
votes
2answers
64 views

sort a file based on length of the column/row

I need to sort a file based on the number of chars in the first column. I have no idea on how to go about this. (On Linux, so sed/awk/sort is available). An example: .abs is bla bla 12 .abc is bla ...
1
vote
0answers
141 views

Conky - string formatting

I have conky the record: ${color lightblue} Down: ${downspeed wlan0} Up: ${upspeed wlan0} Conky prints link speed, but the text moves: Down: OB Up: OB Down: 60B Up: 60B Down: 148B Up: 148B ...
3
votes
3answers
168 views

How can awk cut certain fields and add to the end of each line?

I would like to convert a list containing bunch of svn URLs of issues: cat list.txt //svn.server.address/repos/project/module1/branches/issue-001-name1 ...
1
vote
2answers
311 views

Parsing string by awk and get only elements without pipes or semi colons

I would like get this string line AUGUSTYN|Stanisław|3589238 without | and : I tried something like that: cat baza|grep "AUGUSTYN" -n|awk -F '|' '{print $1,$2,$3,$4}'` (baza is my file with ...
3
votes
1answer
90 views

No output from inotifywait | awk

I'm attempting to use part of a one-liner found here: Script to monitor folder for new files? When I try the following procedure I get no output whatsoever and I cannot figure out why. In terminal ...
1
vote
2answers
341 views

Help inserting a new line of text after matching a line of text (sed)?

I need to insert a hidden HTML input tag into any form tag within a bunch of HTML files. I assume this is possible with sed, but need help forming the command. My idea is to search for any instance ...
2
votes
2answers
97 views

Awk: expanding first field along the column

I have a task which requires some awk scripting: Lets say I have a file: A,under1 ,under2 ,under3 ,under4 ,under5 B,new1 ,new2 ,new3 C,sm1 ,sm2 How could I make it look like this using awk? ...
0
votes
2answers
124 views

Errors when searching for special characters in awk

How can I search for a pattern in awk if the text contains special characters? billinAddress || when I search, it is showing an error on pipe sign ||. How can I search for the pipe sign || in awk? ...
2
votes
2answers
337 views

Data pivot with awk

I'm trying to pivot a file using awk. This is an example of my input File: VarName;TimeString;VarValue;Validity;Time_ms A;23.11.201215:03:53;1;1;41236627696,7593 ...
1
vote
1answer
286 views

Validate Unix Timestamp using BASH

My google-fu has failed me so I'm hoping someone here can point me in the right direction. I'm in the middle of doing some data validation efforts and I suspect the system is recording some invalid ...
3
votes
4answers
1k views

removing the first and the last character of every line from command line

I am trying to remove the first and the last characters of everyline in a text file and save the resulting truncated version in a new file. Does anyone have an idea about how to do that efficiently ...
0
votes
2answers
412 views

Replace matches with multiline string using sed [duplicate]

Possible Duplicate: Insert a multiline string into another string I have a file called foo.txt containing: foo bar baz I want to replace every occurence of bar with a multiline string, ...
1
vote
3answers
281 views

linux/solaris + verify duplicate valid IP address from file

what the best way to find duplicate IP from file ( I have ksh script in this script I need to write function that check for duplicate IP ) for example if IP - 192.1926.23.52 exists twice in file - ...
2
votes
1answer
410 views

Bash variable in Awk script

I have this flat file database(ff_servers.db) with following contents: 192.168.154.2 Alaska hp 192.168.157.3 Colorado dell 192.168.156.3 hawaii hp From command line, I could run: awk ...
3
votes
1answer
89 views

Are there awk versions that provide syntax for computing aggregations?

From time to time I find myself writing awk scripts that compute some simple statistics. For example computing a histogram, the average of a value, the standard deviation or even the variance ... ...
1
vote
1answer
158 views

Sed and Awk learning material [closed]

I'm looking for some learning material for sed and awk, can anyone make any suggestions? Please don’t refer to man and help pages. -O-
2
votes
1answer
151 views

Using awk, how to get desired output from multiple file sources

I have two files: file1: A,D B,E C,F file2: G,H,I J,K,L Using awk, I want to print: A,D B,E,I C,F,L How can I use awk to achieve this?
1
vote
1answer
84 views

how to add a description in footer

I enjoy learning how to modify files with awk, sed, and bash. However, I know it could cause issues down the road and I would like to be prepared with a log inside the file. I was thinking about how ...

1 2 3 4 5 … 7