awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
1answer
144 views

Using “case” with awk

Some time ago, I used this part of my awk script: case "AlarmLevel.0" : alarmAlertLevel=$2 break But now, the version of information has changed, and the level is dynamic, I mean now, the ...
3
votes
2answers
41 views

How to Save variables in a script that can be shared between two runs of awk against the same input file in the script? [closed]

I have a log file, whose formatting is very consistent and which I control. It produces pipe-delimited fields of constant length. Two fields are relevant to the filtering process I wish to create, ...
3
votes
3answers
314 views

Splitting a sequence into fixed width

I have a file like this which is a two-column tab-separated file. CTGCAGTTTCCCCAAATGTGGGAAACTTGACTGTATAATTTGTGGCAGTGGTA a1 GATTTCCCCAAATGTGGGAAACTCACTCGGCAGGCGTTGATA a2 I want to get an output ...
1
vote
3answers
60 views

How to do find and replace in one field based on the value in the other field?

I have a file like this. IMS00|20140101|Unix|Jan14 IMS01|20140201|Unix|Feb14 IMS02|20140301|Unix|Mar14 IMS03|20140101|Unix|Jan14 IMS04|20140201|Unix|Feb14 My criteria are, if the column 1 value is ...
2
votes
1answer
32 views

AWK get first line

I'm trying to add a column for each line together, and I do it like this: awk '{s+=$2} END {print s}' Now, when I have the final sum, s, I want to subtract the second column of the first line, like ...
0
votes
1answer
206 views

/usr/bin/awk: Argument list too long

I am trying to run the following bash script: #!/bin/bash file=$1 filename=${file%%.*} line1=$(sed -n 1~2p ${file}) seqs=$(grep -v '^>' ${file}) pos=$(echo "${line1}" | awk -F"[__]" ...
3
votes
1answer
116 views

How to add number to awk array elements?

If I have the following bash variable: $ echo "${pos}" 201 719 744 205 354 ...the following produces... !#bin/bash ( IFS=: awk -v str2="$pos" -v sep="[$IFS]" ' BEGIN { ...
-3
votes
1answer
83 views

Using AWK to define variables in a shell

I've got a live MQTT stream being filtered out and dumped into a CSV file. I use the fourth parameter ($4) and fifth parameter ($5) in the CSV file to locate the information I want. I give this ...
1
vote
3answers
196 views

Append based on line count in awk after pattern match

How can I append lines after a pattern match in awk based on how many lines they are after the pattern match. What I have looks like this: Stuff PATTERN a b c a b c a b c ENDOFSECTION Morestuff I ...
1
vote
2answers
83 views

Bash: How to create an alias in .bashrc for awk with parameters

I'm trying to add an alias in .bashrc file as follows: ... alias cutf="_cutf" ... _cutf() { awk 'NR >= $2 && NR <= $3 { print }' < $1 } (The function's goal is to show the ...
1
vote
1answer
74 views

How to use grep/awk/unix to match all lines from one file in another file, even if they are duplicates

I'm trying to retrieve the the matching lines from a file, however the file that has the "index" or list of matches to retrieve also has duplicate entries and I want to print out the duplicates as ...
0
votes
2answers
160 views

Grep for a word within two inverted commas

How to grep for a word within two inverted commas. The word may contain any number of characters . e.g "two" , "three"
0
votes
1answer
88 views

Parse header in a file and based on the header replace a value in the file

I have a problem I need help with. I'm not very experienced with awk, but it seems to be a good tool for this job. I have the following file Field1|Field2|Field3|Field4|Field5 a|b|c|d|e 1|2|3|4|5 ...
4
votes
2answers
96 views

How to cut from the 1st character to -7 character in awk?

I have a file with multiple lines. Which the second column is the user_id, but the last seven number are random numbers. It need to be trimmed off to get the authentic ID. So what I need is to get ...
1
vote
3answers
63 views

How to can I make Filename Sequencing with the help of Loops

my intention is to keep the code as simple as possible.Here there are 5 files namely A_1.CSV, A_2.CSV, A_3.CSV, A_4.CSV, A_5.CSV The below code retrieves the first row in a CSV file. Code : ...
2
votes
1answer
75 views

How to print incremental count of occurrences of unique values in column 1

I'm trying to come up with an solution to this problem, I need to incrementally count and then print the counts of the unique values in column 1 of a tab delimited text file. Here is an example: ...
0
votes
3answers
46 views

I want to condense a list of ranges (from-to)

I have data involving codes assigned to ranges in the form of From To Code 0 5 A 5 10 A 10 15 A 15 20 B 20 25 B 25 30 A 30 35 A 35 ...
0
votes
1answer
126 views

Avoid conversion to scientific notation in awk

I have a csv file that contains hundreds of columns. I would like to subtract the value of a specific column from a static number and then print the entire line. The values of that column are long ...
1
vote
2answers
55 views

Appending an input file with awk after a pattern (looping?)

I am trying to take a text file called 'input', of the form line1 line2 line3 PATTERN x y z x y z lineN and use awk to place a number (1) after each "x y z" line. I need the lines before the ...
1
vote
1answer
453 views

remove lines where a field's value is less than or equal to 3 - sed or awk?

I need to remove every line that has a value of 2 or less in the 8th field (column). My data looks like this: 12-31 Airport 189 379 41 49.70946503 -124.91377258 2 2880 30.8 01-01 ...
3
votes
5answers
337 views

Pick the latest file based upon the sequence number given in the file name

I have a requirement, I have few files in a directory, I need to pick the oldest one for processing. For Exp: I have below files in /tmp directory: Sample_0000237826-001_xyz.dat ...
0
votes
0answers
11 views

use php variables for awk regex commands [duplicate]

trying to implement autosuggest using php. autosuggest list generated using awk $input= $_GET["term"]; $suggest=array(); exec('awk \'BEGIN{FS=","}/.$input./{print $2,"-cost-",$1}\' ...
2
votes
3answers
131 views

Find the number FN with awk

I have one document, and this document has several columns, like this: ,Flag2,,Flag4,Flag5,,,,,Flag10,Flag11,Flag12,Flag13 If I use awk to know the total number of fields, knowing the separator ...
1
vote
1answer
53 views

delete non important characters from command output

the version command in my Linux machines display the following output version Red.Hat.OS. release..........5.5 APP.Release.............4.2.0.0-02 how to remove the middle "." dot character ...
3
votes
2answers
253 views

How do I insert new columns into a .csv file?

Example Data: "Monday","12345","Friday" In the above file, I want to add a new column in between 12345 and Friday. How can I do that ?
5
votes
2answers
188 views

Transpose a file, and replace missing values

I have Readings for Names, from Machines and sometimes these readings are replicated. Where a reading was not found, it is left as blank. Name Instrument Rep R1 R2 R3 N1 I1 1 1 2 3 N2 I1 1 1 3 4 ...
1
vote
3answers
51 views

Separating names separated by “| character” to individual pairs

I want to separate the names in the same line into separate individual pairs. TMPRSS2|pp9284 AADAT Sample1 ERG TMPRSS2|pp9284 Sample2 TMPRSS2|pp9284 ETV1 Sample3 PDE4A ...
0
votes
3answers
120 views

Awk/sed command to cut on multiple delimiters [closed]

I have input file name as: Sample_0000860156-001-out_20150224150524.xml.gz I want only 0000860156-001 from the file_name to append to my target file name.
1
vote
1answer
263 views

Print lines between two patterns matching a condition in awk

How can I print lines between two patterns which meet a certain condition? For instance for an input file partially containing the following: Time %MEM %CPU %SWAP 00:05:02 7.3 ...
0
votes
1answer
34 views

Printing out range of a pattern with awk

Im trying to print out a section of a txt file using awk. Basically I want to print out the paragraph in the a txt file that begins with certain words all up to a blank line.Here's what I have so far: ...
7
votes
4answers
270 views

Matching and pasting to line

So I have 2 very big text files, consisting of lines like so: First: Robert:Dillain:Other:Other:Other Julian:Brude:Other:Other:Other Megan:Flikk:Other:Other:Other Samantha:Minot:Other:Other:Other ...
2
votes
3answers
157 views

how to use awk/cut to get column data which has space

I have data in below format: 16 SQL*Plus vilconv1 dox-conv2 16 TOAD background query session Disha WORKGROUP\AD now I want to ...
1
vote
2answers
151 views

Last line is wrong when splitting a file with awk

I have a data file main.txt that looks like this 20130826,aaaaaaaaaaaaaa,bbbbbbb 20130826,sdfasdfasdfas,sdfasdfasd 20130826,dfasdfas,asdf2323 ... 20130827,sfasdfasdfasd,sdfasdfwea I used the ...
2
votes
2answers
130 views

Expose awk over tcp (inetd, socat, etc.)

I was playing around with awk, and I wanted to expose it over a tcp socket. At first I tried, but failed with micro-inetd. Thinking that maybe the issue could be in the way that carriage ...
3
votes
3answers
83 views

Single record of a file getting splitted over multiple lines

We are using flat text files to store in teradata table after transformations from Informatica. The file contains 14 fields (~ separated). But as the records are getting scattered over multiple lines ...
0
votes
1answer
76 views

Space removal after a pattern in SED

I have a file instrument.xml bash-4.1$ more instrument.xml <instr instr_id="CPT-2380950-011-001-826" valor_ch="" instr_type="CurrentAccount" instr_name="CPT-2380950-011-001-826" qn_type="1" ...
0
votes
2answers
284 views

Compare two csv files and append value using awk [closed]

I have two files as follows: file1.csv +------------+----------+--------+---------+ | Account_ID | Asset_ID | LOT_ID | FLAG_F1 | +------------+----------+--------+---------+ | 10000 | 20000 | ...
0
votes
3answers
49 views

Split the file comparing two fields in the files

Have two files which need to compare the first two columns. Sample input file1: |CID|SID|order|orderdate|confirmdate |151244127820|7177638911583| 2|2015-02-04 07:14:44|2015-02-04 07:15:32 ...
1
vote
2answers
610 views

extract blocks of text from xml file

I have a file with these contents: <People> <Person> <Name>Joe Blogs</Name> <Address>55 Oxford St</Address> ...lots of other properties ...
2
votes
2answers
145 views

Delete rows from a simplified CSV where some columns matching specific pattern

I have the following simplified CSV file (no separators or newlines embedded in fields): ...
0
votes
2answers
82 views

what are considered old and powerfull commands? [closed]

I came across the terms awk and sed, awk goes once through all lines and performs a task whenever a line meets a certain condition, sed can manipulate a stream of input before it goes further to the ...
2
votes
1answer
62 views

how can I add an extra character after a word search [closed]

I have the following field names demelog voyapro naisjdf naismc decoide decoccm travide travccm equiccm mariccm Below is a sample of my file. There are more columns before the first fck=83... ...
1
vote
2answers
50 views

How to match multiple lines starting with a TAB, and the line before the 1st one in a group?

I've got some third party log files I'm trying to pull errors out of on the command line. The logs look like this: time=1 time=2 time=3 at com.test.com.... at com.test.com.... at ...
0
votes
3answers
117 views

How can I output sample rows from a .csv?

I'm doing a lot of data merge printing and would like to be able to output, say, 10 samples from a large database. My goal would be to extract the 10 rows spread evenly from the supplied .csv ...
1
vote
3answers
87 views

Filter csv file based on extended column values

I have the following csv file: ID,PDBID,FirstResidue,SecondResidue,ThirdResidue,FourthResidue,Pattern RZ_AUTO_1,4tov,1404,1405,1518,1519,CG/AA Canonical ribose-zipper ...
4
votes
3answers
267 views

Can tr commands be chained to avoid multiple tr processes in a pipeline?

I have a bunch of txt files, I'd like to output them lower-cased, only alphabetic and one word-per line, I can do it with several tr commands in a pipeline like this: tr -d '[:punct:]' ...
2
votes
3answers
239 views

How to insert bash variables in awk?

How to insert bash variables in awk (for example i need to do in in some for loop - like this: in first iteration use awk to search string by first column, next by second column and so on with using ...
2
votes
2answers
43 views

how to print the closest column when looking for a specific strings

I have a file with around 12 000 lines. ...
2
votes
4answers
106 views

How to highlight given string in given place?

How can we modify this: <a href="http://foo.bar1">asfdlksafbar1qsasadf</a><br> <a href="http://foo.bar2">svasfbar2saldkfj</a><br> <a ...
0
votes
2answers
175 views

awk print a match only if last record before other match

I want to use awk for printing a match of some pattern, only if it is the last line of the file before some constant text. For example, i want to print the value of "c" only if it is the last line ...