awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
1answer
34 views

Print lines between two patterns matching a condition in awk

How can I print lines between two patterns which meet a certain condition? For instance for an input file partially containing the following: Time %MEM %CPU %SWAP 00:05:02 7.3 ...
-1
votes
0answers
23 views

Count how many times a word occurs in a txt file with awk

Basically i'm trying to count all instances of certain words that occur in a text file then printing the result. It has to be in done using awk. Im not really sure where to start. Any suggestions?
0
votes
1answer
24 views

Printing out range of a pattern with awk

Im trying to print out a section of a txt file using awk. Basically I want to print out the paragraph in the a txt file that begins with certain words all up to a blank line.Here's what I have so far: ...
6
votes
4answers
164 views

Matching and pasting to line

So I have 2 very big text files, consisting of lines like so: First: Robert:Dillain:Other:Other:Other Julian:Brude:Other:Other:Other Megan:Flikk:Other:Other:Other Samantha:Minot:Other:Other:Other ...
2
votes
3answers
41 views

how to use awk/cut to get column data which has space

I have data in below format: 16 SQL*Plus vilconv1 dox-conv2 16 TOAD background query session Disha WORKGROUP\AD now I want to ...
1
vote
2answers
87 views

Last line is wrong when splitting a file with awk

I have a data file main.txt that looks like this 20130826,aaaaaaaaaaaaaa,bbbbbbb 20130826,sdfasdfasdfas,sdfasdfasd 20130826,dfasdfas,asdf2323 ... 20130827,sfasdfasdfasd,sdfasdfwea I used the ...
2
votes
2answers
35 views

Expose awk over tcp (inetd, socat, etc.)

I was playing around with awk, and I wanted to expose it over a tcp socket. At first I tried, but failed with micro-inetd. Thinking that maybe the issue could be in the way that carriage ...
1
vote
1answer
42 views

Single record of a file getting splitted over multiple lines

We are using flat text files to store in teradata table after transformations from Informatica. The file contains 14 fields (~ separated). But as the records are getting scattered over multiple lines ...
0
votes
1answer
54 views

Space removal after a pattern in SED

I have a file instrument.xml bash-4.1$ more instrument.xml <instr instr_id="CPT-2380950-011-001-826" valor_ch="" instr_type="CurrentAccount" instr_name="CPT-2380950-011-001-826" qn_type="1" ...
0
votes
2answers
48 views

Compare two csv files and append value using awk [closed]

I have two files as follows: file1.csv Account_ID,Asset_ID,LOT_ID,FLAG_F1 10000,20000,30000,Y 10001,20001,30001,N 10002,20002,30002,Y 10003,20003,30003,N 10004,20004,30004,Y 10005,20005,30005,N ...
0
votes
3answers
41 views

Split the file comparing two fields in the files

Have two files which need to compare the first two columns. Sample input file1: |CID|SID|order|orderdate|confirmdate |151244127820|7177638911583| 2|2015-02-04 07:14:44|2015-02-04 07:15:32 ...
1
vote
2answers
95 views

extract blocks of text from xml file

I have a file with these contents: <People> <Person> <Name>Joe Blogs</Name> <Address>55 Oxford St</Address> ...lots of other properties ...
2
votes
2answers
81 views

Delete rows from a simplified CSV where some columns matching specific pattern

I have the following simplified CSV file (no separators or newlines embedded in fields): ...
0
votes
2answers
71 views

what are considered old and powerfull commands? [closed]

I came across the terms awk and sed, awk goes once through all lines and performs a task whenever a line meets a certain condition, sed can manipulate a stream of input before it goes further to the ...
2
votes
1answer
48 views

how can I add an extra character after a word search [closed]

I have the following field names demelog voyapro naisjdf naismc decoide decoccm travide travccm equiccm mariccm Below is a sample of my file. There are more columns before the first fck=83... ...
0
votes
2answers
28 views

How to match multiple lines starting with a TAB, and the line before the 1st one in a group?

I've got some third party log files I'm trying to pull errors out of on the command line. The logs look like this: time=1 time=2 time=3 at com.test.com.... at com.test.com.... at ...
0
votes
3answers
47 views

How can I output sample rows from a .csv?

I'm doing a lot of data merge printing and would like to be able to output, say, 10 samples from a large database. My goal would be to extract the 10 rows spread evenly from the supplied .csv ...
1
vote
3answers
52 views

Filter csv file based on extended column values

I have the following csv file: ID,PDBID,FirstResidue,SecondResidue,ThirdResidue,FourthResidue,Pattern RZ_AUTO_1,4tov,1404,1405,1518,1519,CG/AA Canonical ribose-zipper ...
4
votes
3answers
189 views

Can tr commands be chained to avoid multiple tr processes in a pipeline?

I have a bunch of txt files, I'd like to output them lower-cased, only alphabetic and one word-per line, I can do it with several tr commands in a pipeline like this: tr -d '[:punct:]' ...
2
votes
3answers
59 views

How to insert bash variables in awk?

How to insert bash variables in awk (for example i need to do in in some for loop - like this: in first iteration use awk to search string by first column, next by second column and so on with using ...
2
votes
2answers
34 views

how to print the closest column when looking for a specific strings

I have a file with around 12 000 lines. ...
0
votes
0answers
21 views

How to highlight given string in given place?

How can we modify this: <a href="http://foo.bar1">asfdlksafbar1qsasadf</a><br> <a href="http://foo.bar2">svasfbar2saldkfj</a><br> <a ...
0
votes
2answers
64 views

awk print a match only if last record before other match

I want to use awk for printing a match of some pattern, only if it is the last line of the file before some constant text. For example, i want to print the value of "c" only if it is the last line ...
0
votes
1answer
30 views

Run unix command on awk field [duplicate]

I want to run the basename command on a certain awk field. echo "1 /this/is/a/path" | awk '{print $1" "system("/usr/bin/basename " $2)}' but the output always produces a 0 from the system command. ...
0
votes
0answers
31 views

How do I add a \n to the end of the record on file using AWK

I have a file that can contain 1 or many lines oracle1.txt oracle2.txt oracle3.txt How do I using AWK put a \n after the last line only so it will show oracle1.txt oracle2.txt oracle3.txt\n I ...
0
votes
1answer
27 views

Filter by column value and write to file

I have a table like this: BB1520_v1.0_0103 exon 3329 3601 BB1520_v1.0_0103 exon 3650 4040 BB1520_v1.0_0089 exon 1 862 BB1520_v1.0_0089 exon 1312 ...
0
votes
1answer
60 views

Reading a string till a key word and replacing from there with another string

I am new to UNIX and am practising shell scripting. I have an application where I need string searching and replacement. There are some lines in a file 1 and file 2. I have a search key word. If the ...
1
vote
1answer
67 views

When will awk treat newline character as `;` and when not?

I'm reading the famous awk tutorial from http://www.grymoire.com/Unix/Awk.html. In which it mentioned To be precise, in the original AWK you can insert a new line character after the curly braces, ...
1
vote
1answer
101 views

grep (/sed/awk) month range

I've looked around for solutions to this, but it's sort of difficult to search for the solution to such a specific problem (lot of words to put in Google). So, I have a file with data that looks like ...
2
votes
2answers
48 views

how to run awk twice across the same file

I have a log file with user_id's and another column with contest results. I would like to: find all the user_ids of the users who won given those user_id's, return ALL the log entries for those ...
3
votes
3answers
158 views

How to view all the content in an awk array?

In my understanding, awk array is something like python dict. So I write down the code bellow to explore it: awk '{my_dict[$1] = $2} END { print my_dict}' zen And I got: awk: can't read value of ...
1
vote
1answer
37 views

Comparing two files using awk language

I have two files named file1 and file2. I want to compare with the NR of first files with the first field of file2 i.e. $1. Like this: if(file1.NR==file2.$1){ then do some calculations}. Please help ...
3
votes
3answers
271 views

Why awk understand FS=“*” but not for FS=“-*-”?

I got a test file, its content is: a -*- b I used awk 'BEGIN {FS="*"} {print $2}' test, it prints out - b Correct! But when I use awk 'BEGIN {FS="-*-"} {print $2}' test, I got: * I know that ...
1
vote
1answer
14 views

setenv from awk in tcsh

I am trying to set environment variables one per line from awk: $ <source command> awk '/inst/ { n = split($0,a,"/"); cmd = "setenv " a[n-1] "_ver" $0; system(cmd) }' However the out I get is ...
4
votes
1answer
157 views

Why does awk '{print “Hi, Judy”}' print “Hi, Judy” repeatedly when pressing Return?

For my understanding, usually awk follows a file or an output. Then it will exec its code on everyline of that file or output. In my case, my command awk '{print "Hi, Judy"}' didn't follow by any ...
-1
votes
2answers
24 views

Need to merge mulitple unix txt lines [closed]

Hi I need to merge multiple lines too one line in txt using awk,have tried different things but can't get the result I'm after! An example; need it to go from this: 11 01 30 31 33 37 39 39 30 30 30 ...
1
vote
1answer
22 views

How to remove duplicate lines while keeping the order AND ignoring case?

[user@notebook ~]$ printf '1\n5\n3\n3\n4\n2\nA\nB\na\n' 1 5 3 3 4 2 A B a [user@notebook ~]$ [user@notebook ~]$ printf '1\n5\n3\n3\n4\n2\nA\nB\na\n' | awk '!seen[$0]++' 1 5 3 4 2 A B a [user@notebook ...
0
votes
2answers
21 views

How to select specific sets or sturture from a file, if struture contains specific string?

I want to extract the sets with having specific string, for example: file abc.txt contains abc 123 abcd 1234 abcde 12345 abc 123 abcd 123 abcde 12345 abc 123 abcd 1234 abcde 12345 Now from the ...
0
votes
1answer
51 views

Sum two columns from different files

I have two files like this: File1: -54.005 -67.405 0.1 -54.05 -68.7833 0.2 -54.05 -68.7333 0.3 File2: -54.005 -67.4050 70 12.7 -54.05 -68.7833 167 12.5 -54.05 -68.7333 205 12.6 ...
1
vote
1answer
24 views

Merge columns with the same value with awk

It is possible merge columns with the same value in awk? Something like if awk '$1 = $2' then merge I got this AZ14962 AZ14962 ctg_ 104 AZ14962 AZ14962 ctg_ 4 AZ14962 AZ14962 ctg_ 61 ...
0
votes
1answer
31 views

How to use awk to read data between all frequent time intervals

I have a log file in following format [30/Jan/2015:10:10:30 +0000] 12.30.30.204 xff=- reqId=[-] status_check len=- GET /api/getstatus HTTP/1.1 mi=- ec=- 200 425 [30/Jan/2015:10:11:00 +0000] ...
-1
votes
1answer
28 views

how to print certain text in a line

How can I get the result in the below format? Below is the line on which action needs to be performed. Empty Results found in BusListingAction/buslisting.htm for: DepartureCity: BARMER ArrivalCity ...
0
votes
2answers
55 views

Using AWK to Print Output of Combination of Unspecified Number of Files

My objective is to move into a subdirectory, call it SUB, which has many text files with 5 columns each and always the same number of rows. I want one large 'sum' file that adds all columns from 2-5, ...
1
vote
2answers
48 views

Use awk to replace a column without changing format [closed]

I have a PDB file (see below) and I need to change the A in column 5 to a B, AND I need to change it without changing the format of the original file. I was able to change the A to a B using this awk ...
3
votes
3answers
41 views

Append and write out variable-named file from list of variables

I would like to create multiple 'wget' shell scripts to download a batch of considerably large (~3GB) files using our HPC cluster. The names of the files are stored in filenames.txt, like so: $cat ...
1
vote
2answers
158 views

read first element in array

defaults read -g AppleLanguages produces something like this: ( en, de ) I'd like to extract just the first element, in this case "en", defaults read -g AppleLanguages | awk '/\(/ ...
1
vote
1answer
46 views

split line based on space and delete the second part

I have a big file >fid|29290408|locus|VBIEntCas2262_0001| Phosphoglycolate phosphatase (EC 3.1.3.18) [Enterococcus casseliflavus EC20] ...
-1
votes
1answer
42 views

multiple input in a awk file

Please guide me to get out of it. Thanks in advance. I am having a situation that I have to write a awk script which takes the input of two .out files and generate a single .txt file. $ cat ...
3
votes
7answers
282 views

can we print the last word of each line in linux using sed command?

suppose , if there is a file consisting of following lines , if they are 12345 567 7878 66 er3 t45t y6y46y 4y6 y656y y5y 46y6 65y7 y66uyuy yy46y6y The output has to look like: 66 ...
0
votes
1answer
20 views

Insert space at specified position in a file

I have this sample file: 234234|111||J1X0H8|SERP||R|D|666|M 234235|222||J1X0N5|SERP||C||555|M For each line, I need to insert a space after the 3rd character of the 4th column, like so: ...