awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
3answers
14 views

Find the number FN with awk

I have one document, and this document has several columns, like this: ,Flag2,,Flag4,Flag5,,,,,Flag10,Flag11,Flag12,Flag13 If I use awk to know the total number of fields, knowing the separator ...
2
votes
2answers
25 views

How do I insert new columns into a .csv file?

Example Data: "Monday","12345","Friday" In the above file, I want to add a new column in between 12345 and Friday. How can I do that ?
0
votes
0answers
22 views

delete non important characters from command output

the version command in my Linux machines display the following output version Red.Hat.OS. release..........5.5 APP.Release.............4.2.0.0-02 how to remove the middle "." dot character ...
0
votes
3answers
38 views

Separating names separated by “| character” to individual pairs

I want to separate the names in the same line into separate individual pairs. TMPRSS2|pp9284 AADAT Sample1 ERG TMPRSS2|pp9284 Sample2 TMPRSS2|pp9284 ETV1 Sample3 PDE4A ...
3
votes
0answers
54 views

Transpose a file, and replace missing values by

Please guide with the following data formatting. I have Readings for Names, from Machines and sometimes these readings are replicated. Where a reading was not found, it is left as blank. Name ...
0
votes
3answers
31 views

Awk/sed command to cut on multiple delimiters [on hold]

I have input file name as: Sample_0000860156-001-out_20150224150524.xml.gz I want only 0000860156-001 from the file_name to append to my target file name.
0
votes
0answers
25 views

How to highlight given string in given place?

How can we modify this: <a href="http://foo.bar1">asfdlksafbar1qsasadf</a><br> <a href="http://foo.bar2">svasfbar2saldkfj</a><br> <a ...
7
votes
4answers
190 views

Matching and pasting to line

So I have 2 very big text files, consisting of lines like so: First: Robert:Dillain:Other:Other:Other Julian:Brude:Other:Other:Other Megan:Flikk:Other:Other:Other Samantha:Minot:Other:Other:Other ...
1
vote
1answer
41 views

Print lines between two patterns matching a condition in awk

How can I print lines between two patterns which meet a certain condition? For instance for an input file partially containing the following: Time %MEM %CPU %SWAP 00:05:02 7.3 ...
4
votes
4answers
2k views

Redirecting Output of print to multiples files in awk

I want to print same lines to 2 different files. What I am doing right now is: print ID[i]" "Value[i]" "TS1[i]" "TS2[i] > "file1.txt"; print ID[i]" "Value[i]" "TS1[i]" "TS2[i] > "file2.txt"; ...
0
votes
1answer
24 views

Printing out range of a pattern with awk

Im trying to print out a section of a txt file using awk. Basically I want to print out the paragraph in the a txt file that begins with certain words all up to a blank line.Here's what I have so far: ...
1
vote
3answers
739 views

Trouble in adding a column with awk

I'm trying to add a column to a csv file by using: awk -F "," '{print $0"new_col"}' file.csv but I got : new_col8,0.405,0.101 new_col4,0.374,0.100 new_col23,0.342,0.100 new_col130,0.298,0.100 ...
-1
votes
0answers
26 views

Count how many times a word occurs in a txt file with awk

Basically i'm trying to count all instances of certain words that occur in a text file then printing the result. It has to be in done using awk. Im not really sure where to start. Any suggestions?
2
votes
3answers
42 views

how to use awk/cut to get column data which has space

I have data in below format: 16 SQL*Plus vilconv1 dox-conv2 16 TOAD background query session Disha WORKGROUP\AD now I want to ...
1
vote
2answers
92 views

Last line is wrong when splitting a file with awk

I have a data file main.txt that looks like this 20130826,aaaaaaaaaaaaaa,bbbbbbb 20130826,sdfasdfasdfas,sdfasdfasd 20130826,dfasdfas,asdf2323 ... 20130827,sfasdfasdfasd,sdfasdfwea I used the ...
1
vote
1answer
44 views

Single record of a file getting splitted over multiple lines

We are using flat text files to store in teradata table after transformations from Informatica. The file contains 14 fields (~ separated). But as the records are getting scattered over multiple lines ...
46
votes
2answers
69k views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
11
votes
8answers
9k views

How to print certain columns by name?

I have the following file: id name age 1 ed 50 2 joe 70 I want to print just the id and age columns. Right now I just use awk: cat file.tsv | awk '{ print $1, $3 }' However, this ...
2
votes
2answers
37 views

Expose awk over tcp (inetd, socat, etc.)

I was playing around with awk, and I wanted to expose it over a tcp socket. At first I tried, but failed with micro-inetd. Thinking that maybe the issue could be in the way that carriage ...
3
votes
3answers
8k views

Insert text in specific lines of a file?

How can insert some text in specific lines of a file? What is most simplest method that I can use? (bash, sed, awk?) What I want to do, might be simple, but I don't know where I should start. It's ...
0
votes
1answer
56 views

Space removal after a pattern in SED

I have a file instrument.xml bash-4.1$ more instrument.xml <instr instr_id="CPT-2380950-011-001-826" valor_ch="" instr_type="CurrentAccount" instr_name="CPT-2380950-011-001-826" qn_type="1" ...
0
votes
2answers
53 views

Compare two csv files and append value using awk [closed]

I have two files as follows: file1.csv Account_ID,Asset_ID,LOT_ID,FLAG_F1 10000,20000,30000,Y 10001,20001,30001,N 10002,20002,30002,Y 10003,20003,30003,N 10004,20004,30004,Y 10005,20005,30005,N ...
9
votes
9answers
26k views

Show sum of file sizes in directory listing

The Windows dir directory listing command has a line at the end showing the total amount of space taken up by the files listed. For example, dir *.exe shows all the .exe files in the current ...
1
vote
2answers
461 views

Remove lines based on duplicates within one column without sort

I have large 3-column files (~10,000 lines) and I would like to remove lines when the contents of the third column of that line appear in the third column of another line. The files' sizes make sort a ...
3
votes
4answers
125 views

Replace a marker in a text file with arbitrary new text

I have a file which contains some text and - on a single line - a marker which indicates where new content should be added foo bar %SUBSTITUTE% foo The line line substitute should be replaced by a ...
24
votes
12answers
5k views

Remove duplicate $PATH entries with awk command

I am trying to write a bash shell function that will allow me to remove duplicate copies of directories from my path environment. I was told that it is possible to achieve this with a one line ...
0
votes
3answers
41 views

Split the file comparing two fields in the files

Have two files which need to compare the first two columns. Sample input file1: |CID|SID|order|orderdate|confirmdate |151244127820|7177638911583| 2|2015-02-04 07:14:44|2015-02-04 07:15:32 ...
2
votes
1answer
48 views

how can I add an extra character after a word search [closed]

I have the following field names demelog voyapro naisjdf naismc decoide decoccm travide travccm equiccm mariccm Below is a sample of my file. There are more columns before the first fck=83... ...
2
votes
2answers
87 views

Delete rows from a simplified CSV where some columns matching specific pattern

I have the following simplified CSV file (no separators or newlines embedded in fields): ...
1
vote
1answer
101 views

grep (/sed/awk) month range

I've looked around for solutions to this, but it's sort of difficult to search for the solution to such a specific problem (lot of words to put in Google). So, I have a file with data that looks like ...
1
vote
2answers
100 views

extract blocks of text from xml file

I have a file with these contents: <People> <Person> <Name>Joe Blogs</Name> <Address>55 Oxford St</Address> ...lots of other properties ...
2
votes
3answers
62 views

How to insert bash variables in awk?

How to insert bash variables in awk (for example i need to do in in some for loop - like this: in first iteration use awk to search string by first column, next by second column and so on with using ...
1
vote
3answers
54 views

Filter csv file based on extended column values

I have the following csv file: ID,PDBID,FirstResidue,SecondResidue,ThirdResidue,FourthResidue,Pattern RZ_AUTO_1,4tov,1404,1405,1518,1519,CG/AA Canonical ribose-zipper ...
6
votes
4answers
723 views

find length of longest line in all text files in a directory

I know how to get length of the longest line in a text file with awk awk ' { if ( length > L ) { L=length} }END{ print L}' file.txt but how can I get the length of the longest line of all files ...
1
vote
3answers
3k views

Awk printf number in width and round it up

I need to printf a number out, but with given width and rounded (with awk!) %10s I have this and somehow I need to connect the %d but everything I do, ends up with too much parametres for awk ...
0
votes
2answers
72 views

what are considered old and powerfull commands? [closed]

I came across the terms awk and sed, awk goes once through all lines and performs a task whenever a line meets a certain condition, sed can manipulate a stream of input before it goes further to the ...
0
votes
3answers
48 views

How can I output sample rows from a .csv?

I'm doing a lot of data merge printing and would like to be able to output, say, 10 samples from a large database. My goal would be to extract the 10 rows spread evenly from the supplied .csv ...
4
votes
3answers
190 views

Can tr commands be chained to avoid multiple tr processes in a pipeline?

I have a bunch of txt files, I'd like to output them lower-cased, only alphabetic and one word-per line, I can do it with several tr commands in a pipeline like this: tr -d '[:punct:]' ...
0
votes
2answers
28 views

How to match multiple lines starting with a TAB, and the line before the 1st one in a group?

I've got some third party log files I'm trying to pull errors out of on the command line. The logs look like this: time=1 time=2 time=3 at com.test.com.... at com.test.com.... at ...
0
votes
2answers
66 views

awk print a match only if last record before other match

I want to use awk for printing a match of some pattern, only if it is the last line of the file before some constant text. For example, i want to print the value of "c" only if it is the last line ...
2
votes
2answers
556 views

Delete rows in a simplified csv file (one newline per row) based on a specific column value

I have a csv file with the following headers - Interaction_ID, PDB_ID, First_Residue, First_Chain, Second_Residue, Third_Residue, Third_Chain & Pattern: ...
-1
votes
1answer
362 views

search for a character in one column and replace value in another column of matched records

I have a requirement to find value 2 in column 4 and wherever it matches replace 6th column with '-'sign in the beginning suppose if i have csv file - file1 like this abc,0,def,1234,ijk ...
1
vote
2answers
77 views

Remove a line if regex exists only in 1st column of each line in a csv file

I have the following comma-delimited data (spaces used for visual clarity): 1 2014 23 2352 19.24 7.18 1 2014 23 2353 19.78 4.65 24 2014 23 2354 20.26 25.34 1 2014 23 2355 26.65 42.76 1 2014 23 ...
2
votes
2answers
35 views
8
votes
4answers
425 views

Output part of each line to a separate file

I have a file like this: a AGTACTTCCAGGAACGGTGCACTCTCC b ATGGATTTTTGGAGCAGGGAGATGGAATAGGAGCATGCTCCAT c ATATTAAATGGATTTTTGGAGCAGGGAGATGGAATAGGAGCATGCTCCATCCACTCCACAC d ...
0
votes
0answers
31 views

How do I add a \n to the end of the record on file using AWK

I have a file that can contain 1 or many lines oracle1.txt oracle2.txt oracle3.txt How do I using AWK put a \n after the last line only so it will show oracle1.txt oracle2.txt oracle3.txt\n I ...
0
votes
1answer
31 views

Run unix command on awk field [duplicate]

I want to run the basename command on a certain awk field. echo "1 /this/is/a/path" | awk '{print $1" "system("/usr/bin/basename " $2)}' but the output always produces a 0 from the system command. ...
0
votes
1answer
27 views

Filter by column value and write to file

I have a table like this: BB1520_v1.0_0103 exon 3329 3601 BB1520_v1.0_0103 exon 3650 4040 BB1520_v1.0_0089 exon 1 862 BB1520_v1.0_0089 exon 1312 ...
0
votes
1answer
60 views

Reading a string till a key word and replacing from there with another string

I am new to UNIX and am practising shell scripting. I have an application where I need string searching and replacement. There are some lines in a file 1 and file 2. I have a search key word. If the ...
1
vote
1answer
67 views

When will awk treat newline character as `;` and when not?

I'm reading the famous awk tutorial from http://www.grymoire.com/Unix/Awk.html. In which it mentioned To be precise, in the original AWK you can insert a new line character after the curly braces, ...