awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
0answers
62 views

How can I combine information of three files to create a fourth one?

I have three csv files, each of them containing the information I need to get a fourth one. I'm describing toy files but the actual ones contain much more columns and rows, still on the same models. ...
-1
votes
1answer
35 views

Extending awk one-liner from identifying matching pair of columns (row by row), to multiple columns

I am using Win7 with Cygwin. I have a one-liner that will work for identifying matches in a PAIR of columns, but when I try to introduce additional columns (i.e. >2) I cannot get it to work. So the ...
2
votes
4answers
50 views

Awk - Print row if number is between column 1 and column 2

I have a file with data formatted like this. Input file "1930","1946","FOO","BAR","GREEN" "1949","1962","FOO","BAR","BLUE" "1970","1980","FOO","BAR","YELLOW" Numbers are not continuous, with some ...
1
vote
3answers
41 views

Data extraction from text file

I have a text file in the following format:- $DATA1 SOURCE='HSPICE' VERSION='I-2013.12-SP2-1 32-BIT' .TITLE 'cmos inverter transfer characteristics' index t_r t_f ...
-1
votes
2answers
44 views

Error in file comparison - awk [duplicate]

I have two files file1.txt, file2.txt. I want to select records which are common to both file1.txt and file2.txtbased on column 2 and column 3 and copy them to out.txt with additional column(column 4 ...
0
votes
1answer
48 views

How do I add a \n to the end of the record on file using AWK

I have a file that can contain 1 or many lines oracle1.txt oracle2.txt oracle3.txt How do I using AWK put a \n after the last line only so it will show oracle1.txt oracle2.txt oracle3.txt\n I ...
8
votes
9answers
20k views

How to print only last column?

echo -e 'one two three\nfour five six\nseven eight nine' one two three four five six seven eight nine how can I do some "MAGIC" do get this output?: three six nine UPDATE: I don't need it in this ...
0
votes
3answers
39 views

awk or sed or perl: remove only characters on specific position

I have multiple files to rename leng-1494-001 leng-1464-002 leng-2414-004 leng-7894-005 leng-1323-006 I want to rename it ferr-1494-001 ferr-1464-002 ferr-2414-004 ...
-1
votes
1answer
31 views

Webpage content - search for pattern & save to file

I have a web page, say https://test.salesforce.com. I would like to search for a pattern, e.g. "src" at the beginning of a line in the entire webpage content. If the pattern is found, I need to ...
0
votes
2answers
58 views

Print text between two patterns not containing a particular word

I wanted to print some text between two patterns which doesn't contain a particular word input text is HEADER asdf asd asd COW assd TAIL sdfsdfs HEADER asdf asd sdfsd DOG sdfsdfsdf TAIL sdfsdfs ...
4
votes
3answers
907 views

awk + print lines from the first line until match word

I want to print all lines from file until the match word please advice how to do that with awk for example I want to print all lines until word PPP remark the first line chuld be diff from AAA ( ...
2
votes
2answers
23 views

Copy files with parent folder to a new directory

I want to copy some files and include their parent directory in the process. The current structure is similar to: /Module1/A/downloads/m13312/file.pdf /Module1/A/downloads/m17312/file.pdf ...
5
votes
2answers
143 views

Transpose a file, and replace missing values

I have Readings for Names, from Machines and sometimes these readings are replicated. Where a reading was not found, it is left as blank. Name Instrument Rep R1 R2 R3 N1 I1 1 1 2 3 N2 I1 1 1 3 4 ...
6
votes
2answers
3k views

awk: forcing a return status?

This is a followup to my earlier question. I am validating the number of fields in /etc/passwd using this handy snippit. In the following example, the users 'fieldcount1' and 'fieldcount2' have the ...
2
votes
1answer
24 views

Adding consecutive characters to duplicate words according to order they appear

So I have files like this but for hundreds of different genes per file. >lcl|accession_cds_proteinaccession [gene=dnaA] [protein=description] [protein_id=accession] [location=location] ...
1
vote
2answers
44 views

Use awk to get two specific columns from third line of file based on value in first line

I'm looking to conditionally select two columns from the third line of a file based on the first column of the first line of the file. Here is the file format: v1 shortdesc value1 value2 value3 ...
1
vote
2answers
48 views

Compare three files: lines present in one, but not in the other using column comparison

I have three files file1.txt, file2.txt, file3.txt and they are of same format(same number of columns with same data type). I want to select records from file1.txt which are not present in file2.txt, ...
0
votes
2answers
26 views

Variable in for loop to be used as an array for a csv output

I've been struggling to come up with a solution for this, the goal is to organize the results into comma separated columns as we have to recable our entire datacenter from cat6 to cat6a. $DBFILE ...
0
votes
3answers
34 views

Awk formatting issue for XML

There is an issue with the output XML format due to Record separator (RS = "\n \n"). The group of data is separated by an empty line. Any suggestion would be very much helpful to obtain the desired ...
1
vote
3answers
139 views

Find records present in file 1 also in both file 2 and file 3

I have three files file1.txt, file2.txt, file3.txt and they are of same format. I want to select records from file1.txt which are present in both file2.txt and file3.txt based on column 2 and column ...
1
vote
1answer
41 views

sort file based one colum and group by another column

I have a file of three columns: 1 A 0.5 2 B 0.7 3 A 10 4 C 4 5 B 4 I want to sort the file by increasing order of column 3 and group by column 2 1 A 0.5 3 A 10 ...
3
votes
3answers
232 views

Extracting unique piece of text from file with awk, sed or grep

From the output of pactl list sink-inputs, I need to grab the sink input number for VLC. Before that, I'm trying to extract the piece that contains the output for only VLC. All the methods that I ...
0
votes
0answers
37 views

Output of wc | awk is different in shell script than from cli

When issued via command line, this returns "2": pgrep someprocess | wc | awk '{print $1}' When run in a script like this, it returns "4" #!/bin/bash SOME_PROCESS=`pgrep someprocess | wc | awk ...
2
votes
4answers
143 views

Column editing using awk or sed. how to print the following pattern?

Sample Input: title role subject name-JOHN student math school state NY county street Phone name-TOM student math school ...
1
vote
1answer
27 views

AWK : the ' ' as field separator

I have those lines from db dump INSERT INTO db1 VALUES ('2', '33', '3', '1975', '3', '5', '1'); INSERT INTO db1 VALUES ('4', '23544', '1', '1987', '3', '1', '1'); INSERT INTO db1 VALUES ('2', '343', ...
0
votes
1answer
19 views

awk change one column with another

I want to substitute column 2 os a db file like this INSERT INTO db1 VALUES ('2', '33', '3', '1975', '3', '5', '1'); INSERT INTO db1 VALUES ('4', '23544', '1', '1987', '3', '1', '1'); INSERT INTO db1 ...
-2
votes
1answer
54 views

How to properly use awk in ksh?

I have: 1 #!/bin/ksh 2 BEGIN { print "START" } 3 { print "Hello World" } 4 END { print "STOP" } 5 6 7 exit 0 then when I try to run the file the following errors appears: ...
0
votes
2answers
38 views

.csv Keep only certain email addresses?

I have a large .csv file with my customers Name:Email I have another large list of emails that are part of this list and only want to keep those. How can I remove all contacts from the .csv file that ...
1
vote
3answers
45 views

How to use sed or awk to obtain just a directory

3:ââsdb1 8:17 1 984M 0 part /media/(NAMEOFDRIVEHERE) I want to just have the /media/(NAMEOFDRIVEHERE) location so I can store it in a variable. I've tried some sed commands and have ...
0
votes
1answer
47 views

Cutting logs (and not copying) from a log file between two time stamps

I have a very huge access.log file of 2 Gb containing apache access logs. The date time format in that is as below : 04/May/2015:02:15:25 I want to cut (and not just copy) the logs from this ...
0
votes
2answers
22 views

awk replace specific column in specific line

My data file is sdfsdfsdf 1 0.5000 0.5000 0.5000 8 7 90 135 90 135 1 2 0.5000 0.5000 0.0000 8 7 90 -45 90 -45 1 3 0.0000 0.5000 0.5000 8 7 90 225 90 225 1 4 0.5000 0.0000 0.5000 8 7 90 45 90 ...
1
vote
2answers
35 views

Add string to list using either AWK or SED?

Hi I want to be able to edit the list of name like this using either AWK or SED. Sample input list file: john paul rose lily Desired output: I am john of earth; I am paul of earth; I am rose of ...
3
votes
1answer
45 views

Counting number of entries in column 2 where column 1 is similar

I am trying to count the number of entries in column 2 for the same column1 and also number of times the first entry in column2 for similar column1 occur. For example my input TSV file is: C1 NC01 ...
8
votes
5answers
580 views

Remove duplicate lines from a file that contains a timestamp

This question/answer has some good solutions for deleting identical lines in a file, but won't work in my case since the otherwise duplicate lines have a timestamp. Is it possible to tell awk to ...
33
votes
12answers
48k views

Is there a way to get the min, max, median, and average of a list of numbers in a single command?

I have a list of numbers in a file, one per line. How can I get the minimum, maximum, median and average values? I want to use the results in a bash script. Although my immediate situation is for ...
1
vote
1answer
34 views

Conditional awk across 4 columns

I have a tab-delimited file with 9 columns as follows: 1 25555818 T 14 0 0 0 14 0:0:0:14 2 25555819 T 14 7 0 0 7 7:0:0:7 3 25555820 G 13 0 0 13 0 ...
0
votes
2answers
46 views

Awk Using NR to output iteration number

I am looping through a list of files, extracting the final line, and printing out columns 8, 9, and 10. I need to also print to the output the 'event number', which is essentially the total number of ...
4
votes
2answers
65 views

Awk inplace option with print statement

In recent version of awk there is a inplace option that can be used to do in place operations similar to sed's -i option. However I can't make it work with print statement. Let's see my example. The ...
1
vote
1answer
60 views

Shell intervals using awk

I have a file a 10:10 10:30 b 12:15 13:15 c 16:54 17:54 and I need to enter interval for example 10:15 so my output should be a 10:10 10:30 How I need to do that using awk?
1
vote
1answer
46 views

delete non important characters from command output

the version command in my Linux machines display the following output version Red.Hat.OS. release..........5.5 APP.Release.............4.2.0.0-02 how to remove the middle "." dot character ...
2
votes
1answer
33 views

How to rsync only a specific list of files to the same location on another server

I have a list of files: /location/file.txt Content of file: /location1/file2.abc /location2/location3/file1.abc /location4/location5/fileX.cde I'd like to copy files to another sever like this: ...
5
votes
6answers
299 views

split file into two parts, at a pattern

How to split a large file into two parts, at a pattern? Given an example file.txt: ABC EFG XYZ HIJ KNL I want to split this file at XYZ such that file1 contains lines up-to XYZ and rest of the ...
1
vote
3answers
37 views

How can I swap the order of each two character pair in a string?

I am trying to automate the swapping of characters 1 and 2, 3 and 4, etc, in a an arbitrary-length string of hex characters. Example: A627E39B becomes: 6A723EB9 It seems like this should be ...
2
votes
2answers
41 views

Replace between patterns that start and end in different lines

I have a file with the following format: Whatever1 Whatever2 Whatever3 binaries=( text1 bin2 ohhh3 ) Whatever4 Whatever5 Whatever6 I need to replace whatever inside lines binaries=( and ) with a ...
2
votes
2answers
87 views

Extract substrings from each line. These substrings are in comma separated, “name=value” format

I have lines in a file that are comma separated. There are no column headers, it's mostly comma separated 'name=value' pairs. Here is some test data: ...
3
votes
4answers
205 views

How to truncate the second column to given length

Given input of the form XY981743 foobarlkasdf saflkas asfZR!sgfad asdSAD asdsadf SAdfasdf46lk lksad bar foolkasjfdrte how can I truncate only the second column? The delimiter is TAB and the ...
0
votes
3answers
58 views

Printing column value in the next line

My input is in TSV format: C1 r1 r2 r3 C2 r1 r3 Desired output is: C1 r1 r2 r3 C2 r1 r3 So, basically what I want is printing value of column 1 in next line, is it ...
1
vote
1answer
49 views

Pattern matching inside awk

I intend to modify the output of ls -la to display the output of file command. The below command works well for the purpose find /opt -type f | xargs ls -let | awk 'BEGIN { OFS="\t" } { sprintf("file ...
0
votes
1answer
43 views

Solaris: How to run a system command on top of awk and combine all output in a single line

Have this peculiar requirement to print files in a given directory in the below mentioned format filename1 <type of file> owner group filename2 <type of file> owner ...
3
votes
2answers
827 views

Split log file by time range

I have a log file and would like to cut it into copies of 15min. In the log file the lines start with: dd mmm yyyy hh:mm:ss,xxx e.g. 12 Feb 2013 16:05:02,xxx log text... and the file rotates ...