awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
2answers
14 views

get specific variable using awk

I have this file.txt kespaul pts/14 2001:778:200:180 Fri May 29 11:33 - 16:40 (00:07) kespaul sshd 2001:778:200:180 Fri May 29 16:33 - 16:40 (00:07) algvai pts/2 ...
0
votes
1answer
17 views

finding lines in a file matching patterns defined in another file

I have two files file1.txt kepaul jack mike david and file2.txt 1 london smith 12564 2 miami jack 51616 3 la mike 165156 4 london kepaul 54654 5 miami grag 46546 and I need to print to a new ...
0
votes
3answers
57 views

Find all words beginning with Q

How do I search a text file and list all the words begining with Q? The file I am searching is Python source code. The closest I have found to a soulution is this question but that is not what I ...
1
vote
2answers
36 views

Specify String Lenth in Awk Pattern Search

I'm not sure what I'm doing wrong. Given a test.log of: 123.123.123.123 321.321.321.321 GET /test1234/asdfasdfpioasdfjkhweris,9 123.123.123.123 321.321.321.321 GET ...
0
votes
0answers
30 views

awk search for string containing forward slash in variable [duplicate]

I'm searching for a string in a massive XML file. Snip: <sv:node sv:name="notfound"> <sv:property sv:name="jcr:primaryType" sv:type="Name"> ...
0
votes
2answers
32 views

How to use awk in a while loop?

while read PWW do cat user/dump/bar.txt | awk '/${PWW}/{print $2}' >> user/dump/out.txt done < user/dump/foo.txt foo.txt: 2100001b32 2100001b38 3100001b35 4100001b28 5100001b46 ...
1
vote
1answer
24 views

How to exclude lines that don't include a string from the output of an awk script?

Completely new to scripting in general, and have been working on a script to go through our logs which contain entries for a variety of different things. What I'm interested in, is the entries that ...
2
votes
2answers
99 views

how to print field between the double quote

I have the following file conf.txt: perl /home/site/gen.pl "GG - W C1 Test - sai1" "15072" && ssh 13.4.93.103 perl /home/site/gen.pl "GG - W C1 Test - sai2" "15072" && ssh ...
9
votes
9answers
1k views

How can I quickly sum all numbers in a file?

Each line contains text and numbers in one column. I need to calculate the sum of the numbers in each row. How can I do that? Thx example.log contains: time=31sec time=192sec time=18sec time=543sec ...
2
votes
2answers
45 views

Add a number in a huge ASCII file

Background: (1) Here is a screen capture of a part of my ascii file (over 600Mb): (1.1) Here is a part of the code: 0, 0, 0, 0, 0, 0, 0, 0, 3.043678e-05, 3.661498e-05, 2.070347e-05, ...
0
votes
1answer
29 views

Move a range of words to the end of the string

I have the following string: "5 S comment=whatever Unspecific text fruit=apple animal=mouse sky=blue" I need to move to end of the string, all that begin with comment to the word text. The start ...
2
votes
3answers
53 views

Specific column level substitution

If I have a file with columnar entries, is it possible to substitute specific entries from a specific columns only and save changes to the same file?? Example a file abc.txt: SR|FRUITS|COLOR| ...
2
votes
2answers
66 views

Delete blank columns

This must be a common problem, but I couldn't find a solution. I have several files which have different sets of columns which are completely blank (for all rows except header) , how can I delete ...
-1
votes
0answers
39 views

Extending awk one-liner from identifying matching pair of columns (row by row), to multiple columns [migrated]

I am using Win7 with Cygwin. I have a one-liner that will work for identifying matches in a PAIR of columns, but when I try to introduce additional columns (i.e. >2) I cannot get it to work. So the ...
1
vote
1answer
124 views

How can I combine information of three files to create a fourth one?

I have three csv files, each of them containing the information I need to get a fourth one. I'm describing toy files but the actual ones contain much more columns and rows, still on the same models. ...
1
vote
3answers
41 views

Data extraction from text file

I have a text file in the following format:- $DATA1 SOURCE='HSPICE' VERSION='I-2013.12-SP2-1 32-BIT' .TITLE 'cmos inverter transfer characteristics' index t_r t_f ...
2
votes
4answers
65 views

Awk - Print row if number is between column 1 and column 2

I have a file with data formatted like this. Input file "1930","1946","FOO","BAR","GREEN" "1949","1962","FOO","BAR","BLUE" "1970","1980","FOO","BAR","YELLOW" Numbers are not continuous, with some ...
-1
votes
2answers
48 views

Error in file comparison - awk [duplicate]

I have two files file1.txt, file2.txt. I want to select records which are common to both file1.txt and file2.txtbased on column 2 and column 3 and copy them to out.txt with additional column(column 4 ...
0
votes
3answers
43 views

awk or sed or perl: remove only characters on specific position

I have multiple files to rename leng-1494-001 leng-1464-002 leng-2414-004 leng-7894-005 leng-1323-006 I want to rename it ferr-1494-001 ferr-1464-002 ferr-2414-004 ...
-1
votes
1answer
31 views

Webpage content - search for pattern & save to file

I have a web page, say https://test.salesforce.com. I would like to search for a pattern, e.g. "src" at the beginning of a line in the entire webpage content. If the pattern is found, I need to ...
2
votes
2answers
25 views

Copy files with parent folder to a new directory

I want to copy some files and include their parent directory in the process. The current structure is similar to: /Module1/A/downloads/m13312/file.pdf /Module1/A/downloads/m17312/file.pdf ...
0
votes
2answers
62 views

Print text between two patterns not containing a particular word

I wanted to print some text between two patterns which doesn't contain a particular word input text is HEADER asdf asd asd COW assd TAIL sdfsdfs HEADER asdf asd sdfsd DOG sdfsdfsdf TAIL sdfsdfs ...
1
vote
2answers
46 views

Use awk to get two specific columns from third line of file based on value in first line

I'm looking to conditionally select two columns from the third line of a file based on the first column of the first line of the file. Here is the file format: v1 shortdesc value1 value2 value3 ...
0
votes
2answers
27 views

Variable in for loop to be used as an array for a csv output

I've been struggling to come up with a solution for this, the goal is to organize the results into comma separated columns as we have to recable our entire datacenter from cat6 to cat6a. $DBFILE ...
1
vote
1answer
45 views

sort file based one colum and group by another column

I have a file of three columns: 1 A 0.5 2 B 0.7 3 A 10 4 C 4 5 B 4 I want to sort the file by increasing order of column 3 and group by column 2 1 A 0.5 3 A 10 ...
1
vote
2answers
51 views

Compare three files: lines present in one, but not in the other using column comparison

I have three files file1.txt, file2.txt, file3.txt and they are of same format(same number of columns with same data type). I want to select records from file1.txt which are not present in file2.txt, ...
0
votes
0answers
38 views

Output of wc | awk is different in shell script than from cli

When issued via command line, this returns "2": pgrep someprocess | wc | awk '{print $1}' When run in a script like this, it returns "4" #!/bin/bash SOME_PROCESS=`pgrep someprocess | wc | awk ...
2
votes
1answer
28 views

Adding consecutive characters to duplicate words according to order they appear

So I have files like this but for hundreds of different genes per file. >lcl|accession_cds_proteinaccession [gene=dnaA] [protein=description] [protein_id=accession] [location=location] ...
3
votes
3answers
243 views

Extracting unique piece of text from file with awk, sed or grep

From the output of pactl list sink-inputs, I need to grab the sink input number for VLC. Before that, I'm trying to extract the piece that contains the output for only VLC. All the methods that I ...
0
votes
1answer
19 views

awk change one column with another

I want to substitute column 2 os a db file like this INSERT INTO db1 VALUES ('2', '33', '3', '1975', '3', '5', '1'); INSERT INTO db1 VALUES ('4', '23544', '1', '1987', '3', '1', '1'); INSERT INTO db1 ...
1
vote
1answer
27 views

AWK : the ' ' as field separator

I have those lines from db dump INSERT INTO db1 VALUES ('2', '33', '3', '1975', '3', '5', '1'); INSERT INTO db1 VALUES ('4', '23544', '1', '1987', '3', '1', '1'); INSERT INTO db1 VALUES ('2', '343', ...
2
votes
4answers
146 views

Column editing using awk or sed. how to print the following pattern?

Sample Input: title role subject name-JOHN student math school state NY county street Phone name-TOM student math school ...
0
votes
2answers
41 views

.csv Keep only certain email addresses?

I have a large .csv file with my customers Name:Email I have another large list of emails that are part of this list and only want to keep those. How can I remove all contacts from the .csv file that ...
1
vote
3answers
143 views

Find records present in file 1 also in both file 2 and file 3

I have three files file1.txt, file2.txt, file3.txt and they are of same format. I want to select records from file1.txt which are present in both file2.txt and file3.txt based on column 2 and column ...
0
votes
1answer
50 views

Cutting logs (and not copying) from a log file between two time stamps

I have a very huge access.log file of 2 Gb containing apache access logs. The date time format in that is as below : 04/May/2015:02:15:25 I want to cut (and not just copy) the logs from this ...
1
vote
3answers
45 views

How to use sed or awk to obtain just a directory

3:ââsdb1 8:17 1 984M 0 part /media/(NAMEOFDRIVEHERE) I want to just have the /media/(NAMEOFDRIVEHERE) location so I can store it in a variable. I've tried some sed commands and have ...
0
votes
2answers
22 views

awk replace specific column in specific line

My data file is sdfsdfsdf 1 0.5000 0.5000 0.5000 8 7 90 135 90 135 1 2 0.5000 0.5000 0.0000 8 7 90 -45 90 -45 1 3 0.0000 0.5000 0.5000 8 7 90 225 90 225 1 4 0.5000 0.0000 0.5000 8 7 90 45 90 ...
1
vote
2answers
38 views

Add string to list using either AWK or SED?

Hi I want to be able to edit the list of name like this using either AWK or SED. Sample input list file: john paul rose lily Desired output: I am john of earth; I am paul of earth; I am rose of ...
-2
votes
1answer
59 views

How to properly use awk in ksh?

I have: 1 #!/bin/ksh 2 BEGIN { print "START" } 3 { print "Hello World" } 4 END { print "STOP" } 5 6 7 exit 0 then when I try to run the file the following errors appears: ...
3
votes
1answer
47 views

Counting number of entries in column 2 where column 1 is similar

I am trying to count the number of entries in column 2 for the same column1 and also number of times the first entry in column2 for similar column1 occur. For example my input TSV file is: C1 NC01 ...
1
vote
1answer
35 views

Conditional awk across 4 columns

I have a tab-delimited file with 9 columns as follows: 1 25555818 T 14 0 0 0 14 0:0:0:14 2 25555819 T 14 7 0 0 7 7:0:0:7 3 25555820 G 13 0 0 13 0 ...
0
votes
2answers
47 views

Awk Using NR to output iteration number

I am looping through a list of files, extracting the final line, and printing out columns 8, 9, and 10. I need to also print to the output the 'event number', which is essentially the total number of ...
2
votes
1answer
34 views

How to rsync only a specific list of files to the same location on another server

I have a list of files: /location/file.txt Content of file: /location1/file2.abc /location2/location3/file1.abc /location4/location5/fileX.cde I'd like to copy files to another sever like this: ...
1
vote
3answers
37 views

How can I swap the order of each two character pair in a string?

I am trying to automate the swapping of characters 1 and 2, 3 and 4, etc, in a an arbitrary-length string of hex characters. Example: A627E39B becomes: 6A723EB9 It seems like this should be ...
2
votes
2answers
42 views

Replace between patterns that start and end in different lines

I have a file with the following format: Whatever1 Whatever2 Whatever3 binaries=( text1 bin2 ohhh3 ) Whatever4 Whatever5 Whatever6 I need to replace whatever inside lines binaries=( and ) with a ...
4
votes
2answers
65 views

Awk inplace option with print statement

In recent version of awk there is a inplace option that can be used to do in place operations similar to sed's -i option. However I can't make it work with print statement. Let's see my example. The ...
3
votes
4answers
206 views

How to truncate the second column to given length

Given input of the form XY981743 foobarlkasdf saflkas asfZR!sgfad asdSAD asdsadf SAdfasdf46lk lksad bar foolkasjfdrte how can I truncate only the second column? The delimiter is TAB and the ...
0
votes
3answers
58 views

Printing column value in the next line

My input is in TSV format: C1 r1 r2 r3 C2 r1 r3 Desired output is: C1 r1 r2 r3 C2 r1 r3 So, basically what I want is printing value of column 1 in next line, is it ...
1
vote
1answer
50 views

Pattern matching inside awk

I intend to modify the output of ls -la to display the output of file command. The below command works well for the purpose find /opt -type f | xargs ls -let | awk 'BEGIN { OFS="\t" } { sprintf("file ...
5
votes
6answers
307 views

split file into two parts, at a pattern

How to split a large file into two parts, at a pattern? Given an example file.txt: ABC EFG XYZ HIJ KNL I want to split this file at XYZ such that file1 contains lines up-to XYZ and rest of the ...