awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

8
votes
3answers
1k views

What are the exact differences between awk and cut with grep? [closed]

We know that we can get the second column of the line we want from a file using these two techniques: awk '/WORD/ { print $2 }' filename or grep WORD filename| cut -f 2 -d ' ' My questions are: ...
8
votes
2answers
211 views

Are there 2 ways to set awk vars via command line?

I noticed an O'Reilly awk example (1997) which assigned an awk variable by setting it on the command line after the program-text. It does work, but I can't find this syntax in man / info awk. Have I ...
7
votes
3answers
3k views

Why does awk do full buffering when reading from a pipe

I'm reading from a serial port connected to a gps device sending nmea strings. A simplified invocation to illustrate my point: $ awk '{ print $0 }' /dev/ttyPSC9 ...
6
votes
2answers
572 views

How would I sort these directory names numerically?

I'm aware that I can somehow sort this output numerically (so cpu1/ follows cpu0/) ... I could probably get something to work eventually by splitting up the string various ways with awk, but is there ...
5
votes
5answers
83 views

Replace data at specific positions in txt file using data from another file

I have a text file in the below format: $data This is the experimental data good data This is good file datafile 1 4324 3673 6.2e+11 7687 67576 2 3565 8768 8760 5780 8778 "This is line ...
5
votes
5answers
539 views

Pattern matching across multiple lines

What is a quick and easy way to validate if the pattern "301 domaname.com 200" is present within BASH when spread across multiple lines of output? I was using something like the following: awk ...
5
votes
4answers
178 views

How to add each output of awk

How would you perform addition on the output of every line from awk? For example cat foo | awk '{print $1} 42 42 42 I'm looking for the answer to 42+42+42 (126). I'm trying to write a for loop ...
4
votes
3answers
100 views

awk separator preferences

I write ad hoc awk commands in cygwin to process tab delimited files. Pretty much every command starts with: BEGIN {FS="\t";OFS="\t";ORS="\r\n";} How can I make these separators the default to ...
4
votes
2answers
820 views

Inotifywait for large number of files in a directory

What I want to do is, to monitor a directory (not recursive, just one) for new files created and append those files to one single big file as they are being written. The number of files that are ...
4
votes
1answer
814 views

Pass colors from ls through pipe to awk 'print' statement

This is a follow-up to my question from yesterday, Show sum of file sizes in directory listing. Thanks to Zero Piraeus and a point in the right direction by Mauritz Hansen, I now have function ...
4
votes
3answers
383 views

How to parse a file to extract 3 digits numbers kept in a “group number”

I am a beginner, and trying to write a shell script to parse a text file that is extracted from a standardization pdf file. I would like for each test group (identified by Group 0, Group 1... to get ...
3
votes
4answers
543 views

merge 2 rows based on the same column values

I have a file like below. 47196436 47723284 name1 1.77273 42672249 52856963 name2 1.06061 52856963 430695 name2 1.16667 55094959 380983 name3 1.55613 17926380 55584836 name4 1.02461 3213456 34211 ...
3
votes
2answers
1k views

What is the best way to find a list of several strings within a large text file

The short, general question is: In Unix/Linux, what is the best way to find a list of several (about 150) strings within a large text file? I am asking this to all Unix/Linux experts as a general ...
3
votes
1answer
267 views

grep list of names and information from bigger file

I have two files: one with list of names (500 entries) and other having some more information for each entry in A.txt and extra entries too. File A.txt (each line is starting with > (fasta format) ...
3
votes
3answers
2k views

Using sed to edit lines in a file with a variable

I need to edit lines in a file using sed. Now the problem is I am replacing a particular pattern with a combination of text and number. This number is a variable which keeps on incrementing for every ...
2
votes
2answers
94 views

Merging two tables including multiple ocurrence of column identifiers

I would like to merge two tables: File 1: 1 today 2 tomorrow 3 red File 2: 1 a lot 1 sometimes 2 at work 2 at home 2 sometimes 3 new Desired output (file 3): 1 today ...
2
votes
2answers
63 views

Merging columns from two separate files

How to create a new file merging selective columns from two separate files using awk? Without messing up the elements orders of BOTH files. Exemple: File 3 may contain column 1,2,3 from File 1 and ...
2
votes
3answers
87 views

replace a character from a string that is in between first and second search using sed or awk

I need to replace _ (underscore) with ? (question mark) in a file containing email addresses. The file looks like below: EFT_020034-E015133 20140624 /ACC/EMAIL+test_1@mysite.com SHR ...
2
votes
3answers
1k views

Comparing text files using Bash and AWK

I have two text files, and I want to compare their corresponding values according to their rows and columns. By comparing, I mean to check if the values are equal and echo if the values are the same ...
2
votes
2answers
2k views

Diff, show modified line from right file only

I have file a and b and I would like to output lines of b that changed since it was cloned from a. Just the modified lines, no surrounding context, no diff offset marks. How can I do that using shell ...
2
votes
1answer
406 views

Replace semicolons to commas between the quotes only [duplicate]

Possible Duplicate: Remove comma between the quotes only I'm trying to use awk function to replace the semicolons between quotes to comma and it is 98% ok for me, because I want to keep the ...
2
votes
1answer
2k views

Time/Date grep for 15 mins

I need to grep a specific type of line in a specific time of the log, do you have any ideas? Here's my working script: cat *.log |grep -E '2011-06-30 (1[0-1]:[0-1][0-5]|10:16)'| grep -ach '0110 ...
1
vote
2answers
81 views

Search for a string in a line and append related lines

I have a file containing some ID (identifier) and DS (description) lines. I'd like to keep the rows containing the string value in their ID line, as well as all the following (related) DS lines before ...
1
vote
2answers
87 views

Merge csv files with conditions

I have 2 CSV files for each date (csv_2014_4_15 and csv_2014_4_16) , with basic structure and couple of unique columns as below. id,name,created_at,updated_at,other columns 12, joe, 2013-1-1 18:30, ...
1
vote
2answers
171 views

Extract data in linux/unix

I have input data which I want to parse and extract values using awk/grep/sed: group-2 9 10 text color=black,from=ABCB11,fromid=4,order=2,thickness=3,to=ACE,toid=11,use=1,z=1 group-3 0 1 text ...
1
vote
3answers
495 views

awk a specific column with unexpected line break

I can't seem to find a solution for this and it's driving me crazy. I know I can use awk to print a column(s). I'm having trouble printing a specific column though because of the way my file system is ...
1
vote
1answer
442 views

Reading two files into an IFS while loop — Is there a way to get a zero diff result in this case?

I have a text file full of several hundred lines of sequences like this: b 29. b 52. c 84. c 83. c 94. c 93. c 61. b 38. c 81. c 92. c 28. c 37. ...
1
vote
1answer
255 views

Need help processing a text file with awk to conform to CSV flat file format

I have the following problem. I collected data on reaction time from over 100 participants for an experiment I am running. Unfortunately, the separators between fields were not consistent, but after a ...
0
votes
3answers
67 views

Compare three file and print the highest value

I have three files. All of them having only one column. File1 M 0 M A File2 A 0 M A File3 M 0 A A compare each row in the three files, i.e, for first row (M,A,M) hence the maximum occurrence is ...
0
votes
2answers
5k views

Replacing one test string with another in particular column

I would like to replace "CC" with "C" and "AA" with A" in a particular column of Tab delimited file (using awk probably).