awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
1answer
890 views

Variable Substitution in Awk Print Statement -v

I've spent over an hour poking at this. It can't be this hard... I want to print a column of data from one file to another file. The column I want to print is dependent on what is passed in and stuff ...
2
votes
1answer
96 views

Extracting some files from a folder and creating a list of these files

I am just a beginner with Unix and so my skills in Unix commands is limited. I am currently dealing with a folder containing about 1000 files and I have to extract some filenames from this folder and ...
2
votes
3answers
3k views

Extracting column from comma separated text

I have a long comma-separated delimited file with 20K lines. Here's a sample: "","id","number1","number2","number3","number4","number5","number6","number7" ...
4
votes
1answer
1k views

Awk to remove line if argument is encountered in a specific column

I need to cycle through an entire file of unknown size and remove any line in which a given word (passed in as argument 1) appears in a specified column. In addition, I need to keep track of how many ...
1
vote
4answers
204 views

How to process these DHCP logs to be human readable?

I have this input: Sep 23 13:43 192.168.6.200 Sep 23 13:44 192.168.6.166 Sep 23 13:45 192.168.6.200 Sep 23 13:46 192.168.6.166 Sep 23 13:47 192.168.6.200 Sep 23 13:48 192.168.6.166 Sep 23 13:49 ...
4
votes
1answer
1k views

How to find out common elements between two files?

For an example, I have 2 files having following info: File #1: 12 13 14 15 File #2: 12 1 13 2 14 2 15 6 16 7 17 8 Output File: 1 2 2 6 In the output file, I want only the second column ...
5
votes
2answers
115 views

Why does awk execute both actions?

When I run the command: awk '/from/ {print $7} /to/ { print $7}' erroMuitoDoido.txt The file is: May 19 04:44:43 server postfix/smtpd[32595]: CDAB515013: ...
3
votes
1answer
1k views

File size limit exceeded in bash [closed]

I have tried this shell script on a SUSE 10 server, kernel 2.6.16.60, ext3 filesystem. The script has a line like this: cat file | awk '{print $1" "$2" "$3}' | sort -n > result The file's size ...
4
votes
6answers
313 views

What is the easiest way to check if column A and B values goes both ways with AWK? [closed]

What is the easiest way to check if column A and B values goes both ways? Output to check: Mike John John Mike Pamela Barbara Barbara Pamela Mike Paul Roger Paul Desired output Mike <-> ...
7
votes
7answers
318 views

Swapping an unlimited number of columns

I have a file with columns. See below for an example: a b c ... z 1 2 3 ... 26 I'd like to swap all columns where the 1st becomes the last, the second becomes the one before last...etc.. z y x ...
5
votes
4answers
2k views

Concatenate lines by first column (AWK or SED)

How can I use AWK in the following situation? I want to concatenate lines that start with the same column. Only the first column is kept after the join (in this case aaa, www, hhh). The file may be ...
5
votes
1answer
1k views

Fast elimination of duplicate lines across multiple files

I have a huge amount of data in which each (data-)line should be unique. There are a lot of files in one folder in which this is already true. It is about 15GB splitted into roughly 170 files with ...
0
votes
5answers
3k views

compare two files get identical list

file1.txt (50 lines) TERYUFV00000010753 TERYUFV00000009526 file2.txt (500 lines) TERYUFV00000009526 refids_739_known_8/10_target TERYUFV00000018907 refids_12023_known_21/22_target ...
0
votes
1answer
133 views

awk extended pattern matching (embedding pattern matching in actions for already matched strings) [closed]

I want handle strings of the form: PREFIX_TYPE_N,DATA So, does the *awk (gawk, mawk, nawk) support including pattern matching in the action for already matched string? Something like this (of ...
2
votes
3answers
107 views

Need to parse a double entry table with two pairs

I have this two-entry table resulting from a program output: Variable1 : Value1 Variable2 : Value2 Variable3 : Value3 Variable4 : Value4 Variable5 : Value5 Variable6 : Value6 ...
4
votes
2answers
370 views

AWK help for filling up rest of the columns with &nbsp in file

I'm trying to get AWK filling up following text file for "empty" columns". Basic idea of finding longest line with AWK NF and checking which line has no value on that column. Then adding x times N/A ...
0
votes
2answers
194 views

Getting data from matching list

list.txt GETID_17049_knownids_1/2_Confidence_0.625_Length_2532 GETID_9248_knownids_6/10_Confidence_0.439_Length_2474 GETID_11084_knownids_3/3_Confidence_0.600_Length_1451 ...
2
votes
2answers
315 views

AWK match string, not Regex

How do I match a string with AWK? Say I have the string [(*.+ it should only match strings containing the string [(*.+. The string is entered by the user, so if you know a cli utility that can ...
16
votes
3answers
3k views

Piping from grep to awk not working

I am trying to grep the ongoing tail of file log and get the nth word from a line. Example file: $ cat > test.txt <<EOL Beam goes blah John goes hey Beam goes what? John goes forget it Beam ...
1
vote
3answers
1k views

how to sum output of awk or other expression with xargs

Suppose i have the following bash shell script: #!/bin/bash export count=0; for i in `ls ./mydoc` ;do pdfinfo ./mydoc/$i | egrep Pages |awk {'print $2'} |xargs -+ $count ; ...
8
votes
3answers
3k views

When using awk /pattern/ { print “text”} /patern/ {print “”} is there an ELSE pattern?

Let's say I have text file like: R1 12 324 3453 36 457 4 7 8 R2 34 2342 2525 25 25 26 26 2 2 R3 23 2342 32 52 54 543 643 63 R4 25 234 2342 4 234242 I want to use awk to process these lines ...
1
vote
3answers
3k views

sed/awk/perl - delete spaces between two strings/words + solaris 10

please advice how to delete the spaces between "=" to the value/string ( after the "=" character) , should be implemented by awk or sed or perl command and command must to fit for solaris OS , ( ...
4
votes
2answers
124 views

Column mismatch and substituting

input.txt (tab-delimted) TTTTOTTT00000000008 RTTTT899 5.00E-28 TTTTOTTT00000000046 RTTTWRR 3.00E-31 TTTTOTTT00000000051 2.00E-11 TTTTOTTT00000000051 7.00E-12 TTTTOTTT00000000054 ...
3
votes
1answer
238 views

grep list of names and information from bigger file

I have two files: one with list of names (500 entries) and other having some more information for each entry in A.txt and extra entries too. File A.txt (each line is starting with > (fasta format) ...
3
votes
3answers
537 views

reading entries from multiple files

I have few files like A.txt, B.txt and C.txt, each having a only single number in it i.e A.txt has 45 B.txt has 35 and C.txt has 100. How could I read these entries from each files and report with its ...
2
votes
3answers
116 views

Finding out how many programs I have not installed from a list with grep/awk in archlinux

I want to see how many packages from a list are not installed. I use archlinux, so I use pacman -Q to determine if a package is installed. If it is not installed, it returns a message like: "error: ...
3
votes
2answers
2k views

AWK-Find maximum value in a row & print with header

I need to read a file, find the maximum value in each row, and print the ID of the row, the column the maximum value came from (sno#), and the value from the associated lc# column. If the maximum ...
2
votes
2answers
273 views

Get row data for non-matching column values

I want to read the file (1600 rows) and get rows only the columns have different values (sno1, sno2, sno3 & sno4-should be not be equal value) and it should be above 50%.The example of output ...
2
votes
4answers
5k views

awk-Printing column value without new line and adding comma

input.txt EN1 EN2 EN3 EN4 EN5 output EN1,EN2,EN3,EN4,EN5 I have tried awk.But it is not printing with comma awk 'BEGIN { OFS = ","} { printf $1}' input.txt I have GNU Awk 4.0.0 version
3
votes
2answers
258 views

Print row contain maximum and minimum value

I need to read the file (contains 16K rows)and print the entire row if any of columns and all columns contains max value (100) and all columns contain min value (0).The ouput example is given ...
1
vote
2answers
219 views

Matching values within columns

I have tab-delimited text file containing about more than 1500 rows. input.txt id sno1 EN1 Nucus_2158_mri_1/2_Co_1.0_Le_3104 EN2 Nucus_2158_mri_4/2_Co_1.0_Le_3104 EN3 ...
4
votes
2answers
2k views

Working with columns - awk and sed

I am trying to parse a text file that is generated by an expec script that pulls down some information from a switch. Here is a sample output: 192 0000.0000.0000 1/g23 Dynamic ...
3
votes
3answers
1k views

Piping in awk scripts

I'm trying to write a ls wrapper that uses awk to parse the output of ls -lhF. Right now I've split the program into two files - my_ls.sh and my_ls.awk. my_ls.sh's only purpose is to pipe the output ...
0
votes
1answer
530 views

Extract value from different column and replace in another column

I am a biologist, I need one more help in text file editing. I would like to get the help of computer experts ##fsdfsd ##sdd-ver gen 5.5.7 Xm Gen CDS 1 148 . + . ...
4
votes
2answers
5k views

Compare two file columns

I have long text files with space-delimited fields: cat file1.txt Id leng sal mon 25671 34343 56565 5565 44888 56565 45554 6868 23343 23423 26226 6224 77765 88688 87464 6848 66776 23343 63463 ...
3
votes
2answers
339 views

Parsing a text-file table and aggregating information

I have long text file with the following columns, space-delimited: Id Pos Ref Var Cn SF:R1 SR He Ho NC cm|371443199 22 G A R Pass:8 0 1 0 0 ...
4
votes
2answers
290 views

How can I get awk to print both matches and non-matches?

Basically, all I want to do is alter outputs if the input matches a filter, but the part being altered isn't what is being filtered (or I would use sed). The problem so far is that my awk is only ...
8
votes
6answers
229 views

Grab certain contents of a file

So I know tools exist for this problem because I've heard about them, but I don't know what they are. I want to do something like filter out all data but the usernames in /etc/passwd. For example, I ...
9
votes
3answers
2k views

How to sort the string which combined with string + numeric using bash script?

This is the data what I want to sort. But sort treats the numeric to string, the data it no sorted as I expected. /home/files/profile1 /home/files/profile10 /home/files/profile11 ...
3
votes
6answers
1k views

Text file look-up by column

I have a file in this format: [#] OWNER_NAME NAME SIZE [6] Robottinosino Software 200 [42] Robottinosino Ideas worth zero 188 [12] ...
4
votes
2answers
259 views

only output when first columns are the same

INPUT: [USER@NOTEBOOK ~/proba] find . -type f | xargs -I "{}" md5sum "{}" | sort 18b17ef2e55f6fd2deb044943a8a769d ./yes 2b00042f7481c7b056c4b410d28f33cf ./tt.txt 2b00042f7481c7b056c4b410d28f33cf ...
2
votes
4answers
675 views

shorten long lines in a log file

I keep a logfile of a program's output. The trouble is sometimes errors occur dumping very long lines of 7bit ascii data(encoded binary) I don't care to keep. These lines can be 200KB+ long before ...
3
votes
1answer
945 views

Make awk use bash with the system() command

Is there a way to make awk use bash instead of sh when running system commands using the system() call? I want to use some bash-specific features such as [[ ]] and < > string comparison operators: ...
3
votes
3answers
1k views

Using sed to edit lines in a file with a variable

I need to edit lines in a file using sed. Now the problem is I am replacing a particular pattern with a combination of text and number. This number is a variable which keeps on incrementing for every ...
3
votes
2answers
238 views

Subtracting one text string from another leaving unique characters, perhaps using awk

Is there any way to subtract one column of data containing text from another column containing text and get third column containing unique charcters, for example using awk eg. Input ab a cd d ...
1
vote
1answer
730 views

Extract marked strings from text file using Bash

I have files which are of the following style - these are parametrised configuration files; the values within the #characters are replaced with real values from a database depending on environment. ...
-5
votes
1answer
246 views

sed and awk challenge [closed]

Here is a great challenge for someone to attack! Is there a way to achieve the following using sed and/or awk. Please see myfile.txt below: $cat myfile.txt: entry "entry-header" entry-top name ...
2
votes
3answers
1k views

Grepping several lines from Nmap output

I apologize in advance as this is a simple question. So I'm attempting to scan a Class C to identify all mail servers by IP. The vast majority of IPs aren't mail servers. I'm trying to filter out the ...
1
vote
1answer
526 views

Append timestamp to top command output

All, I'm using top -l 0 -s 1 -pid XXX -stats rsize | awk 'NR%13==0; fflush(stdout)' > output.txt to store the rsize value of a process in a file. Currently the contents of output.txt look ...
7
votes
2answers
2k views

A shell script for joining two files

I want to write a shell script that get two files A and B, and get a result like this: File A: user_a tel_a addr_a user_b tel_b addr_b File B: process_1 user_a process_2 user_a process_3 user_b ...