awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

2
votes
2answers
488 views

AWK match string, not Regex

How do I match a string with AWK? Say I have the string [(*.+ it should only match strings containing the string [(*.+. The string is entered by the user, so if you know a cli utility that can ...
16
votes
3answers
8k views

Piping from grep to awk not working

I am trying to grep the ongoing tail of file log and get the nth word from a line. Example file: $ cat > test.txt <<EOL Beam goes blah John goes hey Beam goes what? John goes forget it Beam ...
1
vote
3answers
2k views

how to sum output of awk or other expression with xargs

Suppose i have the following bash shell script: #!/bin/bash export count=0; for i in `ls ./mydoc` ;do pdfinfo ./mydoc/$i | egrep Pages |awk {'print $2'} |xargs -+ $count ; ...
11
votes
3answers
5k views

When using awk /pattern/ { print “text”} /patern/ {print “”} is there an ELSE pattern?

Let's say I have text file like: R1 12 324 3453 36 457 4 7 8 R2 34 2342 2525 25 25 26 26 2 2 R3 23 2342 32 52 54 543 643 63 R4 25 234 2342 4 234242 I want to use awk to process these lines ...
4
votes
2answers
159 views

Column mismatch and substituting

input.txt (tab-delimted) TTTTOTTT00000000008 RTTTT899 5.00E-28 TTTTOTTT00000000046 RTTTWRR 3.00E-31 TTTTOTTT00000000051 2.00E-11 TTTTOTTT00000000051 7.00E-12 TTTTOTTT00000000054 ...
3
votes
1answer
315 views

grep list of names and information from bigger file

I have two files: one with list of names (500 entries) and other having some more information for each entry in A.txt and extra entries too. File A.txt (each line is starting with > (fasta format) ...
3
votes
3answers
1k views

reading entries from multiple files

I have few files like A.txt, B.txt and C.txt, each having a only single number in it i.e A.txt has 45 B.txt has 35 and C.txt has 100. How could I read these entries from each files and report with its ...
2
votes
3answers
131 views

Finding out how many programs I have not installed from a list with grep/awk in archlinux

I want to see how many packages from a list are not installed. I use archlinux, so I use pacman -Q to determine if a package is installed. If it is not installed, it returns a message like: "error: ...
3
votes
2answers
3k views

AWK-Find maximum value in a row & print with header

I need to read a file, find the maximum value in each row, and print the ID of the row, the column the maximum value came from (sno#), and the value from the associated lc# column. If the maximum ...
2
votes
2answers
351 views

Get row data for non-matching column values

I want to read the file (1600 rows) and get rows only the columns have different values (sno1, sno2, sno3 & sno4-should be not be equal value) and it should be above 50%.The example of output ...
3
votes
2answers
368 views

Print row contain maximum and minimum value

I need to read the file (contains 16K rows)and print the entire row if any of columns and all columns contains max value (100) and all columns contain min value (0).The ouput example is given ...
1
vote
2answers
296 views

Matching values within columns

I have tab-delimited text file containing about more than 1500 rows. input.txt id sno1 EN1 Nucus_2158_mri_1/2_Co_1.0_Le_3104 EN2 Nucus_2158_mri_4/2_Co_1.0_Le_3104 EN3 ...
4
votes
2answers
4k views

Working with columns - awk and sed

I am trying to parse a text file that is generated by an expec script that pulls down some information from a switch. Here is a sample output: 192 0000.0000.0000 1/g23 Dynamic ...
3
votes
3answers
2k views

Piping in awk scripts

I'm trying to write a ls wrapper that uses awk to parse the output of ls -lhF. Right now I've split the program into two files - my_ls.sh and my_ls.awk. my_ls.sh's only purpose is to pipe the output ...
0
votes
1answer
676 views

Extract value from different column and replace in another column

I am a biologist, I need one more help in text file editing. I would like to get the help of computer experts ##fsdfsd ##sdd-ver gen 5.5.7 Xm Gen CDS 1 148 . + . ...
4
votes
2answers
8k views

Compare two file columns

I have long text files with space-delimited fields: cat file1.txt Id leng sal mon 25671 34343 56565 5565 44888 56565 45554 6868 23343 23423 26226 6224 77765 88688 87464 6848 66776 23343 63463 ...
3
votes
2answers
537 views

Parsing a text-file table and aggregating information

I have long text file with the following columns, space-delimited: Id Pos Ref Var Cn SF:R1 SR He Ho NC cm|371443199 22 G A R Pass:8 0 1 0 0 ...
4
votes
2answers
434 views

How can I get awk to print both matches and non-matches?

Basically, all I want to do is alter outputs if the input matches a filter, but the part being altered isn't what is being filtered (or I would use sed). The problem so far is that my awk is only ...
8
votes
6answers
242 views

Grab certain contents of a file

So I know tools exist for this problem because I've heard about them, but I don't know what they are. I want to do something like filter out all data but the usernames in /etc/passwd. For example, I ...
10
votes
3answers
5k views

How to sort the string which combined with string + numeric using bash script?

This is the data what I want to sort. But sort treats the numeric to string, the data it no sorted as I expected. /home/files/profile1 /home/files/profile10 /home/files/profile11 ...
3
votes
6answers
2k views

Text file look-up by column

I have a file in this format: [#] OWNER_NAME NAME SIZE [6] Robottinosino Software 200 [42] Robottinosino Ideas worth zero 188 [12] ...
4
votes
2answers
280 views

only output when first columns are the same

INPUT: [USER@NOTEBOOK ~/proba] find . -type f | xargs -I "{}" md5sum "{}" | sort 18b17ef2e55f6fd2deb044943a8a769d ./yes 2b00042f7481c7b056c4b410d28f33cf ./tt.txt 2b00042f7481c7b056c4b410d28f33cf ...
3
votes
4answers
989 views

shorten long lines in a log file

I keep a logfile of a program's output. The trouble is sometimes errors occur dumping very long lines of 7bit ascii data(encoded binary) I don't care to keep. These lines can be 200KB+ long before ...
3
votes
1answer
2k views

Make awk use bash with the system() command

Is there a way to make awk use bash instead of sh when running system commands using the system() call? I want to use some bash-specific features such as [[ ]] and < > string comparison operators: ...
3
votes
3answers
2k views

Using sed to edit lines in a file with a variable

I need to edit lines in a file using sed. Now the problem is I am replacing a particular pattern with a combination of text and number. This number is a variable which keeps on incrementing for every ...
3
votes
2answers
344 views

Subtracting one text string from another leaving unique characters, perhaps using awk

Is there any way to subtract one column of data containing text from another column containing text and get third column containing unique charcters, for example using awk eg. Input ab a cd d ...
1
vote
1answer
887 views

Extract marked strings from text file using Bash

I have files which are of the following style - these are parametrised configuration files; the values within the #characters are replaced with real values from a database depending on environment. ...
-4
votes
1answer
345 views

sed and awk challenge [closed]

Here is a great challenge for someone to attack! Is there a way to achieve the following using sed and/or awk. Please see myfile.txt below: $cat myfile.txt: entry "entry-header" entry-top name ...
2
votes
3answers
2k views

Grepping several lines from Nmap output

I apologize in advance as this is a simple question. So I'm attempting to scan a Class C to identify all mail servers by IP. The vast majority of IPs aren't mail servers. I'm trying to filter out the ...
1
vote
1answer
732 views

Append timestamp to top command output

All, I'm using top -l 0 -s 1 -pid XXX -stats rsize | awk 'NR%13==0; fflush(stdout)' > output.txt to store the rsize value of a process in a file. Currently the contents of output.txt look ...
7
votes
2answers
3k views

A shell script for joining two files

I want to write a shell script that get two files A and B, and get a result like this: File A: user_a tel_a addr_a user_b tel_b addr_b File B: process_1 user_a process_2 user_a process_3 user_b ...
2
votes
2answers
228 views

How to make a list and count LaTeX keywords in a source file?

I am a beginner, trying to make lists (and eventually count and make statistics) the LaTeX keywords appearing in some documents. Basically a LaTeX "valid" keyword can be identified by: The fact ...
2
votes
2answers
266 views

How do I ad an if statement (regarding punctuation in a word) to this bash script

I have this bash script #!/bin/bash cat $@ | while read line do for word in $line do echo $word | circling-the-square # here's where i need to add the if statement: ...
13
votes
6answers
8k views

How can you combine all lines that end with a backslash character?

Using a common command line tool like sed or awk, is it possible to join all lines that end with a given character, like a backslash? For example, given the file: foo bar \ bash \ baz dude \ happy ...
1
vote
2answers
135 views

How to parse fields out of a text file and write them to another file

I have executed a database command and have generated an output file of the results. Each field name starts with dbt_xxxxxx Each field value is the value after the equal sign. how can I parse the file ...
4
votes
3answers
416 views

How to parse a file to extract 3 digits numbers kept in a “group number”

I am a beginner, and trying to write a shell script to parse a text file that is extracted from a standardization pdf file. I would like for each test group (identified by Group 0, Group 1... to get ...
1
vote
1answer
1k views

AWK - change the file name (special characters)

I have two files: ~/Pulpit/kot$ find . -name "*jpg" -printf "%f\n" 1.jpg `[~!($%^_&*){.}\___"`]`1.jpg I want to change their name to: _home_pic_Pulpit_kot_1.jpg ...
4
votes
1answer
3k views

escaping a single dot with double backslash - awk

"effective awk programming" book has an example on Field-Splitting. here is the example: If you want fields to be separated by a literal period followed by any single character, use ‘FS = ...
1
vote
2answers
3k views

Using awk to process ls output with spaces in filenames/paths

All, I have a script in which I use find command to filter out all png files in a given folder and list them along with their size. I want the output in the following format: someFile.png => 1.2K ...
2
votes
3answers
5k views

How to delete files filtered out by awk

I have the following files in a directory: -rw-r--r-- 1 smsc sys 46 Apr 22 12:09 bills.50.1.3G.MO.X.20120422120453.Z -rw-r--r-- 1 smsc sys 28 Apr 22 12:15 bills.50.1.3G.MO.X.20120422120953.Z ...
0
votes
2answers
2k views

nmap script needed to output text into csv format

ultimately I am looking to insert some info into a database. So I'm trying to output a file from something (probably nmap) that is formatted like this: 10.1.150.1,Up,1/3/2012,host.domain.lan ...
3
votes
3answers
377 views

swapping fields in columns by fixed pattern (yes, no, no, yes)

I want to produce a following list of pages (for imposition work); (number of pages is variable, but I make a sample of my needs using a list of 16 pages, logic is the same for longer lists) 1,16 8,9 ...
4
votes
3answers
4k views

How to list all unique ip address currently connected to a specific port?

Say I want to know how many unique clients are connected to port 5222 on a server. Can you find a way better/faster/stronger than this? netstat -nt | grep ':5222.*ESTABLISHED' | awk '{ print $5 }' \ ...
0
votes
1answer
851 views

Using Awk To Parse Individual Lines

Here is the output from qacct -o Total : OWNER WALLCLOCK UTIME STIME CPU MEMORY IO IOW ...
1
vote
2answers
1k views

How can I delete lines that has an empty first column?

INPUT: CC2352345|m,safnlasndfmnyxdcvyxcvyxcv |klasjdf |3|lasdjflasdf| |2345567356 CC3543353|asdfasdffghntz |klasjdf |3|lsajdfl | ...
4
votes
2answers
742 views

Why does this awk command not play as well with find as sed does?

I am very much a newbie at Unix/Linux command line stuff, and mostly get to where I'm going by copying, pasting, and modifying code I find on the internet. Just mentioning that in hopes people will ...
1
vote
3answers
1k views

Combining Awk with Curl

How would I set my curl statement so that it puts the value of each $3 in for the username, and the value of each $4 for the wallclock information? Some further information I am trying to take the ...
2
votes
1answer
3k views

Use AWK add new line if match exists

I have a file with the following format: aaa@ bbb bbb@ ccc& ddddd@ aaaf ccc@ ddd& eee: abcd& dassda The output should be: aaaa@ bbb bbb@ ccc& ddddd@ aaaf ccc@ ddd& eee@ ...
2
votes
2answers
554 views

Turn a bash and php script into a distributable rpm

I'm semi unix knowledgeable and I want to be able to call down an rpm from my server, run it with something like rpm -i , and have access to the script anywhere. The script is a combination awk and ...
3
votes
2answers
967 views

simple script rotating table

I'm trying to write a simple script, that just rotates csv table. I mean I have some file: head1;head2;head3 field11;field12;field13 and so on. All I want, just to make my file ...