Recently Active 'awk' Questions - Page 4 - Unix & Linux Stack Exchange

2

votes

3answers

222 views

Regex/pattern matching problem to extract data

What is the best way to remove distances in the following lines and output cdv format. Can this be done reasonably easily on the command line? I'd like to avoid having to write a full-blow script. ...

modified Oct 24 '12 at 15:33

manatwork
9,64312847

1

vote

2answers

109 views

Extracting a multiline regex without leading tabs

I've been trying to hack a little code extraction script together, but I can't get it to work. My goal is to examine all .txt files in a directory. If it contains a line which doesn't start with a ...

find awk

modified Oct 22 '12 at 23:04

user1759695
82

2

votes

3answers

77 views

Print only lines that are completely numeric

I'd like to filter through a text file and only print the lines where each column is a valid floating point number. For example: 3 6 2 -4.2 21.2 3 x 4.2 21.2 3 2 2.2.2 Only the first line would ...

text-processing awk numeric-data

modified Oct 22 '12 at 22:47

Gilles
139k13158363

1

vote

1answer

156 views

Combining files with 4 columns each, into 4 files for each column [closed]

So I have a bunch of text files which look like this: T1 r1 a b c d r2 d e f g ... T2 r1 h i j k r2 l m n o ... And I want to make FOUR different text files in the following ...

command-line sed awk perl columns

modified Oct 22 '12 at 19:29

dubiousjim
800310

1

vote

5answers

151 views

frequency of occurence

I have a tab-delimited text file, that is around 3K lines long. I want to calculate frequency of data appearing in it, in 10,000 ranges. Input.txt Price 500 1500 10001 15000 17000 ...

sed awk perl

modified Oct 21 '12 at 13:58

Nahuel Fouilleul
8465

3

votes

2answers

282 views

merging two column

input_file (columns are tab delimited, only column 4 contains spaces) input A B C D;<>X;<>Y 0 C D E F;<>Z;<>X 1 output A B C D;<>X;<>Y;<>apple=0 C D ...

sed awk merge

modified Oct 20 '12 at 9:26

Guru
1,2995

-4

votes

2answers

252 views

Manipulating a file with awk

I have a text file in the following format: Surname, Forename: Day.Month.Year: Degree Sellen, Jo: 03.07.1986: MSc CSE Parfitt, Harry: 20.03.1984: MSc IT How can I write a shell script that uses ...

awk

modified Oct 19 '12 at 1:20

warl0ck
10.7k11166

2

votes

4answers

166 views

Output some sections of a text file [closed]

I have input like the following: >gi|481443199|gb|MH447771.1| 49 0 0 0 1 50 0 0 1 0 >gi|987078004|gb|RTYJ01067061.1| 412 0 0 1 ...

sed awk perl

modified Oct 12 '12 at 16:50

jw013
15.8k12753

2

votes

4answers

333 views

Change currency format to generic number using sed

I am working on a large csv file with all coma separated entries. The entire document is composed of plain numbers, except for a few columns reflecting currencies that are formatted as text. For ...

sed awk regular-expression

modified Oct 11 '12 at 0:44

phunehehe
7,01122680

1

vote

2answers

231 views

List info about files/directories with spaces

I am trying to use ls to get information about files and directories. My current command fails to properly get the name whenever a file/directory has a space. I am using this to list all ...

bash shell shell-script awk ls

modified Oct 10 '12 at 7:57

Krzysztof Adamski
1,538310

2

votes

1answer

490 views

file, awk and iconv to automatically adapt tex-file encoding

I have a set of tex-files with mixed encodings, e.g. (subset of output of file -i *.tex) f1.tex: text/plain; charset=utf-8 f2.tex: text/plain; charset=utf-8 f3.tex: text/x-tex; charset=us-ascii ...

awk conversion character-encoding

modified Oct 8 '12 at 8:00

moooeeeep
23915

1

vote

3answers

561 views

bash merge 2 lists removing duplicated

I have an embedded linux system using Busybox (OpenWRT) - so commands are limited. I have two files that look like: first file aaaaaa bbbbbb cccccc mmmmmm nnnnnn second file mmmmmm nnnnnn ...

bash grep sed awk busybox

modified Oct 6 '12 at 19:08

nowy1
766

2

votes

3answers

405 views

Average rows with same first column

Given a file with two columns: Id ht 510 69 510 67 510 65 5l0 62 510 59 601 29 601 26 601 21 601 20 I need a way to coalesce all rows with the same ID into one that has an average height. In this ...

sed grep awk perl

modified Oct 2 '12 at 16:19

sputnick
3,682813

3

votes

3answers

1k views

AWK - how to reduce the greed of a regular expression?

I have an example: echo "@article{gjn2010jucs, Author = {Grzegorz J. Nalepa}, " | awk '{ sub(/@.*,/,""); print }' Is it possible to write a regular expression that selects the shorter pattern? ...

awk regular-expression

modified Oct 2 '12 at 5:55

nowy1
766

1

vote

1answer

365 views

Variable Substitution in Awk Print Statement -v

I've spent over an hour poking at this. It can't be this hard... I want to print a column of data from one file to another file. The column I want to print is dependent on what is passed in and stuff ...

awk cut variable-substitution

modified Sep 29 '12 at 19:10

ire_and_curses
3,892919

2

votes

1answer

82 views

Extracting some files from a folder and creating a list of these files

I am just a beginner with Unix and so my skills in Unix commands is limited. I am currently dealing with a folder containing about 1000 files and I have to extract some filenames from this folder and ...

shell awk

modified Sep 25 '12 at 23:57

Gilles
139k13158363

2

votes

3answers

1k views

Extracting column from comma separated text

I have a long comma-separated delimited file with 20K lines. Here's a sample: "","id","number1","number2","number3","number4","number5","number6","number7" ...

text-processing sed awk perl csv

modified Sep 25 '12 at 1:33

glenn jackman
5,615514

4

votes

1answer

723 views

Awk to remove line if argument is encountered in a specific column

I need to cycle through an entire file of unknown size and remove any line in which a given word (passed in as argument 1) appears in a specified column. In addition, I need to keep track of how many ...

text-processing awk

modified Sep 24 '12 at 6:27

Stephane Chazelas
29.3k32164

1

vote

4answers

133 views

How to process these DHCP logs to be human readable?

I have this input: Sep 23 13:43 192.168.6.200 Sep 23 13:44 192.168.6.166 Sep 23 13:45 192.168.6.200 Sep 23 13:46 192.168.6.166 Sep 23 13:47 192.168.6.200 Sep 23 13:48 192.168.6.166 Sep 23 13:49 ...

text-processing awk logs

modified Sep 24 '12 at 3:47

Craig Sanders
4,015513

4

votes

1answer

688 views

How to find out common elements between two files?

For an example, I have 2 files having following info: File #1: 12 13 14 15 File #2: 12 1 13 2 14 2 15 6 16 7 17 8 Output File: 1 2 2 6 In the output file, I want only the second column ...

shell-script text-processing awk sort

modified Sep 21 '12 at 23:10

Gilles
139k13158363

5

votes

2answers

107 views

Why does awk execute both actions?

When I run the command: awk '/from/ {print $7} /to/ { print $7}' erroMuitoDoido.txt The file is: May 19 04:44:43 server postfix/smtpd[32595]: CDAB515013: ...

awk

modified Sep 21 '12 at 10:11

Gilles
139k13158363

3

votes

1answer

490 views

File size limit exceeded in bash [closed]

I have tried this shell script on a SUSE 10 server, kernel 2.6.16.60, ext3 filesystem. The script has a line like this: cat file | awk '{print $1" "$2" "$3}' | sort -n > result The file's size ...

shell awk sort cat

modified Sep 20 '12 at 9:30

H.-Dirk Schmitt
70517

4

votes

6answers

248 views

What is the easiest way to check if column A and B values goes both ways with AWK? [closed]

What is the easiest way to check if column A and B values goes both ways? Output to check: Mike John John Mike Pamela Barbara Barbara Pamela Mike Paul Roger Paul Desired output Mike <-> ...

bash text-processing awk python perl

modified Sep 18 '12 at 6:10

Paddy3118
21014

5

votes

7answers

250 views

Swapping an unlimited number of columns

I have a file with columns. See below for an example: a b c ... z 1 2 3 ... 26 I'd like to swap all columns where the 1st becomes the last, the second becomes the one before last...etc.. z y x ...

text-processing sed awk columns

modified Sep 17 '12 at 22:37

Stephane Chazelas
29.3k32164

4

votes

4answers

983 views

Concatenate lines by first column (AWK or SED)

How can I use AWK in the following situation? I want to concatenate lines that start with the same column. Only the first column is kept after the join (in this case aaa, www, hhh). The file may be ...

sed awk

modified Sep 11 '12 at 15:23

ire_and_curses
3,892919

5

votes

1answer

771 views

Fast elimination of duplicate lines across multiple files

I have a huge amount of data in which each (data-)line should be unique. There are a lot of files in one folder in which this is already true. It is about 15GB splitted into roughly 170 files with ...

bash files awk performance

modified Sep 11 '12 at 12:47

Stephane Chazelas
29.3k32164

3

votes

2answers

343 views

What is the best way to find a list of several strings within a large text file

The short, general question is: In Unix/Linux, what is the best way to find a list of several (about 150) strings within a large text file? I am asking this to all Unix/Linux experts as a general ...

shell grep sed awk ibm-unix-system-services

modified Sep 9 '12 at 19:18

GBH
211

0

votes

5answers

1k views

compare two files get identical list

file1.txt (50 lines) TERYUFV00000010753 TERYUFV00000009526 file2.txt (500 lines) TERYUFV00000009526 refids_739_known_8/10_target TERYUFV00000018907 refids_12023_known_21/22_target ...

grep sed awk

modified Sep 9 '12 at 15:35

Kevin
10.6k31645

0

votes

1answer

103 views

awk extended pattern matching (embedding pattern matching in actions for already matched strings) [closed]

I want handle strings of the form: PREFIX_TYPE_N,DATA So, does the *awk (gawk, mawk, nawk) support including pattern matching in the action for already matched string? Something like this (of ...

awk string patterns regular-expression

modified Sep 5 '12 at 15:35

janos
1,18129

2

votes

3answers

81 views

Need to parse a double entry table with two pairs

I have this two-entry table resulting from a program output: Variable1 : Value1 Variable2 : Value2 Variable3 : Value3 Variable4 : Value4 Variable5 : Value5 Variable6 : Value6 ...

sed grep awk perl

modified Sep 3 '12 at 14:19

Thor
4,721827

4

votes

2answers

194 views

AWK help for filling up rest of the columns with &nbsp in file

I'm trying to get AWK filling up following text file for "empty" columns". Basic idea of finding longest line with AWK NF and checking which line has no value on that column. Then adding x times N/A ...

text-processing awk

modified Aug 31 '12 at 23:10

Gilles
139k13158363

0

votes

2answers

168 views

Getting data from matching list

list.txt GETID_17049_knownids_1/2_Confidence_0.625_Length_2532 GETID_9248_knownids_6/10_Confidence_0.439_Length_2474 GETID_11084_knownids_3/3_Confidence_0.600_Length_1451 ...

sed awk perl

modified Aug 30 '12 at 16:20

Thor
4,721827

2

votes

2answers

216 views

AWK match string, not Regex

How do I match a string with AWK? Say I have the string [(*.+ it should only match strings containing the string [(*.+. The string is entered by the user, so if you know a cli utility that can ...

awk regular-expression

modified Aug 30 '12 at 14:06

Thor
4,721827

14

votes

3answers

873 views

Piping from grep to awk not working

I am trying to grep the ongoing tail of file log and get the nth word from a line. Example file: $ cat > test.txt <<EOL Beam goes blah John goes hey Beam goes what? John goes forget it Beam ...

bash grep rhel awk tail

modified Aug 30 '12 at 12:20

Craig Sanders
4,015513

1

vote

3answers

546 views

how to sum output of awk or other expression with xargs

Suppose i have the following bash shell script: #!/bin/bash export count=0; for i in `ls ./mydoc` ;do pdfinfo ./mydoc/$i | egrep Pages |awk {'print $2'} |xargs -+ $count ; ...

bash shell awk xargs

modified Aug 30 '12 at 1:34

Craig Sanders
4,015513

8

votes

3answers

2k views

When using awk /pattern/ { print “text”} /patern/ {print “”} is there an ELSE pattern?

Let's say I have text file like: R1 12 324 3453 36 457 4 7 8 R2 34 2342 2525 25 25 26 26 2 2 R3 23 2342 32 52 54 543 643 63 R4 25 234 2342 4 234242 I want to use awk to process these lines ...

awk

modified Aug 29 '12 at 21:56

jw013
15.8k12753

1

vote

3answers

1k views

sed/awk/perl - delete spaces between two strings/words + solaris 10

please advice how to delete the spaces between "=" to the value/string ( after the "=" character) , should be implemented by awk or sed or perl command and command must to fit for solaris OS , ( ...

solaris perl sed awk

modified Aug 29 '12 at 13:00

yael

3

votes

2answers

100 views

Column mismatch and substituting

input.txt (tab-delimted) TTTTOTTT00000000008 RTTTT899 5.00E-28 TTTTOTTT00000000046 RTTTWRR 3.00E-31 TTTTOTTT00000000051 2.00E-11 TTTTOTTT00000000051 7.00E-12 TTTTOTTT00000000054 ...

text-processing sed awk perl columns

modified Aug 27 '12 at 6:35

Paddy3118
21014

3

votes

1answer

186 views

grep list of names and information from bigger file

I have two files: one with list of names (500 entries) and other having some more information for each entry in A.txt and extra entries too. File A.txt (each line is starting with > (fasta format) ...

text-processing grep awk

modified Aug 26 '12 at 14:48

Gilles
139k13158363

2

votes

3answers

168 views

reading entries from multiple files

I have few files like A.txt, B.txt and C.txt, each having a only single number in it i.e A.txt has 45 B.txt has 35 and C.txt has 100. How could I read these entries from each files and report with its ...

awk wc

modified Aug 23 '12 at 13:45

jw013
15.8k12753

13

votes

9answers

6k views

Tool in UNIX to subtract dates

Is there any tool in Solaris UNIX (so no GNU tool available) to subtract dates? I know that in Linux we have gawk that can subtract one date from another. But in Solaris the maximum we have is nawk ...

shell solaris awk utilities date

modified Aug 15 '12 at 15:40

nolan6000
2887

1

vote

3answers

96 views

Finding out how many programs I have not installed from a list with grep/awk in archlinux

I want to see how many packages from a list are not installed. I use archlinux, so I use pacman -Q to determine if a package is installed. If it is not installed, it returns a message like: "error: ...

linux grep awk pacman

modified Aug 14 '12 at 18:08

Ulrich Dangel
10.9k11633

3

votes

2answers

999 views

AWK-Find maximum value in a row & print with header

I need to read a file, find the maximum value in each row, and print the ID of the row, the column the maximum value came from (sno#), and the value from the associated lc# column. If the maximum ...

text-processing awk text

modified Aug 7 '12 at 21:42

Peter.O
8,01531549

2

votes

2answers

189 views

Get row data for non-matching column values

I want to read the file (1600 rows) and get rows only the columns have different values (sno1, sno2, sno3 & sno4-should be not be equal value) and it should be above 50%.The example of output ...

sed awk perl

modified Aug 7 '12 at 20:25

jw013
15.8k12753

2

votes

4answers

2k views

awk-Printing column value without new line and adding comma

input.txt EN1 EN2 EN3 EN4 EN5 output EN1,EN2,EN3,EN4,EN5 I have tried awk.But it is not printing with comma awk 'BEGIN { OFS = ","} { printf $1}' input.txt I have GNU Awk 4.0.0 version

text-processing awk

modified Aug 4 '12 at 19:37

Peter.O
8,01531549

3

votes

2answers

171 views

Print row contain maximum and minimum value

I need to read the file (contains 16K rows)and print the entire row if any of columns and all columns contains max value (100) and all columns contain min value (0).The ouput example is given ...

sed awk python perl

modified Aug 3 '12 at 17:45

Tim Kennedy
4,728417

3

votes

5answers

478 views

extract fields from “tail -f” of a syslog stream

Example line from syslog file: Aug 1 10:25:50 10.10.10.1 id=firewall sn=XXXX time="2012-08-01 14:35:18 UTC" fw=x.x.x.x pri=6 c=1024 m=537 msg="Connection Closed" f=11 n=195273698 ...

sed awk syslog text

modified Aug 2 '12 at 14:08

sebnukem
463

1

vote

2answers

143 views

Matching values within columns

I have tab-delimited text file containing about more than 1500 rows. input.txt id sno1 EN1 Nucus_2158_mri_1/2_Co_1.0_Le_3104 EN2 Nucus_2158_mri_4/2_Co_1.0_Le_3104 EN3 ...

sed awk

modified Jul 28 '12 at 0:46

Chandra Ravoori
84827

3

votes

2answers

1k views

Working with columns - awk and sed

I am trying to parse a text file that is generated by an expec script that pulls down some information from a switch. Here is a sample output: 192 0000.0000.0000 1/g23 Dynamic ...

bash sed awk

modified Jul 25 '12 at 23:10

lesmana
3,21711433

3

votes

3answers

876 views

Piping in awk scripts

I'm trying to write a ls wrapper that uses awk to parse the output of ls -lhF. Right now I've split the program into two files - my_ls.sh and my_ls.awk. my_ls.sh's only purpose is to pipe the output ...

awk io-redirection

modified Jul 18 '12 at 0:49

laebshade
982210

Tagged Questions

Related Tags