awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

3
votes
1answer
57 views

How to edit files in-place while performing operation on the values matching the searched pattern?

Given a list files containing random text and many datetimes in ISO format in it (e.g "2012-07-02T10:47:24+02:00"), how can I find all these datetimes and add 2 hours to each one ? Ideally a solution ...
3
votes
2answers
308 views

Extract Data between comma

I have an output file which contains different numbers seperated by commas. But the issue is I do not know the number of commas. I would in turn like this data to be grep on another file. For ...
3
votes
1answer
314 views

Split file based on a pattern with leading zeros

I have a book in text format. I would like to split the book into several files where each file contains a single chapter. Therefore I'm using the following command: awk '/Chapter/{i++}{print > ...
3
votes
1answer
945 views

Make awk use bash with the system() command

Is there a way to make awk use bash instead of sh when running system commands using the system() call? I want to use some bash-specific features such as [[ ]] and < > string comparison operators: ...
3
votes
2answers
2k views

use awk to replace parentheses?

How can I use awk to replace the parentheses in these 2 scenarios? Change empty parens to the word "None" 192.168.0.24,Up,() Remove parens to only host.domain.com is left ...
3
votes
3answers
185 views

What does . match?

In working with regular expressions, I have been told that a dot character . will match everything. Except for newlines \n. Are there any other exceptions? What about the NUL character \0, or the ...
3
votes
3answers
1k views

In awk, how to print a range of fields with the original $0 delimiters intact?

How to extract a substring of $0, in awk, when that substring's position and length is based on the positions of fields in the original $0? It effectively equates to removing the first n.lead fields ...
3
votes
3answers
56 views

awk - to add a column after matching the remaining columns

I have a file (file1.txt) with the contents as below. 8145216 3377090 1.5 1405541 53595498 1.53637 I need to generate all the possible combinations for the first 2 columns in the above file. I use ...
3
votes
1answer
47 views

Automatically load a user function at startup with awk?

I guess this answer from awk separator preferences already gives a hint, but I thought I'd ask explicitly: Is there an equivalent to a .bashrc file for awk, where I could define my own library ...
3
votes
1answer
167 views

Walking Multidimensional arrays in Mawk

I'm able to do this properly in gawk, but when I tried to post my code to the machine where it will run, I realized it was using mawk... $ cat multidim.gawk # test of multidimensional arrays // { ...
3
votes
3answers
124 views

Join every other column with sed or awk

I have a large text file (666000 colums) in the format A B C D E F Desired output AB CD EF How can we do it in sed or awk. I have tried a couple of things but nothing seems to be working. Please ...
3
votes
3answers
1k views

importing data from a text file to a bash script

I need to import a text file like this below: AMBI CBNK CCXI CERE CLACW CNYD DAEG DLIA DLLR DNDN DSGX HAST HIBB, HPOL IRDMZ MARK NEPT NFEC NINE NRCIB OMER PLCC PLPM PSUN UNTK There are 25 entries, ...
3
votes
2answers
189 views

Automated partition selection in bash script

I'm trying to write a bash script to run from Linux on a CD or flashdrive that can mount Windows and change stuff, but I don't know how to go about detecting which partition/drive is Windows. My best ...
3
votes
2answers
278 views

AWK for data smoothing

I need some help to smooth some data with awk olny of column 4 and next. This is a data example: Date;time;Time_ms;A;B;C;D 23.11.2012;15:03:00;41236627696;1;2;2;3 ...
3
votes
3answers
727 views

How can awk cut certain fields and add to the end of each line?

I would like to convert a list containing bunch of svn URLs of issues: cat list.txt //svn.server.address/repos/project/module1/branches/issue-001-name1 ...
3
votes
1answer
178 views

Are there awk versions that provide syntax for computing aggregations?

From time to time I find myself writing awk scripts that compute some simple statistics. For example computing a histogram, the average of a value, the standard deviation or even the variance ... ...
3
votes
1answer
238 views

grep list of names and information from bigger file

I have two files: one with list of names (500 entries) and other having some more information for each entry in A.txt and extra entries too. File A.txt (each line is starting with > (fasta format) ...
3
votes
1answer
508 views

merging files and getting column values based on id field

bash-3.2$ cat sample.log sample.log.1 sample.log.2 ID COL1 COL2 COL4 1 col1 col2 col4 2 c1 c2 c4 3 co1 co2 co4 ID COL3 COL1 1 col3 col1 2 c3 c1 3 co3 co1 ID COL1 COL2 COL3 1 col1 ...
3
votes
3answers
1k views

How can I call bc from within awk in this case?

OK, so I love my awk, but yeah, it has precision issues, and unfortunately there's no easy way for me to install the multi-precision extensions known as gawkextlib. What I am doing is, I am working ...
3
votes
2answers
552 views

merging two column

input_file (columns are tab delimited, only column 4 contains spaces) input A B C D;<>X;<>Y 0 C D E F;<>Z;<>X 1 output A B C D;<>X;<>Y;<>apple=0 C D ...
3
votes
1answer
886 views

using system command in awk script

In an AWK script I am using a command system(date) to print the current date in a file but after this command is executed next line is also added implicitly. Is there any way in AWK to print ...
3
votes
4answers
257 views

Remove empty configuration section

Files like ~/.config/vlc/vlcrc are 99% junk if you want to version control only the configuration options. I've got a script to remove the comments, but there's a ton of empty configuration sections ...
3
votes
2answers
539 views

Using grep/sed/awk to classify log file entries

I need to process a very large log file with many lines in different formats. My goal is to extract unique line entries who have the same starting pattern, e.g. '^2011-02-21.*MyKeyword.*Error', ...
3
votes
1answer
28 views

update line based on previous line

I have a issue since I'm not familiar with awk. I have csv file generated from sar -d output converted to csv style: 12:33:41 unix,restarts 12:35:00,lofi4096,0,0.0,0,0,0.0,0.0 ...
3
votes
1answer
44 views

Reorder Multiple Line Blocks with Sed

I am trying to reorder generated output as follows; given a tex fragment as follows, I want to move the /^Constant/ down below /Dummies/, with the line before and after /^Constant/. Further, I would ...
3
votes
1answer
30 views

How to calculate (weighted) majority over columns?

I have two files: the first one contains lines where the first column is an ID, and the remaining columns are votes, e.g. 3242 -1 1 -1 1 1 1 In a first step I want to calculate the majority for ...
3
votes
3answers
158 views

awk read error bad address

I have a file that is roughly 30GB. I am using, awk ' $1 <= 2000 ' myfile.txt >> myfile_new.txt But, I get this error after a while - awk: read error (Bad address) I guess this is ...
3
votes
2answers
307 views

Replace a pattern in a file with a long string

I'm writing a little shell script to generate a directory listing. To make the output easily customizable, the script just builds a HTML-table and than should replace a specific token in a template ...
3
votes
2answers
220 views

sort a file based on length of the column/row

I need to sort a file based on the number of chars in the first column. I have no idea on how to go about this. (On Linux, so sed/awk/sort is available). An example: .abs is bla bla 12 .abc is bla ...
3
votes
1answer
1k views

File size limit exceeded in bash [closed]

I have tried this shell script on a SUSE 10 server, kernel 2.6.16.60, ext3 filesystem. The script has a line like this: cat file | awk '{print $1" "$2" "$3}' | sort -n > result The file's size ...
3
votes
2answers
766 views

simple script rotating table

I'm trying to write a simple script, that just rotates csv table. I mean I have some file: head1;head2;head3 field11;field12;field13 and so on. All I want, just to make my file ...
3
votes
1answer
290 views

Relative line offsets in awk (/pattern/+1 or /pattern/-1)

When I want to print lines between two patterns, excluding the lines with those patterns, I can do it in ex using +1 and -1 after the pattern. That is awesome. Is this possible in awk? Right now, I ...
3
votes
1answer
414 views

Split log file by time range

I have a log file and would like to cut it into copies of 15min. In the log file the lines start with: dd mmm yyyy hh:mm:ss,xxx e.g. 12 Feb 2013 16:05:02,xxx log text... and the file rotates ...
3
votes
1answer
146 views

No output from inotifywait | awk

I'm attempting to use part of a one-liner found here: Script to monitor folder for new files? When I try the following procedure I get no output whatsoever and I cannot figure out why. In terminal ...
3
votes
0answers
131 views

How to use regrex with AWK for string replacement in this example? [duplicate]

Possible Duplicate: How to use regrex with AWK for string replacement in this example? Suppose there is some text from a file: (bookmarks ("Chapter 1 Introduction 1" "#1" ("1.1 Problem ...
2
votes
6answers
1k views

Repeat each line multiple times

Would like to have each line in a file repeated a fixed number of times. e.g. have each line repeated four times: a b c becomes: a a a a b b b b c c c c I've done some searching, and there are ...
2
votes
3answers
500 views

How to merge two files based on the matching of two columns?

I have file1 likes: 0 AFFX-SNP-000541 NA 0 AFFX-SNP-002255 NA 1 rs12103 0.6401 1 rs12103_1247494 0.696 1 rs12142199 0.7672 And a file2: 0 AFFX-SNP-000541 1 0 ...
2
votes
4answers
186 views

Output some sections of a text file [closed]

I have input like the following: >gi|481443199|gb|MH447771.1| 49 0 0 0 1 50 0 0 1 0 >gi|987078004|gb|RTYJ01067061.1| 412 0 0 1 ...
2
votes
1answer
1k views

Diff the output of two `awk` commands

I'm trying to compute the difference between the output of two awk commands but my simple attempts at it seem to be failing. Here is what I'm trying: diff $(awk '{print $3}' f1.txt | sort -u) $(awk ...
2
votes
6answers
218 views

Remove lines that do not start with a pattern from a given set of patterns

I have a file that contains data like this: report aaaaaaaa - .. -th bbbbbbbbb -to ccccccccc .. --. Question: I want to remove any line that does not start with following strings : report ...
2
votes
3answers
708 views

How to truncate the second column from a dot (.)?

INPUT: 10.175.192.16 vgxyxgf.de.foo.net asdf 34efg 10.175.192.17 sdtds.de.foo.net xyyxv e5432 10.175.243.20 asdfycfb.us.foo.net dfvasf 34525 OUTPUT: 10.175.192.16 vgxyxgf ...
2
votes
3answers
3k views

Extracting column from comma separated text

I have a long comma-separated delimited file with 20K lines. Here's a sample: "","id","number1","number2","number3","number4","number5","number6","number7" ...
2
votes
4answers
675 views

shorten long lines in a log file

I keep a logfile of a program's output. The trouble is sometimes errors occur dumping very long lines of 7bit ascii data(encoded binary) I don't care to keep. These lines can be 200KB+ long before ...
2
votes
4answers
2k views

Increment number in bash variable string

VERSION="4.0beta12" NEXT_VERSION=????? Writing an automatic release script, I have the current version in a variable $VERSION, and I would like to calculate the next version as $NEXT_VERSION. Rule: ...
2
votes
4answers
204 views

How to multiply a data file with another index data file?

I want your help in multiplying my data file with another index data file. Both data files have the same number of rows. But the data file have more than 1000 columns while the index data has only one ...
2
votes
3answers
91 views

How to run awk for some number of lines.?

I want to extract some lines with awk. Is it possible to do the following task: ls -l | awk 'BEGIN FOR(i=122;i<=129;i++) FNR==i' How can I display the details from line numbers 122 to 129?
2
votes
4answers
305 views

Format a Date field of a .CSV file with multiple commas in a string field

I have a .CSV file (file.csv) whose data are all enclosed in double quotes. Sample format of the file is as below: column1,column2,column3,column4,column5,column6, column7, Column8, Column9, Column10 ...
2
votes
3answers
549 views

Comparing files line by line in awk with delimiter

file1: abc|123|check def|456|map ijk|789|globe lmn|101112|equator file2: abc|123|check def|456|map ijk|789|equator lmn|101112|globe EXPECTED OUTPUT: ijk|789|equator lmn|101112|globe Current ...
2
votes
3answers
138 views

how to create a data file with a column of date from day1 to dayN?

I was wondering if I could generate a hourly or daily time step date in a column starting from some year in the past to some day in the near past or today. to be more clear, I want to create one ...
2
votes
4answers
5k views

awk-Printing column value without new line and adding comma

input.txt EN1 EN2 EN3 EN4 EN5 output EN1,EN2,EN3,EN4,EN5 I have tried awk.But it is not printing with comma awk 'BEGIN { OFS = ","} { printf $1}' input.txt I have GNU Awk 4.0.0 version