awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

6
votes
3answers
284 views

Print random lines respecting the order of the source file

I have a large text file (~500K lines) with short sentences (couple of words long). Additionally, there is some XML markup in most of the lines. Finally, the text file has been sorted before the ...
1
vote
1answer
49 views

Awkward awk exit code

I'm writing some awk scripts for data validation. These are intended to be selected and invoked by another proces which needs to see the exit code from the awk script to determine if the content was ...
4
votes
1answer
167 views

awk/nawk alternative on SunOs and Linux

I have a script that basically checks the Java version on the box it's running on and does whatever based on the version number. My problem is that I want to use the same command on both SunOs and ...
2
votes
2answers
42 views

filter based on a field value in awk

I want to print the countries which which have to after ~:. welcome~:to~:Germany welcome~:no please~:Italy welcome~:to~:Brazil welcome~:not ok~:China Note: I do not know what will be there other ...
1
vote
1answer
22 views

Compare records value with each others.

Please consider below file: foo,boo,900 foo,boo,900 foo,boo,850 I need to compare the a field ($3) with the next record, if the difference is equal or more than 50, then print the record. i.e ...
2
votes
4answers
106 views

Fill empty lines in specific column with values

I have data of the following format: 2342 1 6473 2 7463 2 3647 2734 1 9343 The lines that do not have a value in the 2nd column are empty in that spot (i.e. there is no space, tab etc.). I am ...
1
vote
3answers
38 views

Remove words (letters followed by space) from a specific column

I have a file and the format is as follows: cat dog AHF123432 | 123432 | dhfshfjdh lion AFG23412 |23412 | dfshjhfjdhj I need to remove those words from first column which contains only alphabets ...
0
votes
1answer
24 views

Timestamp comparison in awk

Considering below file, how can I compare two timestamps, noting that the time part is negligible. foo,boo,2038-01-18T12:00:00+02:00 foo,boo,2015-09-12T01:31:24+03:00 ...
4
votes
1answer
175 views

In AWK, is it possible to specify a range for operation?

Example: Given a comma separated file "foo" with 50 fields per line, I want to print $0 if $10 != the range from 1001 to 1010. I can do it using below code: awk -F, '{if($10!=1001 && ...
0
votes
2answers
28 views

how to print tilt char in awk

I have a file str.txt with the following sample records. 31,2713810299,1,11-Aug-15 19:52:10 32,2713810833,1,11-Aug-15 21:36:18 Now I want to print output with awk as below. cat str.txt|awk -F, ...
0
votes
1answer
24 views

Transcribe a csv column in awk, accounting for empty fields

Let's say I have a csv file with several columns. NF is not constant. At the moment, to replace columns > 4 with lowercase, I'm doing: #!/usr/bin/awk #call with `awk -f lowercase.awk in.csv > ...
1
vote
2answers
41 views

join two file based on column when there is no one by one corespondness in bash script (awk, grep , sed)

file1.txt 112|9305|/inst.exe 112|9305|/lkj.exe 112|9305|/dje.jar 112|9305|/ind.pdf 112|9306|/ma.exe 112|9306|/ngg.pdf 112|9307|/jhhh.dat 112|9312|/ee.dat 112|9312|/qwq.dll file2.txt ...
1
vote
5answers
41 views

How to use awk or sed to convert rows to columns

I need to transpose a file. Input file: 1/1/1111 1 2 3 4 2/2/2222 5 6 7 8 Output: 1/1/1111 1 2 3 4 2/2/2222 5 6 7 8
0
votes
2answers
64 views

Bash to join column from multiple files

I have around 20 files in different folders for which I have created a separate file containing path information called pathtofiles.in; /home/users/gray_wolf/unix/File_1.tsv ...
0
votes
1answer
39 views

How to get a X line from file1 and Y line from file2 using shell command?

In file 1, 'Server restoration is complete now. It will be operational within few seconds. Please keep all the log file once the restoration is complete.' In file 2, 'Restoration was successfully ...
0
votes
1answer
43 views

Array with two awk input files

I have two files, one with NF=7 and the other has NF=47 I need to check the number of occurrence of $2 of file with NF=7 if the occurrence is <=2 then I need to make an array from the same file ...
-1
votes
2answers
52 views

Using awk to select column data from separate files

I am not an experienced Linux user. I am trying to select column data from 2 separate files and write to a third file using awk. I have tried to paste the files together i.e. paste file1 file2 and ...
0
votes
2answers
54 views

To add line numbers in every line using shell command?

My file, PSS-A (Primary A) PSS-B (Primary B) PSS-C (Primary C) PSS-D (Primary D) PSS-E (Primary E) PSS-F (Primary F) PSS-G (Primary G) PSS-H (Primary H) PSS-I (Primary I) SPARE ...
7
votes
3answers
435 views

awk does not end

I have the following code in a file named awktest1.awk: #!/bin/awk -f BEGIN{print "start"} {print $2, "\t", $5} END{print "end"} employee.txt where employee.txt contains the following data: 100 ...
2
votes
2answers
65 views

Delimiter being changed with fields

I have a file with values organized in columns, and separated by commas, as below file: 324,01,1,113333600000,1,, 016,01,1,134954200000,1,, 770,01,1,109069200000,1,, 853,01,1,111518800000,1,, When ...
0
votes
2answers
25 views

Total value for a field?

Considering below file: 0,2,,,10 0,2,,,15 0,1,,,984 0,2,,,9 1,14,,,5 Using awk, I need to calculate the total value in $5 per each $2. The desired output would look like below: 2,34 1,984 14,5 ...
-1
votes
2answers
100 views

How to remove a column or multiple columns from file using shell command?

My file, ARCHIVE B1_NAME B2_NAME B3_NAME ELEMENT INFO_NAM WERT PROCID -------- -------- -------- -------- -------- -------- ---- ------ ...
1
vote
1answer
19 views

Removing Zeroes before and after data sample

I have several data samples that I need to grop the zeroes before and after the data sample. However there are zeroes between the data sample that I must keep for obvious reasons. How can I do this ...
0
votes
2answers
49 views

How to detect and remove newline character within a column in a csv file?

I have a large (2 Million rows) csv file exported from a SQL Server database. I don't have access to the database, and there's some newline character within a column, which makes it difficult to ...
3
votes
3answers
91 views

Organize entries as a spreadsheet (csv) using awk

I have information for entries (N= 1000) saved on a computer in a delimited text file/per entry ( every entry has one text file). The information recorded in the same format as the following: ...
1
vote
1answer
31 views

how to make awk print 3 items on the same line then switch to a new line

I am trying to parse the output of my top results to pickout the timestamp, used Mem and used Swap: top - 12:06:52 up 3:36, 37 users, load average: 0.00, 0.02, 0.00 Tasks: 563 total, 1 running, ...
4
votes
2answers
66 views

Find if any line of a file is a subset of data in another file

I have a file file1.txt with the following data: MIME_HTML_ONLY SUSPICIOUS_RECIPS SORTED_RECIPS HFILTER_HELO_5 and another file file2.txt with lines as: HFILTER_FROM_BOUNCE FORGED_OUTLOOK_HTML ...
2
votes
4answers
159 views

How to take the absolute value using awk?

If I have below two dates: 2015-09-12,2015-08-13 And I need to get the number of days between them, I will use the below code: awk -F'[-,]' '{print 360*($4-$1)+30*($5-$2)+($6-$3)}' The output ...
2
votes
2answers
74 views

search for exact string in unix

I have a file, xyz.txt, which has one string per line. How do I search xyz.txt for an exact match? I get an input string, eg. defabc, from another app. If defabc is found, it should return true, else ...
1
vote
2answers
46 views

Apply same field from different records to an operation.

Considering below file: 4,5,8,3,41,86,32, 0,0,0,0,38,0,8,8, 9,8,8,9,42,8,9,8, 1,1,7,1,11,2,5,1, How can I compare each $5 with the same field from the next record. Ex for the first two records: ...
1
vote
1answer
30 views

Get the filename that the record come from

I have a huge number of .gz files, which being uncompressed using zcat together. I am looking to get from where each record came from, as below example: File1: 1,2,3,4,5,6,7,NEW,9,10,11,12 ...
-1
votes
2answers
36 views

How to remove line break if line character count less than n chars [closed]

Could anyone help me to concatenate the line with next line if the line char count is less than x chars.
1
vote
1answer
32 views

Form a output by combining multiple lines

I have a log file as below: /export/home/got/logs/o2877612.01:job_1432787863184_159408 Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: ...
2
votes
1answer
94 views

Using portion of a field in awk

How to use portion of a field for awk script? The input file: 123456789,0,0,4,1,6,8,6 987654321,0,0,8,4,6,9,8 251335549,8,9,6,3,1,8,8 897465419,8,1,8,9,4,7,9 I need to select the 3rd character ...
2
votes
1answer
105 views

how to split multiline records by awk?

How to split multi line records by awk, i.e below file: ...
1
vote
3answers
31 views

Using awk to merge 2 files

I would like to merge 2 files based on column 1, 2, 3. I have tried the awk command below but is not working. awk 'NR==FNR {h[$1FS$2FS$3]=$4; next}{k=$1FS$2FS$3; if (k in h) print $1,$2,$3,$4,h[k] ...
1
vote
1answer
33 views

How to add hostname, date, and fix disk size

I'm collecting disk usage data and want to add the date, time, and hostname to this command. How can I do it in this format: YYYY-MM-DD HH:MM:SS,HOSTNAME,DISK_USAGE_DATA Command I have so far: ...
0
votes
1answer
69 views

find and replace with the value in another file

I have two files with different formats with columns tab spaced. I have to compare the columns column1, column2 of file1 with file2. If they matches, I need to replace the value in column6 of file1 ...
1
vote
2answers
42 views

Using awk to store a binary number in text file as string and then convert to hexadecimal

I'm using awk to parse a text file. The text file has four fields of byte data and appears as such: 11110000 10100000 10110000 10010000 I want to read the fields and convert them from binary to ...
0
votes
1answer
23 views

Using awk to edit numbers in a column

lets say I have a basic text file that looks like this: 91 N 91 H 1.48 95 N 95 H -0.26 97 N 97 H 1.79 98 N 98 H -0.76 I want to change columns 1 and 3 so that the integer values are n-3 like this: ...
1
vote
2answers
67 views

Difference between dates by awk

I have the below file: 1,2,3,4,5,2015-10-13,7,8,2037-01-15 1,2,3,4,5,2015-09-05,7,8,2040-01-15 1,2,3,4,5,2015-09-15,7,8,2016-06-15 1,2,3,4,5,2015-10-13,7,8,2015-07-08 ...
0
votes
1answer
45 views

AWK summary file from a filtered list

I have 2 files, the first being a file of records that tell me which rows from the second file I am interested in (I'll call this the FILTER_FILE). This file can contain duplicate lines and I only ...
21
votes
9answers
3k views

How to delete every second line from a file?

File: Data inserted into table. Total count 13 No error occurred Data inserted into table. Total count 45 No error occurred Data inserted into table. Total count 14 No error occurred Data inserted ...
-1
votes
1answer
44 views

How to add a word at the first of every lines? [duplicate]

File1: Data will be inserted on Thursday Messages has been sent at 6pm Office hours is from 9am to 6pm All the employees are present Output what I want: DMS Data will be inserted on ...
2
votes
1answer
71 views

Show nth occurrence of a record

Using awk, how to show the nth occurrence of a record and delete the other occurrence, for example" Input File: 79933,2 79933,4 79933,5 55251,1 49494,4 49494,2 49494,2 49494,1 in this example, ...
2
votes
1answer
40 views

Format sar samples live with paste and awk

I'm combining live sar samples with paste and trying to format the output with awk live. It works as expected to format the output, but it doesn't do the formatting live on each sample and instead ...
1
vote
3answers
164 views

Why do awk split() make first field to be last element in array?

I'm probably missing something really simple here, but when I say echo 'The quick brown fox jumped over the lazy dog.' | \ awk '{ split($0, WORDS, " "); for ( WORD in WORDS ) { ...
1
vote
2answers
34 views

Append mth and nth columns of a file with the columns of another file

I have 2 files having different number of columns and with huge number of rows: bash-3.00$ cat fileA a1,a2,a3,a4,a5,a6,a7,a8,a9 q,w,,,,y,u,, a,z,,,,q,n,, ......................... z,p,,,,w,e,, ...
1
vote
3answers
25 views

Extract specific value from block of data

I have the following block of data and need to extract the percent value in XYZ FREEBYTES, i.e. captured value should be equal to 64. As far as I know, I can grep and cut the data. However, the digits ...
2
votes
3answers
61 views

Extract specific value from blocks of data

I have the blocks of data which are differentiated by the global Ids as below. I need to extract the last block of data with the highest id value, in the below example, id="82". <gc type="global" ...