awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

5
votes
5answers
225 views

Merging columns in a file using Awk

Input: ABC,SSSD,12345,NSS,12345,xxx,TS11 ,,,,,,TS21 ,,,,,,TS22 ,,,,,,BS26 ,,,,,,GPRS ABC,SSSD,12356,NSS,12356,xxx,TS11 ,,,,,,TS21 ,,,,,,TS22 ,,,,,,GPRS ,,,,,,BS26 Output: ...
0
votes
2answers
42 views

Monitor a logfile and execute different commands depending on different conditions

I have a logfile which I want to monitor and depending on which condition is met, different commands should be executed. I found a solution that comes close to this here. Unfortunately it doesn't ...
0
votes
0answers
35 views

Subtract a Number from variable which contains a sentence

This is extended question from http://stackoverflow.com/questions/32081168/reading-a-log-file-with-different-sequences-using-shell-script I'm new to shell. I've written a shell script a code with ...
2
votes
4answers
57 views

Join pattern line and consecutive non-empty lines, separated by commas and enclosed in ()

I have a file that has click css_add_violation_false click css_add_claim_false and select css_driver1_birth_date_month birth_date_month select css_driver1_birth_date_day birth_date_day How ...
1
vote
3answers
54 views

Find lines in (muliple) files that contain the given line in an input file

So, if I have a number of files like the following: file1: 123 456 789 012 file2: line1 922 line2 392 line3 456 line5 291 line6 201 ... file3: line1 111 line2 123 line3 19 line5 542 ...
1
vote
2answers
48 views

How to subtract lines, in reverse, only if greater than previous line

I am working on a shell script that will read the lines of a file, in reverse. I need it to subtract the previous line from the latter, but only if the latter is larger. If it's smaller, then I just ...
0
votes
2answers
31 views

How to add a new text after every matched string using shell command?

Input: Job name: ns, Job ID: 2312, Status: ODB_ACTIAVTION SUCCESSFUL Job name: ps, Job ID: 3353, Status: ODB_ACTIAVTION SUCCESSFUL Job name: pm, Job ID: 1265, Status: ODB_ACTIAVTION SUCCESSFUL Job ...
2
votes
1answer
682 views

Can i update my commands( like awk,sed) to the latest release in solaris?

bash-3.00# awk '{print NR}' f1.txt 1 2 3 4 but FNR does not print anything. awk '{print FNR}' f1.txt the above line does not gives me any output,seems that the awk that i am having in my ...
10
votes
2answers
952 views

grep: display filename once, then display context with line numbers

Our source code has error codes scattered throughout. Finding them is easy with grep, but I'd like a bash function find_code that I can execute (eg. find_code ####) which will provide output along ...
1
vote
1answer
42 views

printing in awk

I have a file number.txt with following record. Number.txt: 123456,2711448,1,14-Feb-15 09:24:25,14-Jul-15 09:24:25,120,20150814163821,13-Aug-15 09:24:25,,,12-Sep-15 09:24:25,,,12-Oct-15 ...
0
votes
1answer
2k views

shell script to print rows if there is a value in column 2

1 2 3 4 3 1 4 2 1 4 3 2 4 2 1 3 How can I print all rows, on which there is a 2 in the second column.
0
votes
2answers
35 views

AWK, SED or GREP for extracting data from HTML file

Can anyone tell me which is best to use to extract the following data from a html file which was received with curl . <script>document.getElementById("test-summary").innerHTML = ...
8
votes
4answers
732 views

Splitting file for every 10000 numbers ( not lines )

I have a file that looks like the following: chr19 61336212 + 0 0 CG CGT chr19 61336213 - 0 0 CG CGG chr19 61336218 + ...
0
votes
5answers
63 views

Print all data related to a certain field

Considering below file: foo,5 foo,7 foo,9 boo,5 boo,10 boo,10 What I am looking to, is printing all the data in $2 that is related to $1 in one record. For this example the needed output will be: ...
1
vote
1answer
40 views

add pairwise values looking up from a file

I have several variables (col1) which have 3 values (col 3,4,5) for some levels (col2) mango123 L1 1 2 3 mango123 L2 4 5 6 squash378 L1 7 8 9 squash378 L5 10 12 13 squash378 L15 0 1 3 ...
3
votes
3answers
65 views

Multiple column matching and adjusting with awk

I have a problem that's giving me a bit of a headache, but it doesn't tend to take much as a relative beginner. I have a file that looks like this ID A1 A2 A3 1 A G A 2 T G A ...
6
votes
3answers
286 views

Print random lines respecting the order of the source file

I have a large text file (~500K lines) with short sentences (couple of words long). Additionally, there is some XML markup in most of the lines. Finally, the text file has been sorted before the ...
4
votes
1answer
167 views

awk/nawk alternative on SunOs and Linux

I have a script that basically checks the Java version on the box it's running on and does whatever based on the version number. My problem is that I want to use the same command on both SunOs and ...
1
vote
1answer
52 views

Awkward awk exit code

I'm writing some awk scripts for data validation. These are intended to be selected and invoked by another proces which needs to see the exit code from the awk script to determine if the content was ...
2
votes
2answers
42 views

filter based on a field value in awk

I want to print the countries which which have to after ~:. welcome~:to~:Germany welcome~:no please~:Italy welcome~:to~:Brazil welcome~:not ok~:China Note: I do not know what will be there other ...
1
vote
1answer
23 views

Compare records value with each others.

Please consider below file: foo,boo,900 foo,boo,900 foo,boo,850 I need to compare the a field ($3) with the next record, if the difference is equal or more than 50, then print the record. i.e ...
2
votes
4answers
106 views

Fill empty lines in specific column with values

I have data of the following format: 2342 1 6473 2 7463 2 3647 2734 1 9343 The lines that do not have a value in the 2nd column are empty in that spot (i.e. there is no space, tab etc.). I am ...
1
vote
3answers
39 views

Remove words (letters followed by space) from a specific column

I have a file and the format is as follows: cat dog AHF123432 | 123432 | dhfshfjdh lion AFG23412 |23412 | dfshjhfjdhj I need to remove those words from first column which contains only alphabets ...
9
votes
2answers
7k views

Conditional block vs conditional statement (if)

Say I have a file: PRO 1 GLN 5.55112e-17 ILE -6.245e-17 THR 5.55112e-17 I want every line that has a number unequal to 1 in the second column to change it to 0 and keep the rest. If I use if (i.e ...
0
votes
1answer
44 views

Array with two awk input files

I have two files, one with NF=7 and the other has NF=47 I need to check the number of occurrence of $2 of file with NF=7 if the occurrence is <=2 then I need to make an array from the same file ...
0
votes
1answer
25 views

Timestamp comparison in awk

Considering below file, how can I compare two timestamps, noting that the time part is negligible. foo,boo,2038-01-18T12:00:00+02:00 foo,boo,2015-09-12T01:31:24+03:00 ...
1
vote
2answers
71 views

Difference between dates by awk

I have the below file: 1,2,3,4,5,2015-10-13,7,8,2037-01-15 1,2,3,4,5,2015-09-05,7,8,2040-01-15 1,2,3,4,5,2015-09-15,7,8,2016-06-15 1,2,3,4,5,2015-10-13,7,8,2015-07-08 ...
4
votes
1answer
175 views

In AWK, is it possible to specify a range for operation?

Example: Given a comma separated file "foo" with 50 fields per line, I want to print $0 if $10 != the range from 1001 to 1010. I can do it using below code: awk -F, '{if($10!=1001 && ...
0
votes
1answer
25 views

Transcribe a csv column in awk, accounting for empty fields

Let's say I have a csv file with several columns. NF is not constant. At the moment, to replace columns > 4 with lowercase, I'm doing: #!/usr/bin/awk #call with `awk -f lowercase.awk in.csv > ...
0
votes
2answers
29 views

how to print tilt char in awk

I have a file str.txt with the following sample records. 31,2713810299,1,11-Aug-15 19:52:10 32,2713810833,1,11-Aug-15 21:36:18 Now I want to print output with awk as below. cat str.txt|awk -F, ...
1
vote
5answers
41 views

How to use awk or sed to convert rows to columns

I need to transpose a file. Input file: 1/1/1111 1 2 3 4 2/2/2222 5 6 7 8 Output: 1/1/1111 1 2 3 4 2/2/2222 5 6 7 8
1
vote
2answers
41 views

join two file based on column when there is no one by one corespondness in bash script (awk, grep , sed)

file1.txt 112|9305|/inst.exe 112|9305|/lkj.exe 112|9305|/dje.jar 112|9305|/ind.pdf 112|9306|/ma.exe 112|9306|/ngg.pdf 112|9307|/jhhh.dat 112|9312|/ee.dat 112|9312|/qwq.dll file2.txt ...
0
votes
2answers
67 views

Bash to join column from multiple files

I have around 20 files in different folders for which I have created a separate file containing path information called pathtofiles.in; /home/users/gray_wolf/unix/File_1.tsv ...
0
votes
1answer
58 views

How do I add a \n to the end of the record on file using AWK

I have a file that can contain 1 or many lines oracle1.txt oracle2.txt oracle3.txt How do I using AWK put a \n after the last line only so it will show oracle1.txt oracle2.txt oracle3.txt\n I ...
-1
votes
2answers
100 views

How to remove a column or multiple columns from file using shell command?

My file, ARCHIVE B1_NAME B2_NAME B3_NAME ELEMENT INFO_NAM WERT PROCID -------- -------- -------- -------- -------- -------- ---- ------ ...
0
votes
1answer
39 views

How to get a X line from file1 and Y line from file2 using shell command?

In file 1, 'Server restoration is complete now. It will be operational within few seconds. Please keep all the log file once the restoration is complete.' In file 2, 'Restoration was successfully ...
7
votes
3answers
435 views

awk does not end

I have the following code in a file named awktest1.awk: #!/bin/awk -f BEGIN{print "start"} {print $2, "\t", $5} END{print "end"} employee.txt where employee.txt contains the following data: 100 ...
-1
votes
2answers
52 views

Using awk to select column data from separate files

I am not an experienced Linux user. I am trying to select column data from 2 separate files and write to a third file using awk. I have tried to paste the files together i.e. paste file1 file2 and ...
0
votes
2answers
54 views

To add line numbers in every line using shell command?

My file, PSS-A (Primary A) PSS-B (Primary B) PSS-C (Primary C) PSS-D (Primary D) PSS-E (Primary E) PSS-F (Primary F) PSS-G (Primary G) PSS-H (Primary H) PSS-I (Primary I) SPARE ...
9
votes
9answers
23k views

How to print only last column?

echo -e 'one two three\nfour five six\nseven eight nine' one two three four five six seven eight nine how can I do some "MAGIC" do get this output?: three six nine UPDATE: I don't need it in this ...
2
votes
2answers
65 views

Delimiter being changed with fields

I have a file with values organized in columns, and separated by commas, as below file: 324,01,1,113333600000,1,, 016,01,1,134954200000,1,, 770,01,1,109069200000,1,, 853,01,1,111518800000,1,, When ...
0
votes
2answers
25 views

Total value for a field?

Considering below file: 0,2,,,10 0,2,,,15 0,1,,,984 0,2,,,9 1,14,,,5 Using awk, I need to calculate the total value in $5 per each $2. The desired output would look like below: 2,34 1,984 14,5 ...
1
vote
1answer
19 views

Removing Zeroes before and after data sample

I have several data samples that I need to grop the zeroes before and after the data sample. However there are zeroes between the data sample that I must keep for obvious reasons. How can I do this ...
0
votes
2answers
51 views

How to detect and remove newline character within a column in a csv file?

I have a large (2 Million rows) csv file exported from a SQL Server database. I don't have access to the database, and there's some newline character within a column, which makes it difficult to ...
3
votes
3answers
93 views

Organize entries as a spreadsheet (csv) using awk

I have information for entries (N= 1000) saved on a computer in a delimited text file/per entry ( every entry has one text file). The information recorded in the same format as the following: ...
21
votes
9answers
3k views

How to delete every second line from a file?

File: Data inserted into table. Total count 13 No error occurred Data inserted into table. Total count 45 No error occurred Data inserted into table. Total count 14 No error occurred Data inserted ...
5
votes
3answers
352 views

Join: Two files - but only append last two columns

Given the files: 1.txt 1, abc, 123, 456, 789 2, lmn, 123, 456, 789 3, pqr, 123, 456, 789 2.txt 1, abc, 123, 000, 000 3, lmn, 123, 000, 000 9, opq, 123, 000, 000 OUTPUT.txt ID, NAME, X, ...
4
votes
2answers
66 views

Find if any line of a file is a subset of data in another file

I have a file file1.txt with the following data: MIME_HTML_ONLY SUSPICIOUS_RECIPS SORTED_RECIPS HFILTER_HELO_5 and another file file2.txt with lines as: HFILTER_FROM_BOUNCE FORGED_OUTLOOK_HTML ...
1
vote
1answer
31 views

how to make awk print 3 items on the same line then switch to a new line

I am trying to parse the output of my top results to pickout the timestamp, used Mem and used Swap: top - 12:06:52 up 3:36, 37 users, load average: 0.00, 0.02, 0.00 Tasks: 563 total, 1 running, ...
2
votes
4answers
165 views

How to take the absolute value using awk?

If I have below two dates: 2015-09-12,2015-08-13 And I need to get the number of days between them, I will use the below code: awk -F'[-,]' '{print 360*($4-$1)+30*($5-$2)+($6-$3)}' The output ...