awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
0answers
39 views

How to replace words in a string which has “|” in it [on hold]

For example line, in this loop in a shell script: for ((k=13;`awk -v f="$k" -F " " '{print $f}' line$i.txt` != "|";k++)) I want to replace the words in the line from 13th position until a "|" is ...
6
votes
4answers
158 views

How to add a header and/or footer to a sed or awk stream?

I have a bunch of output going through sed and awk. How can I prefix the output with START and suffix the answer with END? For example, if I have All this code on all these lines and all these ...
3
votes
1answer
45 views

How to use awk to indent a source file based on simple rules?

How can I indent source code based on a couple of simple rules? As an example, I've used sed and ask to transform a selenium HTML source table to the following rspec like code. How could I ...
0
votes
1answer
33 views

Print concat vars inside loop

I have a code like this: cat input | while read p; do lynx -dump "www.xyz.com/p?=$p" | grep -m 1 -A 1 "Hello" | awk -F: '{print $2}' >> output done Where the content looks like this: [$p ...
1
vote
5answers
47 views

Making a *nicely* formatted list of number of files in directories - awk & sed probably

I have lots and lots of files in a series of directories, and would like a nicely formatted list of how many files are in each directory. The directories are ordered thus: Parent directories named ...
2
votes
2answers
27 views

adding column in CSV file using awk

I need to a add a column in CSV file from an array using awk. For example, input.csv a,10 b,11 array = (100 200) output.csv should look like a,10,100 b,11,200 I tried this command but it does ...
12
votes
2answers
127 views

Fastest way to delete duplicates in large wordlist?

I need to deduplicate a large wordlist. I tried several commands and did some research here and here where they explain that the fastest way to deduplicate a wordlist seems to be using awk. awk ...
2
votes
3answers
91 views

FNR and NR variables problem

I have two files, one of them has 10 records, and the other one with 15 record, the problem is when I cat them together, always FNR=NR. For example, consider below files: File1: 1,boo 2,foo 3,boo ...
0
votes
0answers
18 views

Sort Checkpoint Firewall log into easier output

I am trying to sort out this log: Aug 27 01:23:38 1.1.1.1/1.1.1.1/1.1.1.1 fw1-loggrabber[0000]: loc=12569|time=2015-08-27 ...
0
votes
1answer
30 views

Big XML - copy data from nth Occurrences to another file

<XML> <Employee> <firstname></firstname><lastname></lastname> </Employee> <Employee> ...
1
vote
5answers
109 views

lines to columns with awk

I have to following sample output: <HARDWARE> <NAME>WIN1</NAME> <OS>Windows 7</OS> <IP>1.2.3.4</IP> <DOMAIN>contoso.com</DOMAIN> ...
3
votes
4answers
214 views

Conditional if clause in awk

Please consider below file: foo,1000,boo,A foo,1000,boo,B foo,1001,boo,B foo,1002,boo,D And we have below rules: If $2 equal 1000, $4 should be equal A If $2 equal 1001, $4 should be equal B If ...
0
votes
1answer
34 views

How to add new column, where the value is based on existing columns with awk

I have a file of multiple columns. I would like to make an additional column based on the values of 2 columns from this file. Example input: A B C D E F 1 2 T TACA A 3 2 3 4 I R 8 2 9 3 A C 9 3 ...
2
votes
3answers
42 views

Assigning “awk” or “nawk” to a variable in a shell script [closed]

I have a script to be run on both Solaris and Red Hat systems. I want to use "nawk" in Solaris and "awk" in Red Hat systems as text processing tool. For that, I have a code snippet as below in my ...
0
votes
1answer
50 views

SED - Find and replace with special characters (#, , % )

I'm aware that this is a really simple question, but I'm struggling to find a solution to this . I need to automatically find and replace in the /etc/aliases file the following section: # Person ...
3
votes
3answers
47 views

Find a pattern and insert # at the beginning of 2 line befoe that and 1 line after that

I have a big file (more than 2000 lines). Where i have to insert a # at the beginning of 2 above lines and at the beginning of 1 below line after finding a pattern. Also insert # at the beginning of ...
1
vote
1answer
43 views

How to print the line numbers with corresponding line that matches a pattern using shell command? [duplicate]

File, .false alarm is bla no. 11 .no alarm generated is bla .application will be killed is bla wall command .doc file is document file with .doc extension .no authority for this selection bla Now ...
1
vote
1answer
36 views

Identifying duplicate fields and print both with awk

I have a file with multiple columns and want to identify those where specific column values (cols 3-6) have been duplicated. The following code finds the duplicates but I want to display both ...
0
votes
5answers
68 views

Appending One File to the Second Until End of File

I have two files. The first is just a single number. This number is calculated and changes every time. The second file has a bunch of columns, and I do not know the number of rows (which can change). ...
-1
votes
1answer
28 views

replace numbers with zero while we have the lines number [closed]

I am trying to write a bash script (containing awk or sed ) to replace some numbers with zero. for example : replace line 3 second and third number, and line 4 first and third number and line 5 first ...
0
votes
2answers
25 views

awk edit column value if numeric

What I would like to do is search for lines where the first column does not begin with 'rs' or 'chr' THEN if those lines begin with a number, append 'chr' to the first column value, otherwise leave as ...
0
votes
2answers
38 views

awk if-then-else statement error

I am writing an if-then-else statement using awk in a bash script. What I would like to do is identify lines with col 1 values not matching a particular string (rs or chr) and append a prefix (chr) ...
0
votes
1answer
33 views

Repeatedly extract every second and third line of a file with 3-line data blocks

I have a file where I have data in 3-line sets. I want the 2nd and 3rd line from each set. How do I get that? For an example from: [set no][no of line in set] 11 12 13 21 22 23 31 32 33 I want: ...
4
votes
4answers
130 views

Current date in awk

Is there a constant variable in awk, that store today's date? if no, is there a way that can store today's date for daily use? let's say we have below file: boo,foo,2016-08-30 foo,boo,2016-07-31 ...
3
votes
1answer
25 views

Executing awk on each file found with find, then redirecting the result to a new filename

I am trying to take the results from find and then execute awk to print the first two columns. Then I want to redirect those results to a new file name for each file, using the existing file name with ...
1
vote
0answers
51 views

Delete the data based on the output of awk command [duplicate]

I have a file test.dat with delimited text. Delimiter is | containing 4 columns in a file. But due to some issue, there is some discrepancy in the number of columns for some of the rows. I have found ...
1
vote
2answers
20 views

Script that only counts Established, TIME_WAIT, and Closed Wait connections using Netstat

I'm trying to make a script (using Perl, but it isn't necessary) that will only count the number of Established, Time_Wait, and Closed_Wait connections on a system and print them in terminal. So far ...
-2
votes
2answers
98 views

Grouping input using awk,sed,grep

I have an input file as below: 1001 Ivanov I.I: chess 2021 Petrov P.P: chess, football 3352 Sidorov S.S: 1000 Putin V.V: judo 8773 Schwarzenegger A: judo, chess, football I expect ...
0
votes
0answers
28 views

Convert vnstat log file to csv

My goal is to log daily network activity (rx/tx) from each ethernet interface. I installed vnstat, and while it reports daily and monthly stats, it doesn't offer a nice way to convert to csv to ...
1
vote
2answers
77 views

grep first column uniq values

Two columns A and B: Column A have repeated ids and column B has different values for each corresponding repeated value. I would like to read every uniq value from Column A with only the first hit ...
1
vote
2answers
49 views

Awk & regular expression to process pgn chess games file

Below an extract of the sample.pgn file [Event "rated blitz match"] [Site "Internet Chess Server"] [Date "2015.08.20"] [Round "?"] [White "Villain"] [Black "Hero"] [Result "0-1"] [WhiteElo "1643"] ...
0
votes
1answer
42 views

Read all the values of a particular column of a file and store it in another file

I have a file.txt which contains data like below: col11 col12 col13 col21 col22 col23 I fetch 2nd column of all the available rows (no. of rows may vary) using awk '{print $2}' file.txt Now i ...
1
vote
3answers
95 views

change of date format in awk command

I have a file mdn.txt with following sample record. mdn.txt: 123456,2711448,1,20150214092425,20150714092425,120,20150814163821,,123,,,123,20150214092425,,123,,,123,20150214092425,,123,Y Now I want ...
0
votes
2answers
41 views

Monitor a logfile and execute different commands depending on different conditions

I have a logfile which I want to monitor and depending on which condition is met, different commands should be executed. I found a solution that comes close to this here. Unfortunately it doesn't ...
0
votes
0answers
35 views

Subtract a Number from variable which contains a sentence

This is extended question from http://stackoverflow.com/questions/32081168/reading-a-log-file-with-different-sequences-using-shell-script I'm new to shell. I've written a shell script a code with ...
5
votes
5answers
222 views

Merging columns in a file using Awk

Input: ABC,SSSD,12345,NSS,12345,xxx,TS11 ,,,,,,TS21 ,,,,,,TS22 ,,,,,,BS26 ,,,,,,GPRS ABC,SSSD,12356,NSS,12356,xxx,TS11 ,,,,,,TS21 ,,,,,,TS22 ,,,,,,GPRS ,,,,,,BS26 Output: ...
1
vote
3answers
54 views

Find lines in (muliple) files that contain the given line in an input file

So, if I have a number of files like the following: file1: 123 456 789 012 file2: line1 922 line2 392 line3 456 line5 291 line6 201 ... file3: line1 111 line2 123 line3 19 line5 542 ...
1
vote
2answers
48 views

How to subtract lines, in reverse, only if greater than previous line

I am working on a shell script that will read the lines of a file, in reverse. I need it to subtract the previous line from the latter, but only if the latter is larger. If it's smaller, then I just ...
0
votes
2answers
31 views

How to add a new text after every matched string using shell command?

Input: Job name: ns, Job ID: 2312, Status: ODB_ACTIAVTION SUCCESSFUL Job name: ps, Job ID: 3353, Status: ODB_ACTIAVTION SUCCESSFUL Job name: pm, Job ID: 1265, Status: ODB_ACTIAVTION SUCCESSFUL Job ...
2
votes
4answers
56 views

Join pattern line and consecutive non-empty lines, separated by commas and enclosed in ()

I have a file that has click css_add_violation_false click css_add_claim_false and select css_driver1_birth_date_month birth_date_month select css_driver1_birth_date_day birth_date_day How ...
1
vote
1answer
42 views

printing in awk

I have a file number.txt with following record. Number.txt: 123456,2711448,1,14-Feb-15 09:24:25,14-Jul-15 09:24:25,120,20150814163821,13-Aug-15 09:24:25,,,12-Sep-15 09:24:25,,,12-Oct-15 ...
0
votes
2answers
33 views

AWK, SED or GREP for extracting data from HTML file

Can anyone tell me which is best to use to extract the following data from a html file which was received with curl . <script>document.getElementById("test-summary").innerHTML = ...
8
votes
4answers
731 views

Splitting file for every 10000 numbers ( not lines )

I have a file that looks like the following: chr19 61336212 + 0 0 CG CGT chr19 61336213 - 0 0 CG CGG chr19 61336218 + ...
0
votes
5answers
63 views

Print all data related to a certain field

Considering below file: foo,5 foo,7 foo,9 boo,5 boo,10 boo,10 What I am looking to, is printing all the data in $2 that is related to $1 in one record. For this example the needed output will be: ...
1
vote
1answer
40 views

add pairwise values looking up from a file

I have several variables (col1) which have 3 values (col 3,4,5) for some levels (col2) mango123 L1 1 2 3 mango123 L2 4 5 6 squash378 L1 7 8 9 squash378 L5 10 12 13 squash378 L15 0 1 3 ...
3
votes
3answers
64 views

Multiple column matching and adjusting with awk

I have a problem that's giving me a bit of a headache, but it doesn't tend to take much as a relative beginner. I have a file that looks like this ID A1 A2 A3 1 A G A 2 T G A ...
6
votes
3answers
283 views

Print random lines respecting the order of the source file

I have a large text file (~500K lines) with short sentences (couple of words long). Additionally, there is some XML markup in most of the lines. Finally, the text file has been sorted before the ...
1
vote
1answer
47 views

Awkward awk exit code

I'm writing some awk scripts for data validation. These are intended to be selected and invoked by another proces which needs to see the exit code from the awk script to determine if the content was ...
4
votes
1answer
167 views

awk/nawk alternative on SunOs and Linux

I have a script that basically checks the Java version on the box it's running on and does whatever based on the version number. My problem is that I want to use the same command on both SunOs and ...
2
votes
2answers
42 views

filter based on a field value in awk

I want to print the countries which which have to after ~:. welcome~:to~:Germany welcome~:no please~:Italy welcome~:to~:Brazil welcome~:not ok~:China Note: I do not know what will be there other ...