awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
1answer
54 views

Replace certain field from file with user input for each row and save in same file

Im typing to write a script which can read file_name, delimiter and field_number. After that for each row take an input, replace the field (specified by field_number) and output to same file. For ...
3
votes
4answers
159 views

Iterating through subdirectories and get two files as arguments for awk script

I need to iterate through the subdirectories of a directory and take two of the files, as arguments of an awk script. The script will compare the two files and generate other files. I've this. But I ...
0
votes
1answer
32 views

How to replace word in specific column if it does not match an expression?

I have the following file: chr1 157784 157887 U6 0 - chr1 564813 564881 miRNA 0 + chr1 564879 564950 tRNA 0 - chr1 564952 565019 tRNA 0 + chr1 566062 566129 ...
0
votes
1answer
54 views

how to use concatenate command inside the awk command

I am trying to print the multiple columns from the file with different sub commands with the awk command. Below is my command which throws error while using concatenate command with awk awk -F\| '{...
1
vote
1answer
111 views

Script to Create multipath.conf Alias

I found a script to create alias for multipath for Nimble volumes and I modified it to work in freenas the content of it is #/bin/bash # This script will scan the system for attached Nimble Volumes #...
0
votes
3answers
59 views

Print lines after pattern in several files

I have a file with the same pattern several times. Something like: time 12:00 12:32 23:22 time 10:32 1:32 15:45 I want to print the lines after the pattern, in ...
6
votes
3answers
233 views

How can I display 3 lines of text at a time onto 1 line?

I have a log file that displays data 3 lines at a time. Like this: 1 data 2 data 3 data 1 data 2 data 3 data 1 data 2 data 3 data I'd like to take each 3 lines and ...
1
vote
3answers
55 views

Iterate through subdirectories and execute an awk script against certain files

I have a directory named ../words that contains a number of subdirectories(words/blabla,words/blabla), in turn each subdirectory contains two types of files (.*1.txt) and (.*2.txt) . What I need, it ...
3
votes
2answers
126 views

How to extract several data from a file and store it in a csv file?

I have a file in the following format : 19-08-02 Name appel ok hope local merge (mk) juin nov sept ...
1
vote
2answers
33 views

Effective way of changing the format of numeric file data

So I have a big file containing the values of a matrix with the folowing format: (0,0) 0.6666 (0,1) 0.0000 (0,2) 1.3333 ... And I want to convert it to: 0 0 0.6666e+00 0 1 0.0000e+00 0 2 1.3333e+...
0
votes
2answers
165 views

Convert unix timestamp to hh:mm:ss:SSS (where SSS is milliseconds) in AWK

How can I convert unix timestamp to hh:mm:ss:SSS (where SSS is milliseconds) in AWK. For example: echo 1456478048306 > time cat time | awk ....... > readable_time_format
0
votes
1answer
43 views

How to sum numbers that have matching strings

I currently have a script that results in this file : Score1 5 Score1 9 Score1 4 Score2 6 Score2 11 Score3 10 Score3 1 Score3 2 ... And then a number of other scores, in the hundreds. I want to add ...
4
votes
3answers
128 views

Delete consecutive lines in CSV with duplicate values in one field, but keep the last line

I have a long CSV file with two columns, which includes runs of consecutive duplicates like this: ... 1500,1533 1554,1678 1554,1703 1554,1728 1593,1766 ... I need to delete all of these duplicates ...
0
votes
0answers
60 views

Need some help on awk/grep command on shell script

This is in reference to Need some help on grep command on shell script I have a log file and I am trying to write a shell script to find pause/play happening or not (Success /Failure). The script ...
0
votes
4answers
56 views

Extract ranges of lines from a file, saving each range to a separate file

I have a file that looks like this 740*02/01/2016*00:00* EJ LOG COPIED OK AUTO INIT COPY DRIVE NOT CONFIGURED E1EF3901 [020t*741*02/01/2016*05:45* *TRANSACTION STARTED* [020t CARD INSERTED [...
1
vote
2answers
74 views

Adding a number as a suffix to multiple columns

I have data that looks like: chr1 enhancer_ 875557 876449 . enhancer_ chr1 enhancer_ 876970 877622 . enhancer_ chr1 enhancer_ 995719 996320 . enhancer_ chr1 enhancer_ ...
0
votes
2answers
29 views

can't get an “awk for loop” to work, while using basename

I have many xyz files that I want to run through awk. I just want to create a new file with an "_s" appended, for each one. Having a little trouble getting the basename syntax correct. Here is what I ...
1
vote
2answers
127 views

Append line to a block of text incrementing the value from previous line

I have a text file name class.txt with follow contents. [serverClass:MAIL] whitelist.0=LATE whitelist.1=ONTIME [serverClass:LETTER] whitelist.0=FIRST whitelist.1=SECOND whitelist.2=THIRD whitelist.3=...
0
votes
3answers
75 views

Count all lines between a delimiter

I want to count the number of addresses attached to each device in the below example. Preferably using awk or sed. /dev/1 addr1 sometext sometext sometext addr2 sometext sometext sometext addr3 ...
5
votes
3answers
282 views

How to format floating point number with exactly 2 significant digits in bash?

I want to print the floating point number with exactly two significant digits in bash (maybe using a common tool like awk, bc, dc, perl etc.). Examples: 76543 should be printed as 76000 0.0076543 ...
0
votes
1answer
39 views

How can I replace rows containing ONLY number ' 0' with blank rows using AWK?

I am new to AWK, I am looking for a command that would replace the rows containing ONLY number 0 in their columns, with 'blank' lines. e.g. I have below input: Taj_Mahal ...
5
votes
4answers
294 views

Join two files on two common fields

I have two files file1.txt 78Z|033333157|0000001|PERD1|2150421|D|0507020|3333333311 78Z|033333157|0000001|PERD0|2160208|A|1900460|3333333311 78Z|033333157|0000001|RSAB1|2150421|D|0507070|3333333311 ...
0
votes
0answers
22 views

Order Columns in CSV file

I am trying to order all columns by column name in a csv file. What I have is something like this: name ,adress ,mobile-number Ane ,USA ,12121212 Joane ,England ,234234 and the output I ...
2
votes
2answers
108 views

Print lines of a file between two matching patterns [duplicate]

How can I print all the lines between two lines starting with one pattern for the first line and ending with another pattern for the last line? Update I guess it was a mistake to mention that this ...
-1
votes
3answers
39 views

Transposing rows into columns in absence of few rows using AWK [duplicate]

I have a text file containing below: title1 A1 title3 A3 title4 A4 title5 A5 title1 B1 title2 B2 title5 B5 title1 C1 title2 C2 title4 C4 title5 C5 title1 D1 title2 D2 title3 D3 I would ...
0
votes
2answers
70 views

awk FS with back slashes

I use the command find . -maxdepth 1 -not -type d which generates output like ./filename.1.out I pipe the find command output to awk. The goal is to split on either the literal ./ or .. I have it ...
0
votes
0answers
43 views

Standard deviation of number next to a pattern

I have a file with lots of tags with a number next to them, set out like this: <Score>4 ... other data <Score>2 ... other data <Score>3 and so on... I have used grep to get the ...
3
votes
2answers
97 views

Find each line matching a pattern but print only the line above it

I need to find a string and need to print the line above it. case1: The same line won't have the more than one matching pattern ie) consider a file containing $cat > para returns between the ...
2
votes
3answers
70 views

Copy column with file name as column header

I wanted to organize my dataset by copying the a column from a number of text file and append the file name in the first row of the new files. Here is sample of my data file1 a a b b 1 2 ...
-1
votes
1answer
20 views

Remove some elements of a string and replace with others

How can I convert this : RS.PK.03.01.4200.03.014.01 to man03140101? What needs to be done? Text before 5th "." has to be removed : RS.PK.03.01.4200. The remaining text should be converted 03....
0
votes
6answers
102 views

Merge multiple lines in same file based on column 1

I'm still learning programming and I've tried many things but just cannot get the correct format. I have a tab delimited file with 17 columns and many (around 50.000) rows. The file is already sorted ...
1
vote
2answers
78 views

Split file into multiple files based on pattern [duplicate]

I have a large file that's a couple hundred lines long. This file is partitioned into many parts by a specific identifier, lets say 'ABC'. This line 'ABC' appears 6 times so I want 6 output files. I'm ...
2
votes
2answers
86 views

List PDF files with missing metadata

I have had a few tasks where I needed to make reports on file changes on directories from different servers and have managed to make the reports using simple Unix commands. I used: ls -R -lT /...
0
votes
1answer
20 views

awk: detect content separated by arbitrary character instead of spaces? [duplicate]

Is there a way to make awk detect content separated by commas (or any other character) instead of spaces? e.g. a file (.csv) contains a list of contacts in the format "title","first-name","last-name"...
3
votes
1answer
55 views

Replacing a & in XML correctly?

I want to replace a '&' with &amp; Except the following occurance of XML special characters. " &quot; ' &apos; < &lt; > &gt; & &amp; I tried the ...
-2
votes
1answer
78 views

How to find the word before a match

I have the following log file. I need to know if there are any fault / suspect condition using a shell script. I need to find fault and check the previous word, if it is greater than 0 then there is ...
0
votes
0answers
16 views

Loop through directory and sum value next to specific pattern producing an average

I want to loop through all the files in the directory. the files are set out like this: <Overall>4 other data <Overall>2 other data ...... I have the code: for file in .dat; do awk '...
1
vote
3answers
89 views

Parse file with Awk only when first row matches pattern

I need to interrogate the header of a CSV file, and if a column exists proceed with the data rows. Context is when the data contains columns depending on when and what emitted it. Hoping for a "pure" ...
2
votes
4answers
112 views

Sum value next to specific pattern

I have a file with multiple tags with a number next to it e.g. <Overall>4 other <tags> and data <Overall>2 other <tags> and data <Overall>3 How would I search through ...
0
votes
1answer
29 views

How to add a condition within a for loop during substitution if string is null

I am trying to add a condition in this code, that if there is null string in translation file for either string or repl[string] for example, my file input_chk.txt has following details: input_chk.txt ...
1
vote
1answer
28 views

insert string for missing value

I've run into some problems with editing a text file I've received due to some missing values in a specific column for some rows. For example, here is what the file looks like (mostly): ID10 dad ...
2
votes
2answers
72 views

Reformat date to unix time stamp in csv table

I have a .csv file containing the date and time in the format 01/20/2016 23:53:01 in the first column. My columns are separated by semicolons, i.e. 01/21/2016 03:03:01;18616;0;1 01/21/2016 03:13:01;...
3
votes
2answers
67 views

How can I do a binary search of a text file for a specific character string?

I have a text file that contains a large number of records, each on a single line. Some of the records have special characters that have been corrupted, and I'm attempting to find those by looking for ...
0
votes
1answer
66 views

replace long text string (script with MANY special characters). sed, awk, grep

My site was hacked / infected. I replaced the url of the malicious link, but other elements in the malicious script are still making my site get blocked. Without inserting a hundred or so "escapes", ...
2
votes
1answer
53 views

Regex matching on awk in command line

I'm trying to get the filename from unrar command verbose and deleting them all through command line. The unrar command output is similar to this: Extracting BucketSort.cpp ...
1
vote
2answers
73 views

How to replace multiple blank lines with a specific non-blank line?

The sample input is 1 2 3 4 5 6 7 8 9 The expected output is 1 2 3 --- 4 5 --- 6 7 8 --- 9
0
votes
3answers
244 views

Filtering content from df output

I have the command below to get the filesystem usage on a server: df -k | awk '{print $5 " " $6}' | grep % When executed, the following are the results: 97% / 0% /proc 0% /etc/mnttab 0% /dev/fd 46%...
3
votes
4answers
74 views

display lines with not more than 4 words in the second column

I have a symbol table of the form M07UP49A0870I422.wav <s> haraa keelaa <bn> </s> M07UP49A0870I423.wav <s> <horn> jau <babble> </s> ...
0
votes
1answer
29 views

Delete Non numeric row from a csv

I have used sed to allow only numeric and csv format ,I have used sed '/[^0-9, -, .]/d' as regex. It has still "_","1-0","1-1" character as o/p which worries me. sed 's/'"$(printf '\015')"'//g' ./...
0
votes
1answer
46 views

Convert a column of epochs into a time column starting from 0

What I have Hi, imagine a lot of this files, where first column is epoch, and the other(s) column(s) are some data: 1000333,34,1 1001456,56,0 1005356,34,2 What I need I need to transform them ...