awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
1answer
30 views

Calculate hits within a timespan from log file [closed]

I need to use UNIX utilities that implement out-of-core divide-and-conquer algorithms so to decrease the amount of virtual memory used (cap of 1 mb). I have a large data set that is an apache log ...
3
votes
2answers
78 views

Passing a list of two PIDs to xargs only kills the first using ssh

I'm retrieving a list of two PIDs that I want to kill. My pipeline looks something like ps -ef | grep foo | grep -v grep | awk {'print $2'} | xargs kill -9 Both processes are killed when executing ...
4
votes
2answers
318 views

How do I print the (numerical) ASCII values of each character in a file?

How can I print the numerical ASCII values of each character in a text file. Like cat, but showing the ASCII values only... (hex or decimal is fine). Example output for a file containing the word ...
1
vote
3answers
72 views

Awk, print result

Why does awk '{print 3 + 3}' not work, but: echo 3|awk '{print $1 + 3}' does? Couldn't the first statement just output 6? What is the easiest way of printing the result of a calculation?
1
vote
2answers
39 views

Joining columns from files if they contain a match in another column

I have two files: File1.txt 30 40 A T match1 string1 45 65 G R match2 string2 50 78 C Y match3 string3 File2.txt match1 60 add1 50 add2 ...
0
votes
1answer
16 views

UNIX AWK : Get Logs from 24 Hours

I am trying to extract data from the log files for last 24 hours . I am trying awk -v d="$(date -d'24 hours ago' +'%Y-%m-%d %H:%M')" '$1" "$2>=d &&/ERROR/' data.log This does find ...
3
votes
1answer
70 views

How to optimize 'grep'-and-save?

I want to know on which node my calculations were running on our cluster in case the node crashes and I get no files back to know on which node I have to search. For that case I wrote a little script ...
11
votes
2answers
528 views

awk memory leak?

Base on this I'm running the command < /dev/urandom hexdump -v -e '/1 "%u\n"' | awk '{ split("0,2,4,5,7,9,11,12",a,","); for (i = 0; i < 1; i+= 0.0001) printf("%08X\n", ...
1
vote
2answers
34 views

Reorganizing lines in a text file

So I have the following file: First Line Second Line Third Line 3 0.374 2 0.777 1 0.245 Fourth Line 4 0.687 ... And I want to reorganize it so the lines beginning with an int are appended to the ...
6
votes
1answer
133 views

Make bash exit != 0 when called by AWK and interrupted with ^C

I'm having an issue with the following piece of (G)AWK script: do { ... } while (system("sleep 10")) My intention is to break the loop when the user presses ^C during the sleep, but it's not ...
0
votes
1answer
34 views

Subset rows with awk [duplicate]

I am new to Linux, but my work requires me to do some specific task now, I hope you please help me for this. I have a data set as follow: Cow_ID Age DIM 1 1424 0 1 1425 1 1 1426 2 1 ...
0
votes
4answers
44 views

Change field separator in table

I have several tables having the following format 1971-01-01 1124 1971-01-02 987 1971-01-03 954 ... Instead I would like to change the field separator in order to have the following format of table ...
0
votes
2answers
29 views

placing words corresponding to a key in a singe row

I have a symbol table of the form (key-value pairs) 01 aaluu desii 01 aaluu cipsa 01 amaruuda ilaahabaadii 02 hai 02 nahii 02 txamaatxara To start with, I am supposed to place ...
0
votes
1answer
59 views

Parsing text file using awk

I have a continuously changing text file that contains messages from the Gmail API. I need to parse the contents of it. My text file contains the id of the message, the message, sender details and the ...
1
vote
4answers
62 views

How to use printf with awk to designate the columns separated command output

I want to extract the manufacturer name, card's firmware version and status from the output of fcinfo hba-port, which looks like: root-#> fcinfo hba-port HBA Port WWN: 10000000c96a53c5 OS ...
1
vote
3answers
44 views

replace a string 6 line after matching string

I want to change notification_interval from 15 to 2 for block File_Check only. I tried changing line 6th after matching File_Check line with: sed -e '6 /File_Check/ s/15/2/g' file.txt but it did ...
2
votes
1answer
42 views

awk matching only 1 lines with same pattern

The following awk script will add an entry SAMPLE to block NOTES. I have multiple blocks with the same name. What needs to be changed in this command to add the entry SAMPLE to first block NOTES only? ...
1
vote
1answer
41 views

Multiline(in one paragraph) pattern replace use awk or sed

$cat test.txt blablabla ```{r} blabla ``` blablabla blablabla blablabla ```{r} blablabla blablabla imname <- "FIGNAME1" saveflag <- savepdf(imname) ``` ```{r} blablabla blablabla imname ...
1
vote
2answers
90 views

Replace column in one file with column from another using awk? [duplicate]

I have two files: file1.txt: 111|aaa|444 222|bbb|555 333|ccc|666 file2.txt: 111 (null) 333 replacing column two of file1 with column one of file2 Expected Output new: 111|111|444 222||555 ...
0
votes
0answers
74 views

Replace column in one file with column from another using awk? where column from another may have null values

I have two files: f1: 111|aaa|444 222|bbb|555 333|ccc|666 f2: 111 333 replacing column 2 of f1 with column 1 of f2 required output new: 111|111|444 222||555 333|333|666 I am using ...
1
vote
1answer
51 views

multi line awk regular expression [duplicate]

Consider text.txt with this content: blablabla &&&key word&&& I want all these text and numbers 123 and chars !"£$%& as output &&&key word&&& ...
0
votes
2answers
64 views

Awk run time error: not enough arguments passed

Here is the script: TYPE="${BLOCK_INSTANCE:-mem}" awk -v type=$TYPE ' /^MemTotal:/ { mem_total=$2 } /^MemFree:/ { mem_free=$2 } /^Buffers:/ { mem_free+=$2 } /^Cached:/ { mem_free+=$2 ...
1
vote
2answers
44 views

Can I access an array created in one awk script in a subsequent awk script?

I have a shell script where there are 2 awk strings. first_awk=' BEGIN { } { # create some array here.. } END{ } ' second_awk=' BEGIN { } { ## Access the array created in first awk section. } ...
0
votes
1answer
54 views

Replace certain field from file with user input for each row and save in same file

Im typing to write a script which can read file_name, delimiter and field_number. After that for each row take an input, replace the field (specified by field_number) and output to same file. For ...
3
votes
4answers
157 views

Iterating through subdirectories and get two files as arguments for awk script

I need to iterate through the subdirectories of a directory and take two of the files, as arguments of an awk script. The script will compare the two files and generate other files. I've this. But I ...
0
votes
1answer
31 views

How to replace word in specific column if it does not match an expression?

I have the following file: chr1 157784 157887 U6 0 - chr1 564813 564881 miRNA 0 + chr1 564879 564950 tRNA 0 - chr1 564952 565019 tRNA 0 + chr1 566062 566129 ...
0
votes
1answer
54 views

how to use concatenate command inside the awk command

I am trying to print the multiple columns from the file with different sub commands with the awk command. Below is my command which throws error while using concatenate command with awk awk -F\| ...
1
vote
1answer
81 views

Script to Create multipath.conf Alias

I found a script to create alias for multipath for Nimble volumes and I modified it to work in freenas the content of it is #/bin/bash # This script will scan the system for attached Nimble Volumes ...
0
votes
3answers
59 views

Print lines after pattern in several files

I have a file with the same pattern several times. Something like: time 12:00 12:32 23:22 time 10:32 1:32 15:45 I want to print the lines after the pattern, in ...
6
votes
3answers
233 views

How can I display 3 lines of text at a time onto 1 line?

I have a log file that displays data 3 lines at a time. Like this: 1 data 2 data 3 data 1 data 2 data 3 data 1 data 2 data 3 data I'd like to take each 3 lines and ...
1
vote
3answers
53 views

Iterate through subdirectories and execute an awk script against certain files

I have a directory named ../words that contains a number of subdirectories(words/blabla,words/blabla), in turn each subdirectory contains two types of files (.*1.txt) and (.*2.txt) . What I need, it ...
3
votes
2answers
123 views

How to extract several data from a file and store it in a csv file?

I have a file in the following format : 19-08-02 Name appel ok hope local merge (mk) juin nov sept ...
1
vote
2answers
33 views

Effective way of changing the format of numeric file data

So I have a big file containing the values of a matrix with the folowing format: (0,0) 0.6666 (0,1) 0.0000 (0,2) 1.3333 ... And I want to convert it to: 0 0 0.6666e+00 0 1 0.0000e+00 0 2 ...
0
votes
2answers
143 views

Convert unix timestamp to hh:mm:ss:SSS (where SSS is milliseconds) in AWK

How can I convert unix timestamp to hh:mm:ss:SSS (where SSS is milliseconds) in AWK. For example: echo 1456478048306 > time cat time | awk ....... > readable_time_format
0
votes
1answer
41 views

How to sum numbers that have matching strings

I currently have a script that results in this file : Score1 5 Score1 9 Score1 4 Score2 6 Score2 11 Score3 10 Score3 1 Score3 2 ... And then a number of other scores, in the hundreds. I want to add ...
4
votes
3answers
126 views

Delete consecutive lines in CSV with duplicate values in one field, but keep the last line

I have a long CSV file with two columns, which includes runs of consecutive duplicates like this: ... 1500,1533 1554,1678 1554,1703 1554,1728 1593,1766 ... I need to delete all of these duplicates ...
0
votes
0answers
60 views

Need some help on awk/grep command on shell script

This is in reference to Need some help on grep command on shell script I have a log file and I am trying to write a shell script to find pause/play happening or not (Success /Failure). The script ...
0
votes
4answers
56 views

Extract ranges of lines from a file, saving each range to a separate file

I have a file that looks like this 740*02/01/2016*00:00* EJ LOG COPIED OK AUTO INIT COPY DRIVE NOT CONFIGURED E1EF3901 [020t*741*02/01/2016*05:45* *TRANSACTION STARTED* [020t CARD INSERTED ...
0
votes
1answer
31 views

pattern regex for apache rewrite

I have a bunch of URLs counting to thousands i need to redirect and there are thre main patterns in URL first pattern There is a dash - starting the title url and want to remove that dash e..g ...
1
vote
2answers
68 views

Adding a number as a suffix to multiple columns

I have data that looks like: chr1 enhancer_ 875557 876449 . enhancer_ chr1 enhancer_ 876970 877622 . enhancer_ chr1 enhancer_ 995719 996320 . enhancer_ chr1 enhancer_ ...
0
votes
2answers
27 views

can't get an “awk for loop” to work, while using basename

I have many xyz files that I want to run through awk. I just want to create a new file with an "_s" appended, for each one. Having a little trouble getting the basename syntax correct. Here is what I ...
1
vote
2answers
124 views

Append line to a block of text incrementing the value from previous line

I have a text file name class.txt with follow contents. [serverClass:MAIL] whitelist.0=LATE whitelist.1=ONTIME [serverClass:LETTER] whitelist.0=FIRST whitelist.1=SECOND whitelist.2=THIRD ...
0
votes
3answers
71 views

Count all lines between a delimiter

I want to count the number of addresses attached to each device in the below example. Preferably using awk or sed. /dev/1 addr1 sometext sometext sometext addr2 sometext sometext sometext addr3 ...
5
votes
3answers
191 views

How to format floating point number with exactly 2 significant digits in bash?

I want to print the floating point number with exactly two significant digits in bash (maybe using a common tool like awk, bc, dc, perl etc.). Examples: 76543 should be printed as 76000 0.0076543 ...
0
votes
1answer
37 views

How can I replace rows containing ONLY number ' 0' with blank rows using AWK?

I am new to AWK, I am looking for a command that would replace the rows containing ONLY number 0 in their columns, with 'blank' lines. e.g. I have below input: Taj_Mahal ...
5
votes
4answers
287 views

Join two files on two common fields

I have two files file1.txt 78Z|033333157|0000001|PERD1|2150421|D|0507020|3333333311 78Z|033333157|0000001|PERD0|2160208|A|1900460|3333333311 78Z|033333157|0000001|RSAB1|2150421|D|0507070|3333333311 ...
0
votes
0answers
21 views

Order Columns in CSV file

I am trying to order all columns by column name in a csv file. What I have is something like this: name ,adress ,mobile-number Ane ,USA ,12121212 Joane ,England ,234234 and the output I ...
2
votes
2answers
83 views

Print lines of a file between two matching patterns [duplicate]

How can I print all the lines between two lines starting with one pattern for the first line and ending with another pattern for the last line? Update I guess it was a mistake to mention that this ...
-1
votes
3answers
37 views

Transposing rows into columns in absence of few rows using AWK [duplicate]

I have a text file containing below: title1 A1 title3 A3 title4 A4 title5 A5 title1 B1 title2 B2 title5 B5 title1 C1 title2 C2 title4 C4 title5 C5 title1 D1 title2 D2 title3 D3 I would ...
0
votes
2answers
64 views

awk FS with back slashes

I use the command find . -maxdepth 1 -not -type d which generates output like ./filename.1.out I pipe the find command output to awk. The goal is to split on either the literal ./ or .. I have it ...