awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
2answers
50 views

insert complex line before or after complex line

i have file with complex text print("ERROR: passwords don't match") password = hash_func(password.encode("UTF-8")).hexdigest() i need to insert this text between them with ...
1
vote
3answers
43 views

Count occurance of number and delete line of matches value

Could I get some help please with my problem? I have a file that looks like this (comma separated): A,250,13,820,0,80,201 B,0,0,0,0,3,0 C,0,2,1,0,0,0 D,0,1,105,0,0,10 I would like to ...
0
votes
2answers
53 views

How to output some data to different cells of an Excel File?

Example: I have a file that has following data: cvrta2100 Error 404 Fixed cvrta2111 Licensed Needs Fixing cvrta2123 Error 404 Fixed cvrta2333 Licensed Needs Fixing I am using awk to print ...
6
votes
4answers
7k views

Redirecting Output of print to multiples files in awk

I want to print same lines to 2 different files. What I am doing right now is: print ID[i]" "Value[i]" "TS1[i]" "TS2[i] > "file1.txt"; print ID[i]" "Value[i]" "TS1[i]" "TS2[i] > "file2.txt"; ...
1
vote
3answers
73 views

How to do following formatting using sed or awk or bash script on linux

Suppose, I have two files: file1 having below contents. message_aaa.wav test 1 of 1 test Label : aaa test Sentence : test aaa test String: test message Interpretation ...
1
vote
2answers
51 views

AWK: How to put field ($1) inside regular expression to select all prefixes?

Assume I have a file A: fileA fileB Suppose I have now a file named: fileA_someprefix_20160101.txt Now I want to match all lines from A which prefix this filename, so I thought: ...
3
votes
2answers
56 views

match columns and replace

I am newbie in linux. I have two very large text files with space-delimited fields: File1 527858 51 2 27.92464882 8.63E-07 570289 82 2 30.12532071 2.87E-07 571034 90 2 ...
0
votes
2answers
41 views

using awk to unzip files in directory

I have some zip files in directory when I need to unzip each in specific dir I used ls | awk '{ print "unzip " $1 " -d " $1} ' unzip p21286665_121020_Linux-x86-64.zip -d ...
0
votes
1answer
32 views

Merge files based on matching of first column

I have two files: File1 ARS-BFGL-BAC-10975 0.9303 688423261 1 01/04/2015 0.9983763305 ARS-BFGL-BAC-11025 0.9092 688423261 1 01/04/2015 0.9983763305 ARS-BFGL-BAC-11044 0.9626 688423261 2 01/04/2015 ...
4
votes
3answers
84 views

String replacement using a dictionary

What is a good way to do string replacements in a file using a dictionary with a lot of substituend-substituent pairs? And by a lot, I actually mean about 20 – not much, but many enough so that I want ...
-1
votes
1answer
41 views
1
vote
1answer
47 views

I can't find the process sending mail?

I did something stupid and ran an awk that runs a while loop that sends an email every 10 seconds. I ran this as a background process: egrep '^[1-9]' /etc/hosts | grep -v 'localhost' | awk '{ ...
-1
votes
1answer
25 views

Remove pattern from a line in file with Grep/sed/awk combo, pattern is another column

I have two columns (or could separate as two files) in a huge file and i need to remove the duplicate city name in the state/area column. los angeles los angeles ca, usa, west new york ...
2
votes
6answers
17k views

awk-Printing column value without new line and adding comma

input.txt EN1 EN2 EN3 EN4 EN5 output EN1,EN2,EN3,EN4,EN5 I have tried awk.But it is not printing with comma awk 'BEGIN { OFS = ","} { printf $1}' input.txt I have GNU Awk 4.0.0 version
6
votes
1answer
129 views

Make bash exit != 0 when called by AWK and interrupted with ^C

I'm having an issue with the following piece of (G)AWK script: do { ... } while (system("sleep 10")) My intention is to break the loop when the user presses ^C during the sleep, but it's not ...
1
vote
1answer
29 views

Calculate hits within a timespan from log file [closed]

I need to use UNIX utilities that implement out-of-core divide-and-conquer algorithms so to decrease the amount of virtual memory used (cap of 1 mb). I have a large data set that is an apache log ...
0
votes
3answers
54 views

Awk command is not working properly

I have a very simple problem, but for reasons it does not work properly.. I have these .txt files in the following format 2 250 1 4 250 1 5 250 1 I wanted to subtract 1 from the ...
3
votes
2answers
58 views

Passing a list of two PIDs to xargs only kills the first using ssh

I'm retrieving a list of two PIDs that I want to kill. My pipeline looks something like ps -ef | grep foo | grep -v grep | awk {'print $2'} | xargs kill -9 Both processes are killed when executing ...
10
votes
3answers
13k views

How to sed only that lines that contains given string?

INPUT: Select ASDF 325 sdfg sdflk lk Select TRG 46sdg rasdftz fsgs 45 Select ASDF 6ffg sdfg 4456 sdrg OUTPUT: Select ASDF 325 XXXX sdflk lk Select TRG 46sdg rasdftz fsgs 45 Select ASDF 6ffg XXXX ...
3
votes
2answers
297 views

How do I print the (numerical) ASCII values of each character in a file?

How can I print the numerical ASCII values of each character in a text file. Like cat, but showing the ASCII values only... (hex or decimal is fine). Example output for a file containing the word ...
1
vote
3answers
71 views

Awk, print result

Why does awk '{print 3 + 3}' not work, but: echo 3|awk '{print $1 + 3}' does? Couldn't the first statement just output 6? What is the easiest way of printing the result of a calculation?
0
votes
1answer
12 views

UNIX AWK : Get Logs from 24 Hours

I am trying to extract data from the log files for last 24 hours . I am trying awk -v d="$(date -d'24 hours ago' +'%Y-%m-%d %H:%M')" '$1" "$2>=d &&/ERROR/' data.log This does find ...
2
votes
5answers
2k views

How let `sort | uniq -c` separate the number of occurrences by a tab charactor?

uniq -c separate the occurrence number by spaces, which is hard for to cut or awk to separate it out later. 1000_A1\tB1\n ___1_A2\tB2\n I can solve this problem using sed -r 's/^ *([0-9]+)/\1\t/' ...
3
votes
1answer
70 views

How to optimize 'grep'-and-save?

I want to know on which node my calculations were running on our cluster in case the node crashes and I get no files back to know on which node I have to search. For that case I wrote a little script ...
1
vote
2answers
35 views

Joining columns from files if they contain a match in another column

I have two files: File1.txt 30 40 A T match1 string1 45 65 G R match2 string2 50 78 C Y match3 string3 File2.txt match1 60 add1 50 add2 ...
11
votes
2answers
525 views

awk memory leak?

Base on this I'm running the command < /dev/urandom hexdump -v -e '/1 "%u\n"' | awk '{ split("0,2,4,5,7,9,11,12",a,","); for (i = 0; i < 1; i+= 0.0001) printf("%08X\n", ...
1
vote
6answers
11k views

compare two files get identical list

file1.txt (50 lines) TERYUFV00000010753 TERYUFV00000009526 file2.txt (500 lines) TERYUFV00000009526 refids_739_known_8/10_target TERYUFV00000018907 refids_12023_known_21/22_target ...
1
vote
1answer
199 views

Find and delete lines containing multiple patterns in a file

I have a list of strings and want to find and delete lines containing these strings in a file. A short example of the list of strings is listed as below. File S 1Mo 32,33 2Mo 7,26 2Mo 7,27 2Mo 7,28 ...
1
vote
2answers
34 views

Reorganizing lines in a text file

So I have the following file: First Line Second Line Third Line 3 0.374 2 0.777 1 0.245 Fourth Line 4 0.687 ... And I want to reorganize it so the lines beginning with an int are appended to the ...
1
vote
4answers
61 views

How to use printf with awk to designate the columns separated command output

I want to extract the manufacturer name, card's firmware version and status from the output of fcinfo hba-port, which looks like: root-#> fcinfo hba-port HBA Port WWN: 10000000c96a53c5 OS ...
5
votes
3answers
140 views

How to format floating point number with exactly 2 significant digits in bash?

I want to print the floating point number with exactly two significant digits in bash (maybe using a common tool like awk, bc, dc, perl etc.). Examples: 76543 should be printed as 76000 0.0076543 ...
0
votes
2answers
29 views

placing words corresponding to a key in a singe row

I have a symbol table of the form (key-value pairs) 01 aaluu desii 01 aaluu cipsa 01 amaruuda ilaahabaadii 02 hai 02 nahii 02 txamaatxara To start with, I am supposed to place ...
0
votes
1answer
32 views

Subset rows with awk [duplicate]

I am new to Linux, but my work requires me to do some specific task now, I hope you please help me for this. I have a data set as follow: Cow_ID Age DIM 1 1424 0 1 1425 1 1 1426 2 1 ...
0
votes
4answers
43 views

Change field separator in table

I have several tables having the following format 1971-01-01 1124 1971-01-02 987 1971-01-03 954 ... Instead I would like to change the field separator in order to have the following format of table ...
0
votes
1answer
57 views

Parsing text file using awk

I have a continuously changing text file that contains messages from the Gmail API. I need to parse the contents of it. My text file contains the id of the message, the message, sender details and the ...
1
vote
1answer
41 views

Multiline(in one paragraph) pattern replace use awk or sed

$cat test.txt blablabla ```{r} blabla ``` blablabla blablabla blablabla ```{r} blablabla blablabla imname <- "FIGNAME1" saveflag <- savepdf(imname) ``` ```{r} blablabla blablabla imname ...
0
votes
1answer
54 views

Replace certain field from file with user input for each row and save in same file

Im typing to write a script which can read file_name, delimiter and field_number. After that for each row take an input, replace the field (specified by field_number) and output to same file. For ...
1
vote
3answers
44 views

replace a string 6 line after matching string

I want to change notification_interval from 15 to 2 for block File_Check only. I tried changing line 6th after matching File_Check line with: sed -e '6 /File_Check/ s/15/2/g' file.txt but it did ...
2
votes
1answer
38 views

awk matching only 1 lines with same pattern

The following awk script will add an entry SAMPLE to block NOTES. I have multiple blocks with the same name. What needs to be changed in this command to add the entry SAMPLE to first block NOTES only? ...
1
vote
2answers
120 views

Append line to a block of text incrementing the value from previous line

I have a text file name class.txt with follow contents. [serverClass:MAIL] whitelist.0=LATE whitelist.1=ONTIME [serverClass:LETTER] whitelist.0=FIRST whitelist.1=SECOND whitelist.2=THIRD ...
0
votes
0answers
50 views

Replace column in one file with column from another using awk? where column from another may have null values

I have two files: f1: 111|aaa|444 222|bbb|555 333|ccc|666 f2: 111 333 replacing column 2 of f1 with column 1 of f2 required output new: 111|111|444 222||555 333|333|666 I am using ...
1
vote
2answers
65 views

Replace column in one file with column from another using awk? [duplicate]

I have two files: file1.txt: 111|aaa|444 222|bbb|555 333|ccc|666 file2.txt: 111 (null) 333 replacing column two of file1 with column one of file2 Expected Output new: 111|111|444 222||555 ...
1
vote
1answer
45 views

multi line awk regular expression [duplicate]

Consider text.txt with this content: blablabla &&&key word&&& I want all these text and numbers 123 and chars !"£$%& as output &&&key word&&& ...
0
votes
1answer
53 views

how to use concatenate command inside the awk command

I am trying to print the multiple columns from the file with different sub commands with the awk command. Below is my command which throws error while using concatenate command with awk awk -F\| ...
1
vote
2answers
44 views

Can I access an array created in one awk script in a subsequent awk script?

I have a shell script where there are 2 awk strings. first_awk=' BEGIN { } { # create some array here.. } END{ } ' second_awk=' BEGIN { } { ## Access the array created in first awk section. } ...
0
votes
2answers
47 views

Awk run time error: not enough arguments passed

Here is the script: TYPE="${BLOCK_INSTANCE:-mem}" awk -v type=$TYPE ' /^MemTotal:/ { mem_total=$2 } /^MemFree:/ { mem_free=$2 } /^Buffers:/ { mem_free+=$2 } /^Cached:/ { mem_free+=$2 ...
3
votes
4answers
156 views

Iterating through subdirectories and get two files as arguments for awk script

I need to iterate through the subdirectories of a directory and take two of the files, as arguments of an awk script. The script will compare the two files and generate other files. I've this. But I ...
0
votes
1answer
30 views

How to replace word in specific column if it does not match an expression?

I have the following file: chr1 157784 157887 U6 0 - chr1 564813 564881 miRNA 0 + chr1 564879 564950 tRNA 0 - chr1 564952 565019 tRNA 0 + chr1 566062 566129 ...
11
votes
4answers
24k views

How to extract logs between two time stamps

I want to extract all logs between two timestamps. Some lines may not have the timestamp, but I want those lines also. In short, I want every line that falls under two time stamps. My log structure ...
3
votes
1answer
262 views

How to print incremental count of occurrences of unique values in column 1

I'm trying to come up with an solution to this problem, I need to incrementally count and then print the counts of the unique values in column 1 of a tab delimited text file. Here is an example: ...