awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
2answers
86 views

Adding a number as a suffix to multiple columns

I have data that looks like: chr1 enhancer_ 875557 876449 . enhancer_ chr1 enhancer_ 876970 877622 . enhancer_ chr1 enhancer_ 995719 996320 . enhancer_ chr1 enhancer_ ...
0
votes
2answers
29 views

can't get an “awk for loop” to work, while using basename

I have many xyz files that I want to run through awk. I just want to create a new file with an "_s" appended, for each one. Having a little trouble getting the basename syntax correct. Here is what I ...
0
votes
3answers
81 views

Count all lines between a delimiter

I want to count the number of addresses attached to each device in the below example. Preferably using awk or sed. /dev/1 addr1 sometext sometext sometext addr2 sometext sometext sometext addr3 ...
5
votes
4answers
298 views

Join two files on two common fields

I have two files file1.txt 78Z|033333157|0000001|PERD1|2150421|D|0507020|3333333311 78Z|033333157|0000001|PERD0|2160208|A|1900460|3333333311 78Z|033333157|0000001|RSAB1|2150421|D|0507070|3333333311 ...
0
votes
1answer
39 views

How can I replace rows containing ONLY number ' 0' with blank rows using AWK?

I am new to AWK, I am looking for a command that would replace the rows containing ONLY number 0 in their columns, with 'blank' lines. e.g. I have below input: Taj_Mahal ...
0
votes
0answers
23 views

Order Columns in CSV file

I am trying to order all columns by column name in a csv file. What I have is something like this: name ,adress ,mobile-number Ane ,USA ,12121212 Joane ,England ,234234 and the output I ...
2
votes
2answers
131 views

Print lines of a file between two matching patterns [duplicate]

How can I print all the lines between two lines starting with one pattern for the first line and ending with another pattern for the last line? Update I guess it was a mistake to mention that this ...
0
votes
2answers
72 views

awk FS with back slashes

I use the command find . -maxdepth 1 -not -type d which generates output like ./filename.1.out I pipe the find command output to awk. The goal is to split on either the literal ./ or .. I have it ...
-2
votes
1answer
81 views

How to find the word before a match

I have the following log file. I need to know if there are any fault / suspect condition using a shell script. I need to find fault and check the previous word, if it is greater than 0 then there is ...
-1
votes
3answers
39 views

Transposing rows into columns in absence of few rows using AWK [duplicate]

I have a text file containing below: title1 A1 title3 A3 title4 A4 title5 A5 title1 B1 title2 B2 title5 B5 title1 C1 title2 C2 title4 C4 title5 C5 title1 D1 title2 D2 title3 D3 I would ...
0
votes
0answers
45 views

Standard deviation of number next to a pattern

I have a file with lots of tags with a number next to them, set out like this: <Score>4 ... other data <Score>2 ... other data <Score>3 and so on... I have used grep to get the ...
2
votes
3answers
83 views

Copy column with file name as column header

I wanted to organize my dataset by copying the a column from a number of text file and append the file name in the first row of the new files. Here is sample of my data file1 a a b b 1 2 ...
2
votes
3answers
6k views

Find common elements in a given column from two files and output the column values from each file

I have two files with tab-seperated values that look like this: file1: A 1 B 3 C 1 D 4 file2: E 1 B 3 C 2 A 9 I would like to find rows between files 1 and 2 where the ...
-1
votes
1answer
20 views

Remove some elements of a string and replace with others

How can I convert this : RS.PK.03.01.4200.03.014.01 to man03140101? What needs to be done? Text before 5th "." has to be removed : RS.PK.03.01.4200. The remaining text should be converted 03....
0
votes
6answers
108 views

Merge multiple lines in same file based on column 1

I'm still learning programming and I've tried many things but just cannot get the correct format. I have a tab delimited file with 17 columns and many (around 50.000) rows. The file is already sorted ...
3
votes
1answer
57 views

Replacing a & in XML correctly?

I want to replace a '&' with &amp; Except the following occurance of XML special characters. " &quot; ' &apos; < &lt; > &gt; & &amp; I tried the ...
1
vote
2answers
87 views

Split file into multiple files based on pattern [duplicate]

I have a large file that's a couple hundred lines long. This file is partitioned into many parts by a specific identifier, lets say 'ABC'. This line 'ABC' appears 6 times so I want 6 output files. I'm ...
0
votes
1answer
22 views

awk: detect content separated by arbitrary character instead of spaces? [duplicate]

Is there a way to make awk detect content separated by commas (or any other character) instead of spaces? e.g. a file (.csv) contains a list of contacts in the format "title","first-name","last-name"...
3
votes
1answer
123 views

Print some lines before and after pattern match excluding lines matching another patterns

I use awk on Solaris to print 5 lines before and after a pattern match. Currently, the following one-liner does the job: /usr/xpg4/bin/awk 'c-->0;$0~s{if(b)for(c=b+1;c>1;c--)print r[(NR-c+1)%b];...
11
votes
5answers
4k views

How do you check if a file exists within awk? [-d 'filename'] failing

I'm trying to generate a list of users who have a home directory set which does not exist. It seems I should be able to do this with awk, but something is wrong with my syntax. It keeps telling me "...
3
votes
2answers
70 views

How can I do a binary search of a text file for a specific character string?

I have a text file that contains a large number of records, each on a single line. Some of the records have special characters that have been corrupted, and I'm attempting to find those by looking for ...
1
vote
3answers
93 views

Parse file with Awk only when first row matches pattern

I need to interrogate the header of a CSV file, and if a column exists proceed with the data rows. Context is when the data contains columns depending on when and what emitted it. Hoping for a "pure" ...
0
votes
0answers
18 views

Loop through directory and sum value next to specific pattern producing an average

I want to loop through all the files in the directory. the files are set out like this: <Overall>4 other data <Overall>2 other data ...... I have the code: for file in .dat; do awk '...
2
votes
4answers
116 views

Sum value next to specific pattern

I have a file with multiple tags with a number next to it e.g. <Overall>4 other <tags> and data <Overall>2 other <tags> and data <Overall>3 How would I search through ...
2
votes
4answers
810 views

How to calculate the sum of the data that have the same ID in the first column?

I want to sum the values of the rows of my data that have the same ID in the first column. My data looks like data.txt Id a b c d e 1 1 2 3 4 5 1 2 3 4 5 ...
1
vote
2answers
240 views

How can I reformat tabular data using AWK?

I have a file contain following output. ora.abc.db 1 ONLINE ONLINE serverA Open 2 ONLINE ONLINE serverB Open ora.xyz.db 1 ...
0
votes
1answer
29 views

How to add a condition within a for loop during substitution if string is null

I am trying to add a condition in this code, that if there is null string in translation file for either string or repl[string] for example, my file input_chk.txt has following details: input_chk.txt ...
1
vote
1answer
29 views

insert string for missing value

I've run into some problems with editing a text file I've received due to some missing values in a specific column for some rows. For example, here is what the file looks like (mostly): ID10 dad ...
2
votes
2answers
81 views

Reformat date to unix time stamp in csv table

I have a .csv file containing the date and time in the format 01/20/2016 23:53:01 in the first column. My columns are separated by semicolons, i.e. 01/21/2016 03:03:01;18616;0;1 01/21/2016 03:13:01;...
3
votes
4answers
77 views

display lines with not more than 4 words in the second column

I have a symbol table of the form M07UP49A0870I422.wav <s> haraa keelaa <bn> </s> M07UP49A0870I423.wav <s> <horn> jau <babble> </s> ...
2
votes
1answer
55 views

Regex matching on awk in command line

I'm trying to get the filename from unrar command verbose and deleting them all through command line. The unrar command output is similar to this: Extracting BucketSort.cpp ...
1
vote
2answers
76 views

How to replace multiple blank lines with a specific non-blank line?

The sample input is 1 2 3 4 5 6 7 8 9 The expected output is 1 2 3 --- 4 5 --- 6 7 8 --- 9
4
votes
2answers
276 views

split a large file to new file with unique file names

I need to split a file into unique file names. I can do it with sed command eg, sed -n '/scaffold135_/w 135-scaf.txt' input file.txt but it's time consuming so I need a smart way to do it faster. ...
0
votes
3answers
296 views

Filtering content from df output

I have the command below to get the filesystem usage on a server: df -k | awk '{print $5 " " $6}' | grep % When executed, the following are the results: 97% / 0% /proc 0% /etc/mnttab 0% /dev/fd 46%...
0
votes
1answer
47 views

Convert a column of epochs into a time column starting from 0

What I have Hi, imagine a lot of this files, where first column is epoch, and the other(s) column(s) are some data: 1000333,34,1 1001456,56,0 1005356,34,2 What I need I need to transform them ...
8
votes
1answer
165 views

Trap Ctrl-C in awk script

I believe Ctrl-C can be trapped in bash scripts. Is it also possible to trap it inside an Awk script in order to handle that event? For example, for aborting processing, but printing the results of ...
0
votes
2answers
59 views

Collecting network information

I need to collect information about the network usage of each process. Nethogs presents the data I need in real time, I am trying to save the output to a file in order parse it and plot the data. ...
0
votes
3answers
39 views

List out the original IP Address of the host

I have following content in the file. I would like to print the IP address which is Under Name. Server: 17.128.100.12 Address: 17.128.100.12#53 Name: Testserver1 Address: 17.146.118.156 ...
3
votes
1answer
39 views

grep output of cdrecord

I would like to parse the output of cdrecord/wodim using grep. cdrecord -v outputs lines like Track 01: 49 of 75 MB written (fifo 100%) [buf 100%] 10.2x. The problem is that cdrecord uses ...
1
vote
1answer
123 views

How to subtract rows from first row using awk?

My data file looks like: 10 -0.314690785295 20 -0.251967909317 30 -0.215271387106 40 -0.189228416217 The desired output is: 10 -0.0627229 20 -0.0994193 30 -0.1254623 I am really new in bash, so ...
1
vote
4answers
67 views

Print only unique lines from file not the duplicates

I have a sorted words list, line by line file like so: apple apple grapes lime orange orange pear pear peach strawberry strawberry I only want to print out unique lines and drop duplicates: grapes ...
0
votes
2answers
38 views

How to change the first column of data in a file?

I have .data file containing a table of numbers with the column format: m x y z vx vy vz separated by tabs, with a depth of 2000. The file name is initc.data. I want to change ...
4
votes
5answers
461 views

Command line method to find repeat-word typos, with line numbers

Updated: Clarify line number requirement, some verbosity reductions From the command line, is there a way to: check a file of English text to find repeat-word typos, along with line numbers where ...
1
vote
1answer
49 views

Syntax error removing duplicate lines via awk '!x[$0]++' file

The file contains cat file a b c b d Trying to remove duplicate lines on SunOS via awk '!x[$0]++' file (as found in another postings) results in a syntax error awk: syntax error near line 1 awk:...
0
votes
2answers
31 views

Joining certain lines of text together before modifying them

I have a cricket commentary like - 0.3 Richardson to RG Sharma, 2 runs, gentle, non-swinging length ball on off, Rohit seeks to pick it up. Hit high on the bat and spooned over the returning Mitch ...
3
votes
3answers
305 views

Select the lines with exactly two columns in Linux

I have a file that is a mapping between filename and the corresponding transcripts. The file name and transcripts is separated by a TAB character. The transcription of the file may contain one or ...
1
vote
1answer
216 views

How do you list number of lines of every file in a directory in human readable format.

I have a list of directories and subdirectories that contain large csv files. There are about 500 million lines in these files, each is a record. I would like to know How many lines are in each ...
0
votes
2answers
50 views

Need help in data arrangement

Could you please help me to sort this out, I have data which i would like get output in below format using shell/perl script. Data: Virtual Machine Test status Running. Assigned Server OVS001....
1
vote
1answer
63 views

how can i comment out/in a section in source code with sed or awk

Currently i have files, which contain sections like this code statement1 code statement2 # BEGIN SOMENAME some code some other code # END SOMENAME code statement n +1 code statement n +1 What i ...
4
votes
6answers
35k views

How can I output the difference between 2 files?

My file consists of the the following; roughly: username:username:username:username:username The above line continues to about 600 characters. I use the awk command in order to use it as an ...