awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
2answers
11 views

How to replace value for a given condition in specific column of file

I have file separetd by pipe | and I want that when column 6 contain letter I print 0758000 in column 7 and when the column 6 contain letter A print 0800000 in column 7, I can not find how to do it!!! ...
2
votes
1answer
35 views

Grep on single line

I have a script that streams a pattern in a single line (no linebreaks). I want to grep stock_ticker in that line and output it as soon as I have found one. Now the script is never ending and is ...
1
vote
3answers
69 views

How to pass variables to awk command with conditions?

I have a while loop which reads data from a file with city names: city_name: COIMBATORE MADURAI PONDICHERRY SALEM TRIPUR TIRUCHI TIRUNELVELI TUTUCORIN VELLORE and using an awk command within the ...
-5
votes
0answers
45 views

Error !! missig [: missing `]' please help [duplicate]

I an writing simple script in Bash, to check Linux file system disk usage on root file system and display a warning message if system is greater than 6%. The commands are running in a terminal, ...
2
votes
4answers
108 views

Convert multilines to single line

I have multiline log file and I want to convert it to single line log. Multiline example: 6/13/2015 12:00:47 AM - { 562} START Web 6/13/2015 12:00:47 AM - Requested Web connection from ...
0
votes
0answers
24 views

Multi input files for awk [on hold]

I have 3 files, as below: File1: 5372,5372,0,0,,1,0,1,1,0,1,0,0,0,0,0,0,1107,1107,,1.620000,1,2015-08-12 2110,2110,0,0,,0,0,1,0,0,0,0,1,1,1,1,1,1601,1601,,9.500000,1,2015-05-29 ...
0
votes
1answer
62 views

How can I find and replace only in a range of columns?

I have a text file with 9267 lines. I want to find "A" with 5 spaces after it and replace it with "A" with 5 spaces before and after it on all lines, but only between columns 67 and 88. Text outside ...
2
votes
2answers
40 views

Grouping lines into heterogeneous subsets

I have file with n lines.  (Each line refers to a “question”, and therefore they are labeled Q.1, Q.2, Q.3, ..., Q.n.)  Each line (question) has a “Marks” attribute, which has the value 2, 3, 4, 5, or ...
0
votes
1answer
49 views

Why the number is rounded over ssh? [duplicate]

localhost:~ $ df -k / | awk '{print $3/1024/1024/1024}' 0 0.00618061 localhost:~ $ ssh localhost "df -k / | awk '{print $3/1024/1024/1024}'" 01 01
0
votes
1answer
26 views

How to replace the content of specific column with awk? Tab Delimited File

Here's a similar question but I ran into a problem that was not addressed in the question. How to replace the content of specific column with awk? I want to do something similar, however my file is ...
1
vote
1answer
19 views

AWK: where is my error?

I have a file with :-delimited fields. If the 3rd field of a line is greater than 22, that line should be printed to file44. The file looks like this: 1:2:3:4 1:2:3:411 1:2:33:42 1:2:3:44444 ...
5
votes
4answers
128 views

Remove lines from file up to a pattern, unless the pattern doesn't exist

I am attempting to write a filter using something like sed or awk to do the following: If a given pattern does not exist in the input, copy the entire input to the output If the pattern exists in ...
2
votes
3answers
55 views

Read a column from a file and append a specific column of another file

I have two files as such: fileA: Name,Site Name,Product UPEI,Product Name,NMS Flag,Product Model, SW FileB: NE_Name SW ABC 4.4 ASD 4.3 ... ... Note that ...
0
votes
4answers
83 views

How to extract column name (header) from a CSV file which contains the max value in a row?

I am trying to extract the column name with the maximum value in each row using bash script i.e., the column header value or the value from the same column in the first row. I am using the following ...
0
votes
2answers
24 views

limit outputs by awk

I am trying to get the results for an exact column, that are greater than 0, and less that 10,000,000. The file I has as following: 799,799,0,0,,0,0,1,0,0,0,0,1,1,1,1,1,5155,5155,,0.000000 ...
4
votes
2answers
40 views

Adding a new column using information from subheadings

Replacing lines containing a pattern with sed Thanks to @don_crissti and @Archemar. I reckon my question is not a duplication of the above question. So here is an example of input. Edit: All ...
1
vote
1answer
63 views

To use Pandoc pipe data with AWK

Data ID | Size | Blocks -------|-----------|--------- 206 |80760288 | 157744 207 |80760288 | 157744 328 |241520288 | 471728 Wanted output by AWK ID |BlockSize | Size | ...
1
vote
2answers
70 views

How to remove characters of text in one line before specific character

How to remove a characters of text in one line before specific character I will remove characters to the left of the first letter "F" Example: null###F01|54646466|00K4234001|IFD|1990101 ...
0
votes
1answer
35 views

Replace pattern with line from file using awk/sed [duplicate]

I have a file first.html which contains the following code: <tr> <td class="headerValue">One</td> </tr> <tr> <td class="headerValue">Two</td> </tr> ...
1
vote
1answer
36 views

Split column into separate columns

I want to split the column1 at dot . into two columns and rest of the columns remain same. Input file: chr21.9411551 7 13 53.8461538461538 7 29 24.1379310344828 Expected output file: chr21 ...
1
vote
1answer
54 views

How to make awk work in a while loop

input.txt: /usr/dir1/file1 /usr/dir1/file2 /usr/dir2/file4 /usr/dir3/file5 Script: cat input.txt |while read FILENAME do nawk '{print $1}' ${FILENAME} >> output.txt done Expected output: all ...
2
votes
4answers
56 views

CSV - Only keep certain entries

I have a bunch of data inside "" and separated by , - Example below of two lines "stampthisandthat.com","GANDI ...
5
votes
3answers
133 views

How to print a line if next line does not contain a particular string using nawk or sed

Input.txt: 8B0C remove 8B0D remove 8B0E remove 8B0F 8B10 remove 8B14 remove 8B15 remove 8B16 remove 8B17 remove 8AC0 ...
0
votes
3answers
33 views

How to print empty spaces in first column using awk or sed

Here is my sample input file. ip.txt: john math science paul math science rosy math jill science rob math science hary math Desired output: john paul ...
3
votes
2answers
63 views

comparing the first column of two files and printing the entire row of the second file if the first columns match

I have two files in these formats: file1 : air smell hand dude road cat file 2: air,4,21,01,13,3,2 smell,21,4,2,5,6 dude,1,31,42,1 road,1,4,2,1,4 cat,1,5,6,3,1 hand,1,4,2,1,6 mouse,1,3,5,6,2 ...
1
vote
2answers
45 views

Remove x lines before and y lines after nth line (which matches a pattern) in a file

I'm trying to remove a four line block from a text file if the second line matches the Regex N+. The text file is composed of many repeating 4 line formats, ie: @HEADER1 ACTGCNNNT + 583@#!NMY ...
-1
votes
1answer
23 views

Math calculations using Awk on dynamic rows

I have a sample file with values like below, and I want to calculate weighted average on the parts within [] 6708.1717,07/15/2015,[6708:2],[6708.1609:3],[6708.5:5] 3352.130000000,07/15/2015,[3352:4] ...
2
votes
2answers
133 views

How to pass shell variable to an awk search pattern?

In my bash script I have a variable that I am trying to pass to a pattern to search for using awk. However what I expected to happen is not working. I have the following text file (text.txt): ...
0
votes
1answer
32 views

netstat pipe count host

Unix pipe problem: how to process multiline input to output the line and insert some piped processes. echo "111 22222222" | awk '{print $1 " " $2 " " piped_processes_using_2nd_argument}' A bit of ...
0
votes
2answers
25 views

Replace fields in file inline

I have file (a.txt) with following contents: Redhat master 12345678 V1A 20.0/20.0.1 Windows slave 12345678 V1B 19.0/19.0.1 I want to grep line with word master and replace 4th and fifth field with ...
1
vote
1answer
55 views

How can i use multiple if statement inside another if statement of a awk program?

I have a CSV File and i want to have multiple checks to retrieve the data from the CSV file. I have the following conditions which needs to be taken care of and i am trying to use awk in a shell ...
1
vote
2answers
50 views

`ip addr` in one-line per interface

I would like to print specific information about network configuration for different interfaces over all the servers I manage: the interface name the interface ipv4 address the interface hardware ...
1
vote
1answer
32 views

bash ignorecase match pattern in file and color that line, print all to screen

I have almost zero idea about coding or bash scripting, so please bare with me and excuse my horrible formatting. What I want is to cat a log file and highlight certain lines with color based on a ...
11
votes
9answers
2k views

How can I delete the 5th word of every line in a file?

I want to delete the 5th word of each line in a file. The current content of the file: File is not updated or and will be removed System will shut down f within 10 seconds Please save your work ...
1
vote
1answer
64 views

How to select rows from a CSV file based on different column values?

I have a CSV file with values like the following: col1,col2,col3,col4,col5,col6,col7 1,0,0,0,0,BTS,Active 4,5,3,1,1,LocalMode,Offlne 1,2,5,6,3,PermFault,Offline 1,2,6,6,2,BSC,Active ...
0
votes
3answers
33 views

How to get index values for multiple matches in the same line in awk?

I know the following expression: echo "bob alice robert alice" | awk '{print index($0,"alice")}' 5 Will output the index of the first alice match, is there any way to get the index of all matches?, ...
4
votes
2answers
89 views

Usage of awk to search for string that contains characters

I am facing an issue of searching through lines of text in a text file. Currently, I am using this command check=`awk -F : -v "title=$title" 'tolower($1) ~ tolower(title)' test.txt` It works fine ...
1
vote
3answers
71 views

how to break a line after matching a certain character using shell command?

File1: John Daniel Hommes Cameron;Emily Terry Mussy Barbara Mimi Papu;David Swiss Jen Hans Peter Iril;Kelvin Lilly Gucci Kate Nik;Forum Bill June;Jill and Jack Output file: John Daniel Hommes ...
0
votes
2answers
52 views

How to count the number of words of each sentences in a file using shell command?

I have text files like this: Mr.P.K.Baneerjee has visited the site today at 4.30pm The permit tag has been set in the ds busbar Opearation has been performed in presence of control engineer Opeation ...
0
votes
0answers
21 views

Output a particular character if the same column consists of a few defined characters with AWK

Let say I have a text file as below: G W G A A T G A A T R A T A R A G - G A And I already know that if the same column consist of "A", "G" and "T", I would like to output a "D". For column 2, ...
1
vote
2answers
68 views

How to extract 1st, 3rd, 5th of group of three lines by sed or awk?

Say I have a file: Line1 Line2 Line3 Line4 Line5 Line6 Line7 Line8 Line9 Line10 Line11 Line12 … I want to get: Line1 Line2 Line3 Line7 Line8 Line9 Line13 Line14 Line15 It means that I want ...
0
votes
1answer
31 views

Need to parse nginx access.log by minute, chunks for statistics - is there better way?

I need collect current data from it every minute to monitore amount of requests and errors. It's nginx's frontend log - with a lot of requests. It's formatted, rotated than archived every hour. So - ...
0
votes
1answer
50 views

What does vRS=“\n20” mean on the gawk command line?

Can any one please explain the below command: gawk -vRS="\n20" '/Job-157625/' log.1 We are using this command to get the log for the job. I know gawk is used to find/scan a pattern but I want to ...
1
vote
0answers
64 views

How to search for columns that contain specific values

I have a large tab-delimited text file (~40000 columns x ~20000 rows) that I am trying to search through. Each column in the file has a unique identifier in row 1. I have a separate list of a subset ...
1
vote
2answers
110 views

Grep multiple patterns and print a different number of lines below each of the patterns?

I am trying to find a way to grep/awk multiple patterns and then print a certain number of lines below the first pattern and a certain number of lines below the second pattern. For example .... ...
2
votes
1answer
43 views

How to read and do a min/max/avg from a NMEA stream

I have a weather station attached to my device spitting NMEA 0183 on /tty/USB0 I have alreeady sorted out how to parse the data and echo what I want in a script. What I need to do now is write the ...
0
votes
0answers
37 views

print duplicate fields in different columns from two files using awk

I have two large files. I am attempting to identify and print any duplicate fields to a new file. I have a 19 columns and over 100k rows. I need to compare columns 5-19 for duplicate values. ...
-1
votes
1answer
74 views

how to extract lines between 2 patterns, and save them to different files

hej all, Let' say that we have this file: rrows 35 cols 51 spin 2 %band5 efermi=0.012538 0.004352 0.004317 0.004213 0.004047 0.003828 0.003570 0.003291 0.003011 0.002754 0.002544 0.002408 ...
1
vote
3answers
45 views

compare multiple files(more than two) with two different columns

I have four different files: file1, file2, file3, file4. Each file has 2 different columns separated by tab. I want to match first column of file1 (as reference) with the first column of second file, ...
1
vote
1answer
58 views

Pick columns from a variable length csv file

I have two CSV files, which share a column that is unique to each row in each file, like an ID. The files don't have a header. The file_2 has variable length columns e.g. 1,2,3 4,5,6,7,8 I joined ...