awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

-1
votes
1answer
44 views

How to add a word at the first of every lines? [duplicate]

File1: Data will be inserted on Thursday Messages has been sent at 6pm Office hours is from 9am to 6pm All the employees are present Output what I want: DMS Data will be inserted on ...
1
vote
3answers
163 views

Why do awk split() make first field to be last element in array?

I'm probably missing something really simple here, but when I say echo 'The quick brown fox jumped over the lazy dog.' | \ awk '{ split($0, WORDS, " "); for ( WORD in WORDS ) { ...
2
votes
1answer
40 views

Format sar samples live with paste and awk

I'm combining live sar samples with paste and trying to format the output with awk live. It works as expected to format the output, but it doesn't do the formatting live on each sample and instead ...
2
votes
1answer
71 views

Show nth occurrence of a record

Using awk, how to show the nth occurrence of a record and delete the other occurrence, for example" Input File: 79933,2 79933,4 79933,5 55251,1 49494,4 49494,2 49494,2 49494,1 in this example, ...
2
votes
3answers
501 views

Get last set of non-empty lines

I'm running a cron job which should get only the last result of iostat -d 1 2. This requires some parsing: What is the simplest way to get the last set of non-empty lines from standard input to ...
17
votes
11answers
4k views

There must be a better way to replace single newlines only?

I am in the habit of writing one line per sentence because I typically compile things to LaTex, or am writing in some other format where line breaks get ignored. I use a blank line to indicate the ...
2
votes
3answers
61 views

Extract specific value from blocks of data

I have the blocks of data which are differentiated by the global Ids as below. I need to extract the last block of data with the highest id value, in the below example, id="82". <gc type="global" ...
1
vote
3answers
25 views

Extract specific value from block of data

I have the following block of data and need to extract the percent value in XYZ FREEBYTES, i.e. captured value should be equal to 64. As far as I know, I can grep and cut the data. However, the digits ...
1
vote
2answers
34 views

Append mth and nth columns of a file with the columns of another file

I have 2 files having different number of columns and with huge number of rows: bash-3.00$ cat fileA a1,a2,a3,a4,a5,a6,a7,a8,a9 q,w,,,,y,u,, a,z,,,,q,n,, ......................... z,p,,,,w,e,, ...
1
vote
1answer
55 views

error in awk/sed usage of file manipulation line-by-line [duplicate]

I have a requirement of appending the string " 1" at the end of every line of the file. After the execution of below two commands, an empty file is created. awk '{ print $0 " 1" }' < file.txt > ...
2
votes
4answers
116 views

Multiple files - awk command filter

I use a script say; filter1.sh which contains awk command; bzip2 -dc File_1.tsv.bz2 | awk -F '\t' 'BEGIN {OFS=FS} { if (($7) > 50) print $0 }' > File_1.tsv bzip2 -dc File_2.tsv.bz2 | awk -F ...
0
votes
1answer
32 views

Help with a AWK function

I have problems with AWK command. When I used this code is: $ awk '{a[NR]=$1} END {for (i=0;i<NR;i++) {B=a[i+1];A=a[i];C=(B-A);D=int(C/16)} {for (j=0;j<=D;j++) ...
104
votes
3answers
217k views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
2
votes
2answers
50 views

Awk - keep word “example” and “EXAMPLE” with one input

I am attempting to to only keep customers that contain certain words. Here is an example < newname.txt awk 'BEGIN {FS=","} $7~/USA|United States/' > usa.txt The problem I am having is if they ...
4
votes
6answers
193 views

Extracting lines based on conditions

Each line in a comma-separated file has 5 fields. a,b,c,d,e f,g,c,i, j,k,c,m,n o,p,c,r,s t,u,c,w, x,y,z,aa,bb How can I extract the lines which have c in the 3rd field and their 5th field is NOT ...
3
votes
3answers
213 views

awk comparison using arrays

I have the following file: 6180,6180,0,1,,1,0,1,1,0,0,0,0,0,0,0,0,4326,4326,,0.440000, 6553,6553,0,1,,1,0,1,1,0,0,0,0,1,0,1,0,4326,4326,,9.000000, ...
2
votes
2answers
21 views

How to replace value for a given condition in specific column of file

I have file separetd by pipe | and I want that when column 6 contain letter I print 0758000 in column 7 and when the column 6 contain letter A print 0800000 in column 7, I can not find how to do it!!! ...
1
vote
3answers
130 views

How to pass variables to awk command with conditions?

I have a while loop which reads data from a file with city names: city_name: COIMBATORE MADURAI PONDICHERRY SALEM TRIPUR TIRUCHI TIRUNELVELI TUTUCORIN VELLORE and using an awk command within the ...
3
votes
3answers
1k views

Bash Script To Find Debian Release Number from /etc/debian_version

The following is what I'm currently using for testing the version of Debian. Instead of doing it this way, can I make it more succinct? #!/bin/bash VERSION=NONE if [ $(cat /etc/debian_version) == ...
2
votes
4answers
125 views

Convert multilines to single line

I have multiline log file and I want to convert it to single line log. Multiline example: 6/13/2015 12:00:47 AM - { 562} START Web 6/13/2015 12:00:47 AM - Requested Web connection from ...
-5
votes
0answers
48 views

Error !! missig [: missing `]' please help [duplicate]

I an writing simple script in Bash, to check Linux file system disk usage on root file system and display a warning message if system is greater than 6%. The commands are running in a terminal, ...
0
votes
4answers
169 views

How to extract column name (header) from a CSV file which contains the max value in a row?

I am trying to extract the column name with the maximum value in each row using bash script i.e., the column header value or the value from the same column in the first row. I am using the following ...
0
votes
1answer
50 views

Why the number is rounded over ssh? [duplicate]

localhost:~ $ df -k / | awk '{print $3/1024/1024/1024}' 0 0.00618061 localhost:~ $ ssh localhost "df -k / | awk '{print $3/1024/1024/1024}'" 01 01
2
votes
2answers
43 views

Grouping lines into heterogeneous subsets

I have file with n lines.  (Each line refers to a “question”, and therefore they are labeled Q.1, Q.2, Q.3, ..., Q.n.)  Each line (question) has a “Marks” attribute, which has the value 2, 3, 4, 5, or ...
0
votes
1answer
74 views

How can I find and replace only in a range of columns?

I have a text file with 9267 lines. I want to find "A" with 5 spaces after it and replace it with "A" with 5 spaces before and after it on all lines, but only between columns 67 and 88. Text outside ...
5
votes
4answers
149 views

Remove lines from file up to a pattern, unless the pattern doesn't exist

I am attempting to write a filter using something like sed or awk to do the following: If a given pattern does not exist in the input, copy the entire input to the output If the pattern exists in ...
0
votes
1answer
38 views

How to replace the content of specific column with awk? Tab Delimited File

Here's a similar question but I ran into a problem that was not addressed in the question. How to replace the content of specific column with awk? I want to do something similar, however my file is ...
1
vote
1answer
21 views

AWK: where is my error?

I have a file with :-delimited fields. If the 3rd field of a line is greater than 22, that line should be printed to file44. The file looks like this: 1:2:3:4 1:2:3:411 1:2:33:42 1:2:3:44444 ...
2
votes
3answers
65 views

Read a column from a file and append a specific column of another file

I have two files as such: fileA: Name,Site Name,Product UPEI,Product Name,NMS Flag,Product Model, SW FileB: NE_Name SW ABC 4.4 ASD 4.3 ... ... Note that ...
2
votes
2answers
358 views

Merge some tab-delimited files

I have 100 files with 57,816 rows each.I want to do an outer join of these files by merging on basis of the common column. I am using R programming for this but this is very slow. fileList <- ...
1
vote
1answer
68 views

To use Pandoc pipe data with AWK

Data ID | Size | Blocks -------|-----------|--------- 206 |80760288 | 157744 207 |80760288 | 157744 328 |241520288 | 471728 Wanted output by AWK ID |BlockSize | Size | ...
9
votes
7answers
7k views

How to insert the content of a file into another file before a pattern (marker)?

File1 contents: line1-file1 "1" line2-file1 "2" line3-file1 "3" line4-file1 "4" File2 contents: line1-file2 "25" line2-file2 "24" Pointer-file2 "23" ...
0
votes
2answers
31 views

limit outputs by awk

I am trying to get the results for an exact column, that are greater than 0, and less that 10,000,000. The file I has as following: 799,799,0,0,,0,0,1,0,0,0,0,1,1,1,1,1,5155,5155,,0.000000 ...
7
votes
4answers
323 views

How to insert lines with numbers from X to Y into another file after line Z?

I created two files: echo -e "1\n2\n3\n4\n5" > 123.txt echo -e "a\nb\nc\nd\ne" > abc.txt I want to get the file 123.txt with the following contents: 1 2 3 b c d 4 5 So in other words ...
4
votes
2answers
45 views

Adding a new column using information from subheadings

Replacing lines containing a pattern with sed Thanks to @don_crissti and @Archemar. I reckon my question is not a duplication of the above question. So here is an example of input. Edit: All ...
1
vote
2answers
84 views

How to remove characters of text in one line before specific character

How to remove a characters of text in one line before specific character I will remove characters to the left of the first letter "F" Example: null###F01|54646466|00K4234001|IFD|1990101 ...
4
votes
2answers
228 views

Reorder Multiple Line Blocks with Sed

I am trying to reorder generated output as follows; given a tex fragment as follows, I want to move the /^Constant/ down below /Dummies/, with the line before and after /^Constant/. Further, I would ...
2
votes
2answers
531 views

Calculate median value for a column and say if this value is “OK” or “ERROR” if falls within acceptable range

I would like to calculate the median value of a particular column in a .txt file, output that value and also say if that output value is ok. I can do the initial steps and output the median value, ...
5
votes
3answers
87 views

update line based on previous line

I have a issue since I'm not familiar with awk. I have csv file generated from sar -d output converted to csv style: 12:33:41 unix,restarts 12:35:00,lofi4096,0,0.0,0,0,0.0,0.0 ...
3
votes
4answers
6k views

Extracting column from comma separated text

I have a long comma-separated delimited file with 20K lines. Here's a sample: "","id","number1","number2","number3","number4","number5","number6","number7" ...
11
votes
7answers
22k views

How to read first and last line from cat output?

I have text file. Task - get first and last line from file after $ cat file | grep -E "1|2|3|4" | commandtoprint $ cat file 1 2 3 4 5 Need this without cat output (only 1 and 5). ~$ cat file | ...
0
votes
1answer
49 views

Replace pattern with line from file using awk/sed [duplicate]

I have a file first.html which contains the following code: <tr> <td class="headerValue">One</td> </tr> <tr> <td class="headerValue">Two</td> </tr> ...
2
votes
4answers
58 views

CSV - Only keep certain entries

I have a bunch of data inside "" and separated by , - Example below of two lines "stampthisandthat.com","GANDI ...
1
vote
1answer
58 views

Split column into separate columns

I want to split the column1 at dot . into two columns and rest of the columns remain same. Input file: chr21.9411551 7 13 53.8461538461538 7 29 24.1379310344828 Expected output file: chr21 ...
1
vote
1answer
59 views

How to make awk work in a while loop

input.txt: /usr/dir1/file1 /usr/dir1/file2 /usr/dir2/file4 /usr/dir3/file5 Script: cat input.txt |while read FILENAME do nawk '{print $1}' ${FILENAME} >> output.txt done Expected output: all ...
2
votes
2answers
164 views

How to pass shell variable to an awk search pattern?

In my bash script I have a variable that I am trying to pass to a pattern to search for using awk. However what I expected to happen is not working. I have the following text file (text.txt): ...
0
votes
3answers
42 views

How to print empty spaces in first column using awk or sed

Here is my sample input file. ip.txt: john math science paul math science rosy math jill science rob math science hary math Desired output: john paul ...
1
vote
2answers
53 views

Remove x lines before and y lines after nth line (which matches a pattern) in a file

I'm trying to remove a four line block from a text file if the second line matches the Regex N+. The text file is composed of many repeating 4 line formats, ie: @HEADER1 ACTGCNNNT + 583@#!NMY ...
11
votes
9answers
2k views

How can I delete the 5th word of every line in a file?

I want to delete the 5th word of each line in a file. The current content of the file: File is not updated or and will be removed System will shut down f within 10 seconds Please save your work ...
3
votes
2answers
121 views

comparing the first column of two files and printing the entire row of the second file if the first columns match

I have two files in these formats: file1 : air smell hand dude road cat file 2: air,4,21,01,13,3,2 smell,21,4,2,5,6 dude,1,31,42,1 road,1,4,2,1,4 cat,1,5,6,3,1 hand,1,4,2,1,6 mouse,1,3,5,6,2 ...