awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

3
votes
1answer
293 views

Walking Multidimensional arrays in Mawk

I'm able to do this properly in gawk, but when I tried to post my code to the machine where it will run, I realized it was using mawk... $ cat multidim.gawk # test of multidimensional arrays // { ...
3
votes
2answers
551 views

Replace a pattern in a file with a long string

I'm writing a little shell script to generate a directory listing. To make the output easily customizable, the script just builds a HTML-table and than should replace a specific token in a template ...
3
votes
4answers
723 views

Align text to center with padding on both sides

Is there an easy way to align text to the center with padding on both sides, with the column width being the longest line from the input? For example, this: aaa bbbb c ddddd ee will turn into this ...
5
votes
3answers
257 views

Deleting Lines if there is a dot in the 5th Column

I have a file and I want to remove all the line if it has a dot in the 5th Column. Input File: sc2/80 20 . A C 80 PASS N=2 F=5;U=4 sc2/60 ...
2
votes
1answer
123 views

How to extract single value from single line tuples?

I have a dump file of a database in the following form ('value1','value2',value3','value4'),('value1','value2',value3','value4'),... I want first to have each tuple in a single line and next i ...
1
vote
2answers
630 views

Printing fields using awk

< pool.sam awk ' /./ {printf $1} {printf $7+1,"\t"} {printf $3,"\t"} { if($2 !=16) {print "\t", "+";} else {print "\t","-";} {printf $4,"\t" ,length($10)+$4, "\t", "1"} }' I am ...
3
votes
1answer
275 views

Efficient way to print lines from a massive file using awk, sed, or something else? [duplicate]

If I had a plain-text file containing 8 million lines and I want to print out lines 4,000,010 to 4,000,000 to the screen, which would be more efficient: awk or sed? There is no pattern to the text, ...
1
vote
2answers
84 views

AWK special character

If i want to awk the below pattern in a file Mobilenumber '91_987986787688899' was successful I used the below code awk '/Mobilenumber '91_987986787688899' was successful/ {print $0}' Log Filename ...
0
votes
1answer
790 views

shell script to print rows if there is a value in column 2

1 **2** 3 4 3 1 4 2 1 4 3 2 4 **2** 1 3 How can I print all rows, on which there is a 2 in the second column.
1
vote
2answers
174 views

how to calculate the average value of the column(big number of columns) that have the row ID?

I want to calculate the average value of my data when they have the same ID in the first column. I want to calculate the average values for the remaining columns that have the same value in the first ...
3
votes
3answers
498 views

Count the number of rows in each column of a text file

I have a text file with the following format: 5 3 1 2 3 4 ..... ..... i.e. space separated 3 columns of numbers/ However, some of the rows may be like: 2 3 1 So, I want to detect if the text file has ...
1
vote
1answer
191 views

How to use awk or sed to convert csv diffs into more readable format

Can anyone give me the example of how to use awk or sed (not sure which one, as I haven't use either of these much, as I have mostly been using grep and cut to work with csv data) to convert ...
2
votes
1answer
69 views

How to only output a column that has constant neighbors?

INPUT AA XXX Y1Y ZZZ GG dhz rr (AAAa) XXX Y2Y ZZZ TT GGGG UU OUTPUT Y1Y Y2Y The input lines could vary.. only the XXX before the Y1Y and the ZZZ after Y1Y is constant (they are the neighbors of ...
1
vote
2answers
375 views

Way to write new filename on wildcard input?

I have some experience with Inix terminals from science internships I've been part of, mainly using a few utilities like grep, awk, and sed but there is one thing I've been trying to figure out for a ...
12
votes
8answers
1k views

How to perform a multi line grep

How would you perform a grep for text that appears on two lines? For example: pbsnodes is a command I use that returns the utilization of a linux cluster root$ pbsnodes node1 state = free ...
5
votes
4answers
164 views

How to add each output of awk

How would you perform addition on the output of every line from awk? For example cat foo | awk '{print $1} 42 42 42 I'm looking for the answer to 42+42+42 (126). I'm trying to write a for loop ...
1
vote
2answers
482 views

awk,how to change the value of the file using parameter sets?

I am trying to reset my data using awk. I have a data of 54 columns where I want to change six of the column values using the following script. #!/bin/bash -x # optimal parameters set ...
1
vote
1answer
124 views

Retrieving two matches from repeated pattern

How can I retrieve matches from two repeated patterns: mappedname & its value in the text below?. MapName:abc someDiffName=someDiffValue x=3 y=6 NeedyMappedName=NeedyMappedValue . . Flag=true ...
1
vote
3answers
174 views

How to add Multiple Searches in AWK command

Im trying to search for the files Created in June,July and August months. I used this method ls -lrth|awk '/[Jun][Jul][Aug]/ {print}' but it ain't working. Please help me correct the syntax.
3
votes
2answers
167 views

Join: Two files - but only append last two columns

Given the files: 1.txt 1, abc, 123, 456, 789 2, lmn, 123, 456, 789 3, pqr, 123, 456, 789 2.txt 1, abc, 123, 000, 000 3, lmn, 123, 000, 000 9, opq, 123, 000, 000 OUTPUT.txt ID, NAME, X, ...
3
votes
3answers
197 views

Join every other column with sed or awk

I have a large text file (666000 colums) in the format A B C D E F Desired output AB CD EF How can we do it in sed or awk. I have tried a couple of things but nothing seems to be working. Please ...
6
votes
5answers
230 views

Get filesystem name and usage with `df -h`

df -h | grep test | sed -e 's/*%.^ //g;s/.*[ ^I]//' Output: /test /test/drv0 /test/drv1 df -h | grep test | cut -d '%' -f1 | sed -e 's/*%.^ //g;s/.*[ ^I]//' Output: 10 20 15 I want to ...
2
votes
1answer
84 views

Regular Expressions within a string in AWK using if/then control structure?

PROBLEM I want to specifically use AWK (GAWK, MAWK, whatever) in an AWK script to sum up the values in field 9 for every record that contains "runner__.jpx" (the underscores are placeholders for two ...
2
votes
3answers
1k views

Comparing files line by line in awk with delimiter

file1: abc|123|check def|456|map ijk|789|globe lmn|101112|equator file2: abc|123|check def|456|map ijk|789|equator lmn|101112|globe EXPECTED OUTPUT: ijk|789|equator lmn|101112|globe Current ...
2
votes
4answers
150 views

AWK Search massive file and write to variable name

I want to pull data that I will need to reformat from a simulation file I am working with to prepare it as input for running other simulations. The target data is an optimized chemical z-matrix. The ...
1
vote
2answers
75 views

How can I reformat tabular data using AWK?

I have a file contain following output. ora.abc.db 1 ONLINE ONLINE serverA Open 2 ONLINE ONLINE serverB Open ora.xyz.db 1 ...
1
vote
1answer
681 views

How to find out the content of a XML file using Unix Sed/Awk? [duplicate]

I have a XML file(MyXML.xml) like this : <?xml version="1.0" encoding="UTF-8"?> <S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"> <S:Body> ...
2
votes
4answers
91 views

Alternative regex for {}

I am trying to replace all of this line, but the numbers: looktype="123" so only the numbers show up. Is this possible in any easy way? {sub ("look type=\"[0-9]{0,3}", "TEST")} I am trying this ...
0
votes
1answer
128 views

how to assign values of the column for given conditional statements

I need help with a script to assign values of the column with a given conditional argument. To be specific, here is my data and my condition input data.file: 0.4 3.6 -2.4 0.1 -2.0 1.6 2.2 3.6 ...
6
votes
4answers
639 views

uniq a csv file ignoring a column, awk maybe?

Given this file (annotations are not part of file, but form part of explanation)... x,a,001,b,c,d,y x,a,002,b,c,e,yy x,bb,003,b,d,e,y x,c,004,b,d,e,y x,c,005,b,d,e,y # nb - dupe of row 4 ...
1
vote
1answer
91 views

doubt regarding use of split in awk

arr=($(echo "great#man" | awk '{split($0,a,"#"); } END {print a[0]; print a[1];}')) echo " ${arr[0]} " echo " ${arr[1]}" My expected output is: great man But I am getting great ...
4
votes
3answers
978 views

Summing up an array inside of awk?

I have the following piece of code: sum1= sum2= declare -a a echo $temp | awk '{split($0,a,","); name=a[1] ; for(i=2;i<=4;i++) sum1+=a[i] ; for(i=5;i<=7;i++) sum2+=a[i] }' This code is ...
10
votes
2answers
761 views

Script to quickly test all keyboard keys

I need to check some notebooks for bad keyboard keys, and so I'd like to speed that up as much as possible. I found nothing for this specific task, so my idea is a script that reads the pressed keys ...
2
votes
3answers
259 views

How does this awk align the spacing?

Can someone please tell me what does this do? cat infile| awk ' { for(i=3;i<=NF;i++){ $2=$2" "$i } printf "%-4s %s\n", $1,$2 ; }' > outfile.txt infile contains: 1 John Smith 2 ...
4
votes
4answers
370 views

Joining two files with unique identifier

I have two files with approximately 12900 and 4400 entries respectively, that I want to join. The files contain location information for all landbased weather observing stations around the globe. The ...
1
vote
4answers
2k views

How to extract line from the file on specific condition

File: chromosome position ref alt chr1 1398 A T chr1 2980 A C chr2 3323 C T,A chr2 3749 T G chr3 5251 C T,G chr3 ...
2
votes
2answers
648 views

How to remove an apostrophe ( ' ) from couple of columns of a .CSV file?

I have a .CSV file with 7 fields, and the 3rd and 4th columns of the file has a number starting with an apostrophe ( ' ). Please see the example below. col0,col1,col2,col3,col4,col5,col6, ...
1
vote
2answers
4k views

How to select, copy and paste some columns from different files and create one file from them?

I want to create and organize data in a file from a number of files by selecting parts of the columns of the given files. I have more than 10 file to copy the second, third and forth columns of each ...
2
votes
3answers
117 views

How to run awk for some number of lines.?

I want to extract some lines with awk. Is it possible to do the following task: ls -l | awk 'BEGIN FOR(i=122;i<=129;i++) FNR==i' How can I display the details from line numbers 122 to 129?
1
vote
3answers
2k views

How to convert from one date format to other format?

I have one problem with my date format. I want to change from one format to the other and vice versa. My date formats are Format 1 YYYY-MM-DD Format 2 MM/DD/YYYY I want to change format 1 to ...
4
votes
6answers
98 views

Only show the lines that are in all textfiles at least once

a.txt cat a.txt a b x c b.txt cat b.txt d e a f the q: SOMEMAGICK *.txt a Q: how can I show only the lines that are in all *.txt files?
1
vote
3answers
167 views

how to calculate the average value of the column that have the row ID?

I want to calculate the average value of my data when they have the same ID (in my specific case, the same day but different time in a day). This is meant to calculate the daily average value of my ...
1
vote
1answer
60 views

how to clean free row/column spaces after the row of your data

I have a problem in cleaning my file where there are unnecessary free spaces below my last data row. This is created when I am copying and pasting my data from excel to text. I want my text file to ...
1
vote
2answers
70 views

how to select part of the row and pasting to form other file?

I want you help me how to organize my data by cutting in the following way. I have the input data as below. input.file: 1 2 1 0.6007 0.1 0.3 0.2 0.7 0.7 0 2 0.3073 0.1 0.1 0.2 0.4 0.7 ...
0
votes
4answers
2k views

Match multiple regular expressions from a single file using awk

I'm trying to parse a HTML file using shell scripting. There are 4 different regular expressions that i need to catch: name= , age= , classs= , marks=. Using grep "name=\|age=\|class=\|marks=" ...
2
votes
3answers
157 views

how to create a data file with a column of date from day1 to dayN?

I was wondering if I could generate a hourly or daily time step date in a column starting from some year in the past to some day in the near past or today. to be more clear, I want to create one ...
10
votes
5answers
1k views

Using grep or awk

To capture a particular pattern, awk and grep can be used. Why should we use one over the other. How does this work in the background? Which is faster and why? E.g If I had a log file and I wanted ...
2
votes
3answers
362 views

sed - find string and append

Suppose you have the following text file 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 ...
1
vote
4answers
2k views

Finding out the common lines in two files using 4 fields using awk and UNIX

I have 2 files. If field 1, 2, 4 and 5 of both file1 and file2 matches, then I want to print the whole line of file1 and file2 one after another in my output file. File1: sc2/80 20 . ...
3
votes
3answers
3k views

importing data from a text file to a bash script

I need to import a text file like this below: AMBI CBNK CCXI CERE CLACW CNYD DAEG DLIA DLLR DNDN DSGX HAST HIBB, HPOL IRDMZ MARK NEPT NFEC NINE NRCIB OMER PLCC PLPM PSUN UNTK There are 25 entries, ...