awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
3answers
49 views

Compare two files column values using awk

Here is the content of two files : Judi # cat File1 judi /export/home 76 judi /usr 83 judi # cat File2 judi /export/home 79 judi /usr 82 If column 3 of File2 is greater that column 3 ...
1
vote
1answer
41 views

Unable to run awk program

I have created a simple awk program file in /media/ravan/data/program The code: #! /usr/bin/awk -f BEGIN { print "Don’t Panic bro!" } While executing in terminal: ...
0
votes
1answer
92 views

print column value based on column name match in awk/sed [duplicate]

In my shell script after executing a few commands, I'm getting the output below: Name English Maths science Social studies ------ -------- -------- -------- --------------- James ...
3
votes
2answers
66 views

Remove headers and contents from a flat file if they are below a specific line count

I've a flat file containing about 10 million lines: query ID1 content1 content2 query ID2 content3 content4 ... content21 query ID3 content22 ...
0
votes
1answer
43 views

Split files using awk and generate the results in another directory

I am using a shell script where I split a file according to the values of a certain column (using awk). My script is in a folder X, though, and I need the resulting files to be created in a folder Y. ...
1
vote
1answer
30 views

display users that logged in since a date using last

I am trying to display the user name of the accounts that have logged in during the past month using the last command. At the moment I have this last | awk '{print $1, $4 ,$5 ,$6}' | grep -B 10 Jul ...
1
vote
2answers
89 views

Why is a2p (Awk to Perl translator) installed by default? Why would I want to convert Awk to Perl? [closed]

I am surprised to see the a2p utility installed by default in my Linux distribution. a2p is a command line utility that converts an Awk program from standard input to a perl program that it outputs ...
1
vote
1answer
61 views

Compare 2 Columns in 2 different files

I need to subtract second and third columns between 2 files based on the first column and do not show rows with diff_column_2(DIFF-IO)=diff_column_3(DIFF-SELECT)=0. Noting that the order can be random ...
1
vote
2answers
57 views

Reordering a file, based on a second file

I have a file that is out of order.I would like to reorder it by using a second file. File 1 1 ...
1
vote
2answers
123 views

What's the difference between print and printf in awk?

In awk what is the difference between print and printf? Is there any restrictions to use only print instead of printf and vice-versa
2
votes
3answers
126 views

How can I delete everything between two markers in a file?

I have a text in a text file, where I want everything that is between strings like \{{[} and {]}\} to be deleted - including these strings themselves. These two string can lie on different lines as ...
2
votes
1answer
30 views

need help with awk merging rows by sequence from multiple files into one file

have alot of files and need to merge strings between themself by groups of three files. each file ~100-500 rows and result file ~100-200mb seems like awk or sed can help me with paste or without it or ...
2
votes
4answers
560 views

Compare awk vs. grep

I have run below two commands for a very large files grep -E 'string1|string2' 151103*.log|grep 'string3' | grep string4 awk '/string1|string2/ && /string3/ && /string4/' 151103*.log ...
2
votes
1answer
47 views

I want to update a INI file using awk

I have an ini file which looks like this [backup] [persistence] log_backup_timeout_s = 900 log_mode = normal I want update this file to [backup] data_backup_parameter_file = ...
0
votes
1answer
23 views

using awk to list pids

I am trying to write a script that will list all of the processes running with a pid between 500 & 20000. This is what I have so far. ps awux |awk '{for (i=1; i<NF; i++) {if (i>=800 ...
1
vote
1answer
46 views

Merge records depending on common columns and specify the values which are uncommon in the last column as comma separated

I need to merge records depending on first three common columns and place the values which are uncommon in the last column as comma separated . I have a requirement to do the following- InputFile : ...
1
vote
1answer
57 views

sed, getting an error:sed: unterminated {

I'm using below sed command sed -n '\|<tag>|{:n; \|</tag>|!{N;bn}; y|\n| |;p }' test.xml OUTPUT: <tag> <name>jolly</name> <class>2nd</class> ...
-1
votes
2answers
49 views

First match if we are on the first line of input, if so print the whole line, else print the first word followed by a comma

for eg: CREATE TABLE MWWDATA.ACK997 ( AKTYPE CHAR(2) DEFAULT '' NOT NULL , AKNUM CHAR(9) DEFAULT '' NOT NULL ); CREATE TABLE MWWDATA.APREIDEXC ( EMPLID NUMBER(15, 0) DEFAULT NULL ); ...
1
vote
1answer
43 views

Matching content in two files and creating a new file [closed]

I have a script to read each line of a file called saitem.unl and store 3 variables in LINENO, ORDREF and PROD. I then use these fields to read prod.unl and if matched set GROUP to field $3. I want to ...
0
votes
3answers
32 views

i want to print the line which starts with a paricular word and in the rest records print only 1st field

for eg: CREATE TABLE MWWDATA."VTCat02" ( "ID" NUMBER(10) DEFAULT NULL , "Cat" VARCHAR2(255) DEFAULT NULL , "Style_Code" VARCHAR2(255) DEFAULT NULL , "Vendor_Style_#" VARCHAR2(255) ...
1
vote
1answer
47 views

Comapring columns in 2 files and printing the values that differ

I have a file (file-1) that looks like this, DIP-10097N|refseq:NP_416170|uniprotkb:P30015 DIP-10117N|refseq:NP_414973|uniprotkb:P08177 DIP-10168N|refseq:NP_418766|uniprotkb:P15005 ...
0
votes
1answer
100 views

Shell script to process service restart procedure

We're running a 3rd party TTY service on our servers, to support some thin clients manufactured by the 3rd party. From time to time, connection to a thin client can get locked up. Procedure is to ...
1
vote
2answers
44 views

gensub on multiple lines

I have a file which has many random lines like aaa bbb ccc ddd eee mark: 98 fff ggg ggg jjjj iii jjj kkkk I want to use awk AND only gensub to match the number "98" above. So far I have this code ...
0
votes
2answers
47 views

How to make this awk command work

I want to make the command below work but I can't. Please check this out: echo "random string random string mark: abcde random string random string" | awk '{gensub(/^.*mark: ...
1
vote
1answer
64 views

How to grep several files using part of filename as a pattern?

I have a folder which includes several files named order[ID].log where [ID] is a string e.g. orderID1.log orderID2.log orderID3.log orderID4.log ... I need to write a script in Linux to grep each ...
0
votes
2answers
62 views

watch not displaying output of script

I have a simple script which just uses awk to extract some information from output of ps. When I run this on the terminal, it works fine. But, when I run this script with watch: watch bench_run.sh ...
0
votes
1answer
45 views

AWK Help ending standalone script

I started to do this.. to swap fields in a csv file but I don't get to finish the script. Any help? What I want to do is to concatenate fields from 2 to 7 and group them by the first field. Fields 1, ...
6
votes
2answers
116 views

How to split a file according to a column (including header) and rename the generated files?

I have a .txt that can be exemplified like this: NAME | CODE name1 | 001 name2 | 001 name3 | 002 name4 | 003 name5 | 003 name6 | 003 I need to write a script to split this file according to the ...
0
votes
1answer
63 views

How to convert awk one-liner to standalone script? [closed]

How can i rewrite this awk one-liner as a standalone script? I want to improve readability to help me understand how it works. awk 'BEGIN{FS=OFS=";"} NR==FNR{for(i=2;i<=7;i++) if($i!="\"\"") ...
1
vote
0answers
24 views

exchange a column of many many csv files using awk [duplicate]

There is a similar post, but this one is slightly different. I have a csv file in which the second column contains some alphanumeric data. I need to delete this entire column and replace it with a new ...
2
votes
1answer
24 views

Compare two files and matched line send to new file

I have two files file1.txt a|1|cd a|2|cd a|4|cd file2.txt a|0001|hj|df a|0002|ed|nb a|0003|vf|za a|0004|er|ns a|0005|oi|lk I need a create new file with the lines matched by de second column of ...
1
vote
1answer
26 views

How can I access a column of a line from within a string?

I have a file I'm traversing in a for loop. I want to do something like... for line in file.out; do echo "Some text before $line($2) $line($3)"; done Where, for every line I insert the 2nd and 3rd ...
1
vote
4answers
120 views

Changing column of multiple csv files

I have a csv file called scenario1.csv in which the second column has column names like "0-4 years high risk", "65+ years first responder" etc. There are 20 such values. The 21st row, 2nd column has ...
2
votes
3answers
83 views

Replace a column and preserve spacing

This is a follow up to unix: replace one entire column in one file with a single value from another file I am trying to replace one column of a file (file1) with one specific value from another file ...
3
votes
3answers
216 views

Awk script to extract a portion of input file

How do I write a awk script that scans the input file for "start" and finds the line containing "next" and displays the following line? Something like this: [user]$ cat test.txt start ...
0
votes
2answers
25 views

Search for a non exact match and return column 1 of that file

I have a tab seperated file which looks like this: Adam Group1,Group2,Group8,Group9,Group10 James Group1,Group11,Group12,Group7 Kelly Group18 Alfred Group1,Group8,Group10 Manuel ...
1
vote
1answer
83 views

How to merge multiple files based on a timestamp

I have 2 files File 1: 01:12:00,001 Some text 01:14:00,003 Some text 02:12:01,394 Some text File 2: 01:12:00,001 Some text 01:12:01,029 Some text 01:13:21,123 Some text I need output as ...
0
votes
1answer
46 views

Quick shell (bash+awk) script to extract a column using the output of another command

Have to repeat below 100+ times. So first command takes paramater $1 and get me string A001C7F28AB20. Then second command is using that string. # Command1 ->snmpget -v1 -c elephantblue $1 ...
0
votes
2answers
43 views

Combine rows from different files, if pattern is matched

Input: two tab-delimited files, File1 and File2. File2 ($2) has to be parsed by patterns found in File1 ($1). Expected output: tab-delimited file, File3. File3 has to contain the same rows as File2, ...
2
votes
3answers
240 views

unix: replace one entire column in one file with a single value from another file

I want to replace a column in one file with a single value from another file using unix. File 1 is a pdb file, structured like this: HETATM 14 H4B FAD B 600 95.544 50.240 71.308 1.00 ...
2
votes
5answers
492 views

Replace characters in user part of email address

I have a list that looks like this: user-name@domain.example user.name@domain.example user_name@domain.example ... I would like to replace any occurence of a dot in the username part. So that: ...
2
votes
3answers
97 views

How to handle ^M in csv files by using sed & awk?

I have a file with delimiter ~, and was not able to load into database: Input file: 24~1~2~2~unsubscribed~wap~7~2015-10-23 20:03:00~2015-10-24 20:04:00~2015-10-25 02:53:26~Testing duplicate msisdn ...
2
votes
2answers
30 views

bash: how to add assigned value to file

I have bash script looping through several files in a directory. The file identifier has been saved as a variable $num. What I would like to do is add $num as another column in my data like in the ...
0
votes
1answer
46 views

Remove trailing spaces from first field only

I have a multi-lined, tab-delimited file like this: 99999bbbbbt999999bbt99t99t99t99 (where b=spaces, t=tabs) Trailing spaces in the 2nd field are ok, but the trailing spaces in the 1st field need ...
0
votes
2answers
41 views

Merge two lines of a file and adding number in the end of lines

I have a large text file abc.txt with the following format: BALT -6.110 130.600 4.06874414 -0.03254425 BALT -6.620 154.460 3.92286595 -0.17842244 BARM 32.740 140.680 3.94326190 -0.15802649 I need ...
3
votes
3answers
94 views

awk when both delimiter and quotes are used for a field

I have a file in the following format: field1|field2|field3 field1|"field2|field2"|field3 Notice the second row contains double quotes. The string within the double quotes belongs to field 2. How ...
0
votes
1answer
53 views

Single read filter csv based on 1st column values

I have a gigantic csv file that is around a GB in size. The csv's 1st column contains around 300 unique integer values. I want to process the csv once and save the output to different files based on ...
1
vote
2answers
41 views

Extract keyword from line

I need to extract similar patterns from data below. The pattern starts with '.' or '' then to the keyword I want, then followed by ''. We can assume the format is LSUFE-numbers_ PCRF-numbers_ ...
1
vote
2answers
47 views

Stack data from multiple delimited files into one, with variable columns

I'm new to scripting. Got struck with a file merge issue in Unix. Was looking for some direction and stumbled upon this site. I saw many great posts and replies but couldn't find a solution to my ...
4
votes
1answer
66 views

How to compare float values in a shell script?

I need to alert if my load average is more than 10. I could do it like this if my awk command returned an integer: declare -i MAX_LOAD=10 declare -i L1=`cat /proc/loadavg | awk {'print $1'}` if [ ...