awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

4
votes
5answers
96 views

Count unique associated values in awk (or perl)

I've already found "How to print incremental count of occurrences of unique values in column 1", which is similar to my question, but the answer isn't sufficient for my purposes. First let me just ...
1
vote
2answers
148 views

Using awk and grep for iftop results

I wanted to monitor my server network send and receive rate using iftop. This is what works for me: iftop -t -s 1 -n -N | grep 'Total send and receive rate: ' | awk '{print $8}' Mac address and Ips ...
1
vote
1answer
71 views

How do I keep awk variables in scope?

I am attempting to keep track of the lowest/highest values from a list of counties and populations. I can't figure out how to stop variables from resetting themselves at 0. This is my cmd.awk ...
5
votes
1answer
221 views

Sort output of awk except for first line?

This is a use case I am continually running into in parsing CSV files. When it is an inline awk script embedded in a shell script, I can use this workaround: myfile=$(mktemp) awk '(awk script here)' ...
1
vote
1answer
85 views

Nesting Awk and variable comparison [duplicate]

I found myself having to learn a bit of awk, and it's really cool, but I have a small problem I am facing. I have a script that has an output that is designed to be piped. Let's call it matrix. I ...
1
vote
1answer
51 views

awk is not producing the desired output

Below is my command output. When I process it through awk, it is giving unwanted output. What am I doing wrong? # lvdisplay -m --- Logical volume --- LV Name ...
2
votes
3answers
83 views

How can I use (sed|awk) to mask an e-mail address with asterisks like <j******e@g***.com>

How can I mask a list of e-mail addresses like: John Doe <john.doe@gmail.com> John Doe <johndoe@hotmail.net> Jane Doe <j.doe@outlook.org> ... with sed (or awk) into something ...
1
vote
3answers
136 views

Match lines beginning with the same pattern in two text files

I am now having two text files with some of the lines beginning with the same pattern. For example, File A 1Mo 1,1 I love you. 1Mo 1,2 I like you. Hi 23,3 I am hungry. Hi 23,4 I am foolish. File ...
1
vote
0answers
28 views

Selecting the columns from source file based on control file [duplicate]

I have requirement where I need to construct a file based on control file. The first row of my source file contains column names. The control file contains the columns which I need to output into a ...
2
votes
1answer
47 views

Exclude similar values from awk

Kindly consider below files: File 1: boo,194,2322 foo,999,7559 File 2: boo,2322 boo,4526 foo,4222 foo,4223 I need to link Field1 in File 1 with Field1 in File 2 and get the related Field2 ...
2
votes
1answer
59 views

How to enter a sum derived from awk in the middle of text output [closed]

I am trying to print a sum in the middle of a line. My script prompts for a user name and then pulls a total number of processes and a total of time for that user from a file. I want to print a ...
3
votes
3answers
5k views

Using awk to sum the values of a column, based on the values of another column

I am trying to sum certain numbers in a column using awk. I would like to sum just column 3 of the "smiths" to get a total of 212. I can sum the whole column using awk but not just the "smiths". I ...
4
votes
3answers
177 views

Concatenate several files with a common header

I have several text files, and each file's content is like attribute1,attribute2,... 1,2,... The first lines which describe the names of the columns are the same in all the files. I want to ...
2
votes
2answers
145 views

pipe ls and awk to rsync

Trying to complete a script with the following steps: (1) select 1gig of most recently changed files across a number of subdirectories (2) preferably uses rsync to copy the files to local directory -- ...
3
votes
4answers
163 views

Split columns based on patterns in fields

Example: I'd like to split $2 in input file, putting a tab after pattern surname Input: name surname1 name surname30000 name surname456 Desired output: name surname 1 name ...
1
vote
1answer
54 views

Use awk to count number of /bin/false entries in a file in field7 with a single command for total count

I am having trouble using awk to count the number of /bin/false entries in a file in field7 with a single command. I can use grep no problem. I can use awk to count a few different ways. I have ...
1
vote
2answers
341 views

How to call bash function from within awk?

This is a bit tricky; I'm trying to work out the best approach to this problem. I have a couple of approaches, but they seem really hacky and I'd like something a little more elegant. I want to ...
1
vote
0answers
34 views

tab-delimiting a file in shell

I have a file that I want to put in a table in SQL with 10 columns. I am trying to tab-delimit the file so that the file can go in the table. Here is part of my txt file: Chr12 MSU_osa1r7 exon ...
3
votes
1answer
45 views

AWK sum column in file specify as a argument

I want to write a script that sums up my column in the file. The numbers of columns he wants to give as an argument. It may be several. I do not know how to solve this with awk so that sums up several ...
-3
votes
2answers
30 views

AWK count how many instance after pattern match

For example: RXOTG-1388 holds 3 object RM4FD1,RM4FD2,RM4FD3 RXOTG-1398 holds 3 object VT08D1 VT08D2,VT08D3 and so on. Based on this text file I would like to count, using awk, how many object ...
-1
votes
2answers
172 views

find and print file path, without filename

Please check the below awk '/SOURCE/ {print $3}' /MAC/Delete_Name.txt | while IFS= read -r file; do find /OS/ -iname "$file.pdf" ; done This results in output like this: ...
2
votes
1answer
83 views

Using sed / awk to change words between two patterns

I'm writting the shell script that should process through some text, I've got the text like that: read build { file1 file2 file3 } check build { file2 file3 file4 } read build { file4 file5 file6 } ...
1
vote
1answer
65 views

expand data based on scores

Firstly I have a set of valid combinations of 3 columns (validlist), so the final result must be a subset of this file. a b c c b c p b d d y d p y d x y z I have a score ...
2
votes
1answer
28 views

Help with averaging specific columns awk

I need to process some data for my dissertation and I need to do it though Linux as I have > 2 million points. I've currently gotten my selected data it in a file that looks something like this using ...
-1
votes
2answers
34 views

putting output into sql table

I am trying to convert the output of ps awux into 11 fields with the headers eliminated and then importing that output into an existing table in SQL.
1
vote
1answer
53 views

Combine range of columns with the same key

Input file tab separated: A |B | C | KEY | D| E| F |G | H | I | J | k | ----|----|----|------|---|---|-----|------|---|---|-----|-----| word|word|word| one | 8| ...
2
votes
2answers
78 views

Replacing matching text after two matching lines

I have a YAML file which includes a stanza looks this: admin::common::passwords: alice: password: '$6$oTQhLvN/4VFJPscD$8LYwUMSFi' bob: password: '$6$JKOtLF0wHeZfIskt$W/M5.ugDS' If the ...
1
vote
3answers
474 views

How to extract XML from a text file

I have a text file as below. 1001=<?xml ns=dkfj><home><taga>dkjf</taga></home>1002=<?xml ns=dkfj><home1><taga>dkjf</taga></home1> I need ...
1
vote
3answers
69 views

Compare two files column values using awk

Here is the content of two files : Judi # cat File1 judi /export/home 76 judi /usr 83 judi # cat File2 judi /export/home 79 judi /usr 82 If column 3 of File2 is greater that column 3 ...
2
votes
1answer
70 views

Unable to run awk program

I have created a simple awk program file in /media/ravan/data/program The code: #! /usr/bin/awk -f BEGIN { print "Don’t Panic bro!" } While executing in terminal: ...
0
votes
1answer
134 views

print column value based on column name match in awk/sed [duplicate]

In my shell script after executing a few commands, I'm getting the output below: Name English Maths science Social studies ------ -------- -------- -------- --------------- James ...
3
votes
2answers
81 views

Remove headers and contents from a flat file if they are below a specific line count

I've a flat file containing about 10 million lines: query ID1 content1 content2 query ID2 content3 content4 ... content21 query ID3 content22 ...
0
votes
1answer
85 views

Split files using awk and generate the results in another directory

I am using a shell script where I split a file according to the values of a certain column (using awk). My script is in a folder X, though, and I need the resulting files to be created in a folder Y. ...
1
vote
1answer
36 views

display users that logged in since a date using last

I am trying to display the user name of the accounts that have logged in during the past month using the last command. At the moment I have this last | awk '{print $1, $4 ,$5 ,$6}' | grep -B 10 Jul ...
1
vote
2answers
130 views

Why is a2p (Awk to Perl translator) installed by default? Why would I want to convert Awk to Perl? [closed]

I am surprised to see the a2p utility installed by default in my Linux distribution. a2p is a command line utility that converts an Awk program from standard input to a perl program that it outputs ...
1
vote
1answer
83 views

Compare 2 Columns in 2 different files

I need to subtract second and third columns between 2 files based on the first column and do not show rows with diff_column_2(DIFF-IO)=diff_column_3(DIFF-SELECT)=0. Noting that the order can be random ...
1
vote
2answers
70 views

Reordering a file, based on a second file

I have a file that is out of order.I would like to reorder it by using a second file. File 1 1 ...
1
vote
2answers
422 views

What's the difference between print and printf in awk?

In awk what is the difference between print and printf? Is there any restrictions to use only print instead of printf and vice-versa
2
votes
3answers
190 views

How can I delete everything between two markers in a file?

I have a text in a text file, where I want everything that is between strings like \{{[} and {]}\} to be deleted - including these strings themselves. These two string can lie on different lines as ...
2
votes
1answer
34 views

need help with awk merging rows by sequence from multiple files into one file

have alot of files and need to merge strings between themself by groups of three files. each file ~100-500 rows and result file ~100-200mb seems like awk or sed can help me with paste or without it or ...
2
votes
4answers
1k views

Compare awk vs. grep

I have run below two commands for a very large files grep -E 'string1|string2' 151103*.log|grep 'string3' | grep string4 awk '/string1|string2/ && /string3/ && /string4/' 151103*.log ...
2
votes
1answer
64 views

I want to update a INI file using awk

I have an ini file which looks like this [backup] [persistence] log_backup_timeout_s = 900 log_mode = normal I want update this file to [backup] data_backup_parameter_file = ...
0
votes
1answer
26 views

using awk to list pids

I am trying to write a script that will list all of the processes running with a pid between 500 & 20000. This is what I have so far. ps awux |awk '{for (i=1; i<NF; i++) {if (i>=800 ...
1
vote
1answer
52 views

Merge records depending on common columns and specify the values which are uncommon in the last column as comma separated

I need to merge records depending on first three common columns and place the values which are uncommon in the last column as comma separated . I have a requirement to do the following- InputFile : ...
1
vote
1answer
62 views

sed, getting an error:sed: unterminated {

I'm using below sed command sed -n '\|<tag>|{:n; \|</tag>|!{N;bn}; y|\n| |;p }' test.xml OUTPUT: <tag> <name>jolly</name> <class>2nd</class> ...
-1
votes
2answers
55 views

First match if we are on the first line of input, if so print the whole line, else print the first word followed by a comma

for eg: CREATE TABLE MWWDATA.ACK997 ( AKTYPE CHAR(2) DEFAULT '' NOT NULL , AKNUM CHAR(9) DEFAULT '' NOT NULL ); CREATE TABLE MWWDATA.APREIDEXC ( EMPLID NUMBER(15, 0) DEFAULT NULL ); ...
1
vote
1answer
45 views

Matching content in two files and creating a new file [closed]

I have a script to read each line of a file called saitem.unl and store 3 variables in LINENO, ORDREF and PROD. I then use these fields to read prod.unl and if matched set GROUP to field $3. I want to ...
0
votes
3answers
36 views

i want to print the line which starts with a paricular word and in the rest records print only 1st field

for eg: CREATE TABLE MWWDATA."VTCat02" ( "ID" NUMBER(10) DEFAULT NULL , "Cat" VARCHAR2(255) DEFAULT NULL , "Style_Code" VARCHAR2(255) DEFAULT NULL , "Vendor_Style_#" VARCHAR2(255) ...
1
vote
1answer
53 views

Comapring columns in 2 files and printing the values that differ

I have a file (file-1) that looks like this, DIP-10097N|refseq:NP_416170|uniprotkb:P30015 DIP-10117N|refseq:NP_414973|uniprotkb:P08177 DIP-10168N|refseq:NP_418766|uniprotkb:P15005 ...
0
votes
1answer
197 views

Shell script to process service restart procedure

We're running a 3rd party TTY service on our servers, to support some thin clients manufactured by the 3rd party. From time to time, connection to a thin client can get locked up. Procedure is to ...