awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

2
votes
3answers
43 views

how to extract a number from the line in unix

I need to find the first occurrence of the attribute ICount and its value must be returned. Only the number from the attribute. How to get it? < FCount = "1" ICount = "0" Ccount= "1"> Please ...
0
votes
1answer
68 views

How to write a script that will use a loop structure to process data file?

I have an input file with the following data: November 400 January 200 June 400 March 200 April 200 May 300 July 400 August 300 September 400 February 300 October 300 December 200 I have to make a ...
4
votes
1answer
91 views

Script that Outputs the Average of Multiple Group lines, from a File

This is a "Small" portion of the file car_sales.txt that I am trying to extract information from (the full file has about 700 lines, with more vehicle brands than are listed here): ...
2
votes
3answers
136 views

Arithmetic on variable containing ip address

I've got variable network="198.168.1.x" # for ex. 198.168.1.254 I want to create new variables: next_serv="198.168.1.(x-1)" # for ex. 198.168.1.253 max_host="198.168.1.(x-2)" # for ex. ...
1
vote
1answer
48 views

Need To grab IP of visitors from the Apache logs , How could I do it?

I want to store the IP of the visitors to my website, I want to use PHP , Should I use cron for it or how will be the visitors IP will be automatically stored ??
4
votes
2answers
422 views

How avoid to repeat the same command in bash script

I have this bash script: gunzip -c /var/log/cisco/cisco.log-$(date +%Y%m%d).gz | awk '/ath_bstuck_tasklet/ { print $4 }' | sort | uniq -c > /tmp/netgear_beacon.txt echo "There are `wc -l ...
9
votes
5answers
915 views

Merge two files line by line with the delimiter triple pipe symbol “|||”

I have two parallel files with the same number of lines in two languages and plan to merge these two files line by line with the delimiter |||. E.g., the two files are as follows: File A: 1Mo 1,1 I ...
1
vote
5answers
50 views

awk: Can I find the occurrence of char in given string?

Is there any way to get following result? a c a c a c a a a a a a a a c c c a c 0 0 a c 0 a max a 2nd max c
2
votes
3answers
124 views

Duplicate indexes value in array

Given the files below: file1: 7997,1 7997,2 7997,3 5114,1 5114,2 file2: 7997,52, 5114,12, 4221,52, How can I create an array from the 1st file that has the first column as indices and the ...
12
votes
7answers
11k views

How to insert the content of a file into another file before a pattern (marker)?

File1 contents: line1-file1 "1" line2-file1 "2" line3-file1 "3" line4-file1 "4" File2 contents: line1-file2 "25" line2-file2 "24" Pointer-file2 "23" ...
1
vote
1answer
41 views

how to set a variable to multiple values with awk?

I have a file similar to the following : x 0 1 x 1 1 and I am essentially trying to see if both fields are equal to one another or not for every row. But the problem is that this file contains x ...
-1
votes
1answer
36 views

Merge matching lines as one line

I am stuck in awk command to find and merge duplicate lines as one here is how my file looks (I do not have header) and it is sorted on column 2 already: 1, abc, 123, , , , , , , , , , 2, xyz, 123, ...
0
votes
1answer
31 views

Filter log on final letter in string [closed]

I have a below requirement to filter a word. My logs will be reporting with codes TML2345I,TML8383E & TML9929W. I want to filter the errors only with strings which have the last character as E. ...
6
votes
4answers
146 views

Substitute Column With Result of Command

I'm trying to find a way to quickly and efficiently replace a column value for every row in a file with the output of a command. I need to work on multiple files with about 500,000 rows each daily so ...
5
votes
3answers
62 views

Combine information from two files, line by line

I have 2 files as input: File 1 (space separated) ID POS a0 a1 SNP1 123 A C SNP2 456 T C SNP3 789 G A File 2 (space separated) 0 1 0 1 0 0 0 1 1 1 0 0 1 0 0 1 0 1 1 1 0 0 0 1 Required output A ...
-1
votes
1answer
32 views

How to remove duplicate rows using awk while splitting file

I'm using awk in a shell script so I can split a file according to the 5th column. The code below is similar to the one I'm using: awk -F';' 'NR==1{h=$0; next} !seen[$5]++{f="FILE_"$5".txt";print h ...
3
votes
1answer
97 views

Extract specific columns from text file

There is a text file, with aboot 10000+ lines, in the format: field1 field2 field3 field4 field5 ... fieldX It needs to be converted to look like: field1,field2,field4,field3,field5 to fieldX The ...
4
votes
3answers
138 views

Duplicating a string variable in awk

I would like to precede each line with a number saying how many slashes the line has got. awk '{ l=$0; gsub("[^/]","",l); print length(l),l }' This doesn't work bacause l=$0 seems to assign by ...
13
votes
7answers
21k views

Remove comma between the quotes only in a comma delimited file

I have a input file delimited with commas (,). There are some fields enclosed in double quotes that are having a comma in them. Here is the sample row 123,"ABC, DEV 23",345,534.202,NAME I need to ...
3
votes
1answer
30 views

awk not doing a new line

I'm trying to print out the users who are currently logged in (each logged in user of each terminal). This is what I currently have: NOWDATE=$(date +"%Y-%m-%d") NOWTIME=$(date +"%T") USERS=$(who | ...
1
vote
1answer
62 views

expand data based on scores

Firstly I have a set of valid combinations of 3 columns (validlist), so the final result must be a subset of this file. a b c c b c p b d d y d p y d x y z I have a score ...
1
vote
1answer
38 views

Match data from array to data from file?

I'm trying to match up the data from an array the lists the days of the week like this: MONDAY TUESDAY WEDNESDAY THURSDAY FRIDAY SATURDAY SUNDAY The files data reads like this: Name1,Tuesday ...
3
votes
1answer
70 views

sed: cut line X and append to top of the file

I would like to be able to cut line X and insert the line X on position y (top). This would help me to be able to reorganize some config files. Text example line number 1 <--- Append line 10 ...
2
votes
4answers
45 views

Remove partial duplicates consecutive lines but keep first and last

I have log files with a time stamp and six values in each line i want to reduce the amount of data, by removing consecutive lines with the same values (ignoring time stamps) and keeping the first and ...
1
vote
0answers
109 views

Find and delete lines containing multiple patterns in a file

I have a list of strings and want to find and delete lines containing these strings in a file. A short example of the list of strings is listed as below. File S 1Mo 32,33 2Mo 7,26 2Mo 7,27 2Mo 7,28 ...
2
votes
2answers
109 views

pipe ls and awk to rsync

Trying to complete a script with the following steps: (1) select 1gig of most recently changed files across a number of subdirectories (2) preferably uses rsync to copy the files to local directory -- ...
5
votes
2answers
2k views

Replace string with sequential index

Can someone suggest an elegant way to accomplish this? Input: test instant () test instant () ... test instant () //total 1000 lines output should be: test instant1 () test ...
2
votes
2answers
48 views

Combine columns using awk? (Or other suggestions)

I'm trying to be a bit effective and need a script or command solution. Say I make a file with 2 columns, or two files with one column, whichever is easier: AA1 B2 ZZ1 YYY XX1 AA2 B2 ...
1
vote
3answers
107 views

Match lines beginning with the same pattern in two text files

I am now having two text files with some of the lines beginning with the same pattern. For example, File A 1Mo 1,1 I love you. 1Mo 1,2 I like you. Hi 23,3 I am hungry. Hi 23,4 I am foolish. File ...
3
votes
4answers
208 views

How to get Disk space(Avail) using 'df -h' when Filesystem is long

I am trying to find available disk space through console I found df -h | awk '$NF == "/" { print $4 }' to get required output. [root@tmpe2etstmsdc01 ~]# df -h Filesystem ...
2
votes
2answers
62 views

Trying to convert a simple C program into an AWK program

Hey guys a couple of weeks ago I wrote a C program that prompts the user for the name of a text file and then prompts the user for a word. The program then outputs the input text file with numbers to ...
3
votes
2answers
45 views

Averaging and moving a selection of files

I have a bunch of directories, each of them enclosing an ascii table named accordingly with the directory name; as follow: a1/a1.txt a2/a2.txt a3/a3.txt a4/a4.txt b1/b1.txt b2/b2.txt b3/b3.txt ...
4
votes
5answers
76 views

Count unique associated values in awk (or perl)

I've already found "How to print incremental count of occurrences of unique values in column 1", which is similar to my question, but the answer isn't sufficient for my purposes. First let me just ...
3
votes
3answers
5k views

How to add Multiple Searches in AWK command

Im trying to search for the files Created in June,July and August months. I used this method ls -lrth|awk '/[Jun][Jul][Aug]/ {print}' but it ain't working. Please help me correct the syntax.
1
vote
2answers
99 views

Using awk and grep for iftop results

I wanted to monitor my server network send and receive rate using iftop. This is what works for me: iftop -t -s 1 -n -N | grep 'Total send and receive rate: ' | awk '{print $8}' Mac address and Ips ...
1
vote
1answer
50 views

How do I keep awk variables in scope?

I am attempting to keep track of the lowest/highest values from a list of counties and populations. I can't figure out how to stop variables from resetting themselves at 0. This is my cmd.awk ...
5
votes
1answer
128 views

Sort output of awk except for first line?

This is a use case I am continually running into in parsing CSV files. When it is an inline awk script embedded in a shell script, I can use this workaround: myfile=$(mktemp) awk '(awk script here)' ...
1
vote
1answer
48 views

Nesting Awk and variable comparison [duplicate]

I found myself having to learn a bit of awk, and it's really cool, but I have a small problem I am facing. I have a script that has an output that is designed to be piped. Let's call it matrix. I ...
1
vote
2answers
114 views

How to call bash function from within awk?

This is a bit tricky; I'm trying to work out the best approach to this problem. I have a couple of approaches, but they seem really hacky and I'd like something a little more elegant. I want to ...
2
votes
3answers
58 views

How can I use (sed|awk) to mask an e-mail address with asterisks like <j******e@g***.com>

How can I mask a list of e-mail addresses like: John Doe <john.doe@gmail.com> John Doe <johndoe@hotmail.net> Jane Doe <j.doe@outlook.org> ... with sed (or awk) into something ...
1
vote
1answer
51 views

awk is not producing the desired output

Below is my command output. When I process it through awk, it is giving unwanted output. What am I doing wrong? # lvdisplay -m --- Logical volume --- LV Name ...
18
votes
8answers
21k views

How to print certain columns by name?

I have the following file: id name age 1 ed 50 2 joe 70 I want to print just the id and age columns. Right now I just use awk: cat file.tsv | awk '{ print $1, $3 }' However, this ...
3
votes
1answer
44 views

Exclude similar values from awk

Kindly consider below files: File 1: boo,194,2322 foo,999,7559 File 2: boo,2322 boo,4526 foo,4222 foo,4223 I need to link Field1 in File 1 with Field1 in File 2 and get the related Field2 ...
1
vote
0answers
27 views

Selecting the columns from source file based on control file [duplicate]

I have requirement where I need to construct a file based on control file. The first row of my source file contains column names. The control file contains the columns which I need to output into a ...
2
votes
1answer
52 views

How to enter a sum derived from awk in the middle of text output [closed]

I am trying to print a sum in the middle of a line. My script prompts for a user name and then pulls a total number of processes and a total of time for that user from a file. I want to print a ...
2
votes
3answers
2k views

Awk: match exact string in line

I have this file a deiauk Biking US 200 G b kespaul 202 A c deiauk NY 222 5 Z And I want to match the exact string 200 using awk. So my result should be a deiauk Biking US 200 G Here's my code ...
2
votes
3answers
476 views

Using awk to sum the values of a column, based on the values of another column

I am trying to sum certain numbers in a column using awk. I would like to sum just column 3 of the "smiths" to get a total of 212. I can sum the whole column using awk but not just the "smiths". I ...
4
votes
3answers
156 views

Concatenate several files with a common header

I have several text files, and each file's content is like attribute1,attribute2,... 1,2,... The first lines which describe the names of the columns are the same in all the files. I want to ...
3
votes
4answers
88 views

Split columns based on patterns in fields

Example: I'd like to split $2 in input file, putting a tab after pattern surname Input: name surname1 name surname30000 name surname456 Desired output: name surname 1 name ...
1
vote
1answer
34 views

Use awk to count number of /bin/false entries in a file in field7 with a single command for total count

I am having trouble using awk to count the number of /bin/false entries in a file in field7 with a single command. I can use grep no problem. I can use awk to count a few different ways. I have ...