awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
1answer
47 views

I want to search and replace a pattern

I want to search each record (records are defined by blank lines) in a file for the pattern NAME#AAAA. If it matches, then insert an # in front of the record's age and insert a line age NIL at the end ...
0
votes
2answers
50 views

calculate ratio using awk

awk 'NR==1{$4="ratio"}NR>1{$4 = ($3)/($2)} {print $1 "\t" $2 "\t" $3 "\t" $4}' A B awk: cmd. line:1: (FILENAME=A FNR=18) fatal: division by zero attempted Can anybody help with this error? ...
1
vote
3answers
61 views

Bash script to replace some words present in one file from another file

I have two files, one, file1 is 1000 lines with router configurations in it. A portion of it is like this - ! ipv6 prefix-list CCAV6 seq 5 permit 2217:AB23::/40 route-map DENY-ALL deny 11 ! ip ...
0
votes
0answers
20 views

Printing specific column in bash script and avoiding duplicates [duplicate]

I am writing a Bash script to print a specific column from a file. But that column contains many same strings and I want to just print every content just once. I tried this - awk '!seen{print $9}' ...
2
votes
2answers
122 views

bash source script.sh with exit 0

I use a lot of external files in my scripts. But I have a problem, that if I source a file with exit in the contents of the file, the script exits from loading and the strings that I want to use are ...
1
vote
4answers
234 views

print specific line from multiple files

I have a collection of 5000+ files and I just want to create a output.txt file which contain 27th line of all file along with filename. well I am noob at linux level so what i got from internet is to ...
0
votes
3answers
23 views

How to get second directory name in the perforce file path?

I need to get the directory in the second column after depot. For example given the below file(s) list, I need "Directory2" as the output. //depot/Directory2/Dir3/Dir4/Dir5/Dir6/Filename1 ...
-3
votes
1answer
27 views

AWK interval expressions to clean IP [closed]

Credit to a great many entries from you online and manuals. It works, but, is it efficient enough? Do you approve? {Reading logs +- 5k items fast as I can see} Source data from file "myAwk3.txt": ...
0
votes
4answers
71 views

grep all strings that start with a certain char, and finish with another char

I need to grep all strings that start with "[" and finish with a certain string, e.g. "apal". So all chars in between these 2 chars would be shown as well. Given an input such as: ...
0
votes
1answer
42 views

Sed, Awk Multiline log file, a number {0-999] identify relation

Short version of issue: Need to extract from logs: {Using sed and awk on Linux} Time, PlayerName{Can have many special characters}, GUID, IP from log example at end. Longer version: How to combine 2 ...
2
votes
2answers
29 views

AWK - how to get contents of a file pointed by first column of a file

I have a file f1.txt which has contents like below file1 file2 file3 file4 and file1 has contents like below student1 10 student2 20 Now I want to get the first column from f1.txt and print out ...
-2
votes
2answers
44 views

Can someome explain me the below awk substring command val=$( echo $text $len|awk '{print substr($0,0,$2)}')

command: val=$( echo $text $len|awk '{print substr($0,0,$2)}') The above command works fine. I just want to understand the values taken by $0 and $2. I understand $0 takes the value of $text and $2 ...
4
votes
3answers
59 views

Split file by number of lines including header in each one

I need to split a .txt file into smaller ones containing 100 lines each, including the header. I don't know if this is relevant, but the original file is delimited like this: COLUMN1 | COLUMN2 | ...
3
votes
3answers
49 views

Grep that works with overlapping patterns

I want to count the number of repeating patterns aa in a very big file. So I use the following command: grep -o "aa" ./bwt/dblp.txt | wc -l This works, except when I have a string such as aaa, ...
0
votes
2answers
43 views

How to take the values from two columns in a txt file and match them to values in another

I have two text files formatted as follows: e.g. File1: 900 480 10060.62 2740 -266864.19 3843493.50 2740.00 2740 176.07 900 479 10046.50 2741 ...
3
votes
3answers
55 views

Get lines between two timestamps from a file

I have this large file that I need to go through using date and time. Every line in this file has a date and time in it. I need to search through the lines and get the lines that lie between two ...
0
votes
2answers
56 views

Select rows based on the values of a specific column (not the first column)

I would like to read a file.csv file and keep the rows (i.e., all the columns) of the file if and only if the sixth column of that file matches values in user_ids_to_keep.csv. The file.csv file I ...
-1
votes
2answers
35 views

awk print all matching lines including duplicates [duplicate]

How can I print all matching lines between two files including the duplicates sourcefile: SNP value RS1 0.55 RS2 0.52 RS3 0.25 RS4 0.22 queryfile: SNP RS1 RS2 RS2 RS3 RS4 RS4 Desired output: ...
0
votes
1answer
19 views

Check if stdout is attached to a terminal from an AWK script

Is there any way I can check if an AWK script's stdout is attached to a terminal, just like test -t 1 in a shell?
0
votes
1answer
38 views

Awk to get same column with maximum filed

I have two files. I have to get matching name from file1, print the entire content of file1, and add a fourth column (exp) in file 1 using a second file with max value file1 name value1 ...
0
votes
2answers
38 views

Remove lines in a file where pattern only occurs once

I have a CSV file. It is ordered by an identifier the first field. The identifier in the first field can appear in either 1 or in 2 lines in the field. I want to remove all lines in the file where ...
1
vote
1answer
18 views

Awk problem: an 'if column conditional' replaces the column entry!

I'm trying to get a grip on awk. Suppose I have an input: this header fred blah 1 blank fred blah 1 blank huh blah 2 blank fred blah 3 blank fred blah 3 blank and after I use the following awk ...
0
votes
2answers
53 views

get day from date with append 0 in 1 to 9 digit

i am using awk command to get creation date of file and from that i m fetching day,month and year but for 1-9 digit i am getting output like 2016041 but i want output 20160401. D=$(ls -ltr $line | ...
2
votes
1answer
59 views

POSIX Awk sort array

I would like to sort an array with POSIX Awk, however I have discovered that: asort() asorti() PROCINFO["sorted_in"] are all GNU extensions. Can this be done with without GNU Awk?
0
votes
1answer
48 views

Ternary operator and print

What is wrong with below command awk 'BEGIN {a = 5;b=6; (a==b)? print "a==b" : print "a!=b"}' Output : awk: BEGIN {a = 5;b=6; (a==b)? print "a==b" : print "a!=b"} awk: ...
0
votes
3answers
17 views

find muli lines in file from other file

i have text file that have multi lines i wanna search for lines containing some text and add # at the start of those lines ex: file1 that i wanna change it acnet 6801/tcp # ...
1
vote
1answer
37 views

Awk field printing within the ternary operator

If you're familiar with DISA STIGs, the test for RHEL-06-000518 is /bin/rpm -Va 2>/dev/null \ | /bin/grep '^.M' What I want to do is pull just the file name out of that test, and make pretty ...
0
votes
1answer
29 views

print only lines that starts with digit from specific column

I have a file which has second column as hostname of all hosts in my network: Host example.com (x.x.x.x) is up (0.0041s latency). And so on, but many lines doesn't start with hostname in second ...
0
votes
1answer
44 views

How to make sed as fast as grep when replacing values

If I search with grep in a 30 gig database for some values it finds all of them in no time, if I run the same command with sed to replace those values it takes ages. Why is grep so much faster than ...
3
votes
1answer
41 views

Combining every two column values in file

I have a file with (with only one line) with multiple columns say 4 (in this case) : A B C D And I want the output as every possible combination of column values taking 2 at a time like: AB AC AD ...
2
votes
1answer
77 views

Why do newer version of awk print large integers instead of decimals?

I often use the awk (gawk) calculator: calc(){ awk "BEGIN{ print $* }" ;} In Ubuntu 14.04, it output big numbers in standard form, e.g. $ calc 56777654409823*4674990588446667952594759939814064128 ...
0
votes
1answer
31 views

Extract two lines following a line with a specific value

I have a text file and I would like to extract the two lines of data following a line containing only "Name" as a value. Name 032 Express intent to engage in diplomatic cooperation (such ...
1
vote
1answer
65 views

Break up large log files

I am trying to break a large log file into smaller files based on date. The file is of the following form, where some lines may not have a date. Those lines should be included with the previous ...
1
vote
2answers
50 views

Define a variable to awk pattern matching from bash

I am trying to grab 100 lines after the text "time: X" with X in {0,40,80,...,200}. Here is what I have so far: #!/bin/bash start=1 end=5 for i in $(seq $start $end);do j=$(($i*40)) awk '/time: ...
2
votes
1answer
45 views

Better way to run a series of seven identical commands in sed? [duplicate]

In the post Sed - Replace first k instances of a word in a file, a user suggests: sed 's/old/new/;s/old/new/;s/old/new/;s/old/new/;s/old/new/' filename as a method of replacing the first k ...
2
votes
1answer
45 views

How to convert a timezone in logs as they are being tailed

I would like to manipulate the format of a datestamp from a log, in realtime, as it is being tailed. The current format is 2016-04-06T23:19:20.878Z. I would like to convert it to our local timezone ...
3
votes
2answers
104 views

Filter files generated by `find` by parsed output of `file` command

I'm writing a quick tool to inspect the contents of a node.js node_modules folder or python virtualenv for native dependencies. As a quick first approximation to this I wrote the following command. ...
2
votes
1answer
41 views

Bash script to summarize “ last -a ”

I would like to make a shell script such that it runs the " last -a " command and summarizes it's data in the form : userID : number of login sessions Host or ip1 - number of ...
-3
votes
1answer
35 views

How do i create a sed script to prompt the user to replace a number in a file? [duplicate]

I'm new in Unix, and I'm learning about sed command. I need to write a very small script for searching a number in a file and replacing it. I tried doing grep, and than deleting the specific phone ...
3
votes
1answer
54 views

A common way of combining two files in Solaris and Linux

I am trying to write a script that would combine two files into one and should work in both Solaris 5.10 and a Linux box. File1: dit710 dit710 dit720 dit720 File2: FacA D0000000000000001 FacA ...
0
votes
2answers
44 views

return first instance of characters from a list

Lets say I have the output of this command saved to a file. cat /dev/urandom | tr -dc '[:graph:]' | fold -w 1000 | perl -pe 's/(.)(?=.*?\1)//g' | head -n 50 I would like to compare only the first n ...
1
vote
0answers
17 views

Find and redirect to capture pattern filename

I'm looking a way to do something like: find /path -name "201[2-6]*" | awk -F"/" '{print $5"/"$6"/"$7"/"$8",0,0,0"}' > archives_201\1.csv where archives_201\1.csv could be archives_201[2-6].csv ...
1
vote
1answer
40 views

Split large log file into pieces based on date

I am trying to work out a Linux command to split a large log file into pieces based on date. Using ...
0
votes
1answer
47 views

Column comparison using awk [duplicate]

I would like to compare the first column of file1 with the second column of file2 and only if they match it should display any matching lines in file2 only as the output. The columns are separated by ...
0
votes
3answers
33 views

How can I designate range within an column when making an awk file?

I have this data file which I want to sort out by making an awk file. I want to find the average rate within certain range of ages. For example, from 10<age<20and 20<age<30. And I can't ...
0
votes
1answer
16 views

Separate values based on record

Using awk is it doable to separate values based on a certain record? Let's say we have following file: ACCJSH PREMIUM I5001 7 5976 7 @# 0.2 5195 5231 0.2 @# 39 5197 39 @# If we take the @# for ...
-1
votes
2answers
29 views

Need help in removing XML Tags with empty properties

Input File: <ServiceArea type="STATE" value="DC"> <DaylightSavingsUsed value="true"/> <FemtoSipProxyAddress value="10.13.123.100"/> <TermAuthAAAPSK ...
3
votes
3answers
97 views

Replace multiple lines with a string when the line numbers are stored in a file

I have a file with few million lines, all the same. just for an example: Known Known Known Known Known Known ... I have another file with few thousand line numbers, for an example: 3 5 6 ... I ...
0
votes
1answer
33 views

How can i remove duplicates from the output of tshark output [duplicate]

sudo tshark -i ppp0 'tcp port 80 \ and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)' \ -R'http.request.method == "GET" && http.request.uri contains ...
3
votes
3answers
127 views

Add mtime to grep -c output and sort the output by mtime

I have a directory full of logs named in the following style: info.log00001 info.log00002 info.log00003 ... info.log09999 info.log My current output (using grep -c) I need to analyze the ...