awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
2answers
40 views

Select rows based on the values of a specific column (not the first column)

I would like to read a file.csv file and keep the rows (i.e., all the columns) of the file if and only if the sixth column of that file matches values in user_ids_to_keep.csv. The file.csv file I ...
-1
votes
2answers
34 views

awk print all matching lines including duplicates [duplicate]

How can I print all matching lines between two files including the duplicates sourcefile: SNP value RS1 0.55 RS2 0.52 RS3 0.25 RS4 0.22 queryfile: SNP RS1 RS2 RS2 RS3 RS4 RS4 Desired output: ...
0
votes
1answer
18 views

Check if stdout is attached to a terminal from an AWK script

Is there any way I can check if an AWK script's stdout is attached to a terminal, just like test -t 1 in a shell?
0
votes
1answer
38 views

Awk to get same column with maximum filed

I have two files. I have to get matching name from file1, print the entire content of file1, and add a fourth column (exp) in file 1 using a second file with max value file1 name value1 ...
0
votes
2answers
34 views

Remove lines in a file where pattern only occurs once

I have a CSV file. It is ordered by an identifier the first field. The identifier in the first field can appear in either 1 or in 2 lines in the field. I want to remove all lines in the file where ...
1
vote
1answer
17 views

Awk problem: an 'if column conditional' replaces the column entry!

I'm trying to get a grip on awk. Suppose I have an input: this header fred blah 1 blank fred blah 1 blank huh blah 2 blank fred blah 3 blank fred blah 3 blank and after I use the following awk ...
0
votes
2answers
51 views

get day from date with append 0 in 1 to 9 digit

i am using awk command to get creation date of file and from that i m fetching day,month and year but for 1-9 digit i am getting output like 2016041 but i want output 20160401. D=$(ls -ltr $line | ...
2
votes
1answer
51 views

POSIX Awk sort array

I would like to sort an array with POSIX Awk, however I have discovered that: asort() asorti() PROCINFO["sorted_in"] are all GNU extensions. Can this be done with without GNU Awk?
0
votes
1answer
30 views

Ternary operator and print

What is wrong with below command awk 'BEGIN {a = 5;b=6; (a==b)? print "a==b" : print "a!=b"}' Output : awk: BEGIN {a = 5;b=6; (a==b)? print "a==b" : print "a!=b"} awk: ...
0
votes
3answers
16 views

find muli lines in file from other file

i have text file that have multi lines i wanna search for lines containing some text and add # at the start of those lines ex: file1 that i wanna change it acnet 6801/tcp # ...
1
vote
1answer
34 views

Awk field printing within the ternary operator

If you're familiar with DISA STIGs, the test for RHEL-06-000518 is /bin/rpm -Va 2>/dev/null \ | /bin/grep '^.M' What I want to do is pull just the file name out of that test, and make pretty ...
0
votes
1answer
27 views

print only lines that starts with digit from specific column

I have a file which has second column as hostname of all hosts in my network: Host example.com (x.x.x.x) is up (0.0041s latency). And so on, but many lines doesn't start with hostname in second ...
0
votes
1answer
42 views

How to make sed as fast as grep when replacing values

If I search with grep in a 30 gig database for some values it finds all of them in no time, if I run the same command with sed to replace those values it takes ages. Why is grep so much faster than ...
3
votes
1answer
37 views

Combining every two column values in file

I have a file with (with only one line) with multiple columns say 4 (in this case) : A B C D And I want the output as every possible combination of column values taking 2 at a time like: AB AC AD ...
2
votes
1answer
73 views

Why do newer version of awk print large integers instead of decimals?

I often use the awk (gawk) calculator: calc(){ awk "BEGIN{ print $* }" ;} In Ubuntu 14.04, it output big numbers in standard form, e.g. $ calc 56777654409823*4674990588446667952594759939814064128 ...
0
votes
1answer
29 views

Extract two lines following a line with a specific value

I have a text file and I would like to extract the two lines of data following a line containing only "Name" as a value. Name 032 Express intent to engage in diplomatic cooperation (such ...
1
vote
1answer
60 views

Break up large log files

I am trying to break a large log file into smaller files based on date. The file is of the following form, where some lines may not have a date. Those lines should be included with the previous ...
1
vote
2answers
46 views

Define a variable to awk pattern matching from bash

I am trying to grab 100 lines after the text "time: X" with X in {0,40,80,...,200}. Here is what I have so far: #!/bin/bash start=1 end=5 for i in $(seq $start $end);do j=$(($i*40)) awk '/time: ...
2
votes
1answer
44 views

Better way to run a series of seven identical commands in sed? [duplicate]

In the post Sed - Replace first k instances of a word in a file, a user suggests: sed 's/old/new/;s/old/new/;s/old/new/;s/old/new/;s/old/new/' filename as a method of replacing the first k ...
1
vote
1answer
40 views

How to convert a timezone in logs as they are being tailed

I would like to manipulate the format of a datestamp from a log, in realtime, as it is being tailed. The current format is 2016-04-06T23:19:20.878Z. I would like to convert it to our local timezone ...
3
votes
2answers
88 views

Filter files generated by `find` by parsed output of `file` command

I'm writing a quick tool to inspect the contents of a node.js node_modules folder or python virtualenv for native dependencies. As a quick first approximation to this I wrote the following command. ...
1
vote
1answer
37 views

Bash script to summarize “ last -a ”

I would like to make a shell script such that it runs the " last -a " command and summarizes it's data in the form : userID : number of login sessions Host or ip1 - number of ...
-3
votes
1answer
25 views

How do i create a sed script to prompt the user to replace a number in a file? [duplicate]

I'm new in Unix, and I'm learning about sed command. I need to write a very small script for searching a number in a file and replacing it. I tried doing grep, and than deleting the specific phone ...
3
votes
1answer
54 views

A common way of combining two files in Solaris and Linux

I am trying to write a script that would combine two files into one and should work in both Solaris 5.10 and a Linux box. File1: dit710 dit710 dit720 dit720 File2: FacA D0000000000000001 FacA ...
0
votes
2answers
33 views

Compare and return first instance of n characters from a list

Lets say I have the output of this command saved to a file. cat /dev/urandom | tr -dc '[:graph:]' | fold -w 1000 | perl -pe 's/(.)(?=.*?\1)//g' | head -n 50 I would like to compare only the first n ...
1
vote
0answers
16 views

Find and redirect to capture pattern filename

I'm looking a way to do something like: find /path -name "201[2-6]*" | awk -F"/" '{print $5"/"$6"/"$7"/"$8",0,0,0"}' > archives_201\1.csv where archives_201\1.csv could be archives_201[2-6].csv ...
1
vote
1answer
27 views

Split large log file into pieces based on date

I am trying to work out a Linux command to split a large log file into pieces based on date. Using ...
0
votes
1answer
42 views

Column comparison using awk [duplicate]

I would like to compare the first column of file1 with the second column of file2 and only if they match it should display any matching lines in file2 only as the output. The columns are separated by ...
0
votes
3answers
31 views

How can I designate range within an column when making an awk file?

I have this data file which I want to sort out by making an awk file. I want to find the average rate within certain range of ages. For example, from 10<age<20and 20<age<30. And I can't ...
0
votes
1answer
16 views

Separate values based on record

Using awk is it doable to separate values based on a certain record? Let's say we have following file: ACCJSH PREMIUM I5001 7 5976 7 @# 0.2 5195 5231 0.2 @# 39 5197 39 @# If we take the @# for ...
-1
votes
2answers
25 views

Need help in removing XML Tags with empty properties

Input File: <ServiceArea type="STATE" value="DC"> <DaylightSavingsUsed value="true"/> <FemtoSipProxyAddress value="10.13.123.100"/> <TermAuthAAAPSK ...
3
votes
3answers
79 views

Replace multiple lines with a string when the line numbers are stored in a file

I have a file with few million lines, all the same. just for an example: Known Known Known Known Known Known ... I have another file with few thousand line numbers, for an example: 3 5 6 ... I ...
0
votes
1answer
32 views

How can i remove duplicates from the output of tshark output [duplicate]

sudo tshark -i ppp0 'tcp port 80 \ and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)' \ -R'http.request.method == "GET" && http.request.uri contains ...
3
votes
3answers
119 views

Add mtime to grep -c output and sort the output by mtime

I have a directory full of logs named in the following style: info.log00001 info.log00002 info.log00003 ... info.log09999 info.log My current output (using grep -c) I need to analyze the ...
-1
votes
1answer
51 views

Organize a file [closed]

I want to thank the great help and support. Well, I have the following file: name id alello chr pos snp s1 215 a 1 11 a215 s1 216 a 1 11 a216 s1 217 ...
5
votes
2answers
218 views

Case-insensitive search in awk

I need to search for a keyword using awk, but I want to perform a case-insensitive (non case sensitive) search. I think the best approach is to capitalize both the search term ("key word") and the ...
1
vote
2answers
56 views

Tar files starting with the same pattern

I have some log files named : 2016-02-10_03-52.log 2016-02-10_04-43.log 2016-02-10_02-13.log ... 2016-03-15_07-03.log 2016-03-15_09-08.log Basically the pattern is : YYYY-MM-DD_.log I would ...
1
vote
2answers
44 views

Compare 1st column of the 1st file and 2nd column of the 2nd file and get the the output as following [duplicate]

File1: Excel file (.xls) UN ID St M1 M2 SE DOF PV PA FC 17127159 0 -5.9 297.3 765.7 0.22 4 0.003 0.00389231 2.57536 17127163 2 ...
0
votes
1answer
36 views

How to compare (using greater than and less than symbols) strings using awk [duplicate]

This is my input file: PA43171-7,1,3,/vobs/atlas-idc/src/utils/logger/IDCLogger/IDC.cpp,48,12:01:16.086‌​.660, Valid Sensor Reading,Date = 08-19-2014 , Time = 12:01:16 Voltage Sensor=11 , V$ $=11 , ...
2
votes
2answers
36 views

How To Use Awk To Print Between Two Positional Parameters

Is there a way to print everything after a positional parameters using awk? What I would like the following command to print is "p4 p5 p6 p7" echo p1 p2 p3 p4 p5 p6 p7 | awk '{print $4...}' ...
0
votes
2answers
51 views

Help in making multiple lines as single lines

I have below 4 lines: EXTRACT ETEST Last Started 2016-03-20 20:56 Status RUNNING Checkpoint Lag 00:00:02 (updated 00:00:04 ago) EXTRACT PTEST Initialized 2016-03-31 06:25 Status ...
0
votes
1answer
53 views

Compare 1st column of the 1st file and 2nd column of the 2nd file and matching

File1: Excel file (.xls) UN ID St M1 M2 SE DOF PV PA FC 17127159 0 -5.9 297.3 765.7 0.22 4 0.003 0.00389231 2.57536 17127163 2 ...
0
votes
1answer
43 views

How to compare the strings using < (Greater than symbol)

Below is my input file: PA43410-2,1,3,/vobs/atlas-idc/src/utils/logger/IDCLogger/IDC.cpp,48,19:16:46.523.177, 2 PS Sensor Value = -5.501000 , Min = -5.583000 , Max = -5.319000 ...
4
votes
2answers
282 views

awk - combine two filters into one

I have this variable in a bash script on ubuntu 12.04 which is set to show the available MB on the root partition: AV_MB=$(df -m |awk NR==2 |awk '{print $4}') Is there an elegant way to combine ...
3
votes
1answer
121 views

how to use awk to do subtraction with numbers in a large file

I have a large file like this AT5G44030,TAACARA,1000,+,200,206 AT5G44030,TAACARA,1000,+,418,424 AT5G44030,TAACARA,1000,+,773,779 AT5G44030,NGATT,1000,+,114,118 AT5G44030,NGATT,1000,+,267,271 ...
0
votes
1answer
28 views

AWK weird output behavior when adding extra column

I have a file.csv that contains: ID,FN,LN,NUM 576,Pra,Mar,3 688,India,Desa,3 645,Ami,Path,5 245,Me,Yo,4 I want the output to be ID,FN,LN,NUM,R 576,Pra,Mar,3,R 688,India,Desa,3,R 645,Ami,Path,5,R ...
0
votes
2answers
79 views

Curl and Grep to Parsing Content

I need to parse the output of a curl command: curl --user test:test http://192.168.1.1/security/pkm.html | egrep '@company|config.pkm.password' This returns: <input type="text" ...
0
votes
1answer
27 views

get unique values from 1 column and sum up 2 next columns [closed]

I have a text file that looks like this: A 500 100 ABC B 200 100 BBB B 200 100 ACC D 150 235 CDD A 500 100 RER C 210 653 DDD C 210 653 DWR D 150 653 DDW D 150 653 DER What I try to accomplish is to ...
1
vote
2answers
37 views

awk if loop not able to print

I have a variable Test as below Test="Environment Dev" I want to compare col2 char 1 with others and print the whole name of env this is what i did echo $Test|awk '{ if ($2 == "^D*") print "DEV"; ...
1
vote
1answer
67 views

I need to merge multiple .csv files together side by side with unequal number of rows

I have anywhere from 3-5 .csv files and need to be able to merge them together while keeping everything in its respective columns below is an easy example with files with varying number of rows. file1 ...