awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
1answer
52 views

How to make sed as fast as grep when replacing values

If I search with grep in a 30 gig database for some values it finds all of them in no time, if I run the same command with sed to replace those values it takes ages. Why is grep so much faster than ...
3
votes
1answer
57 views

Combining every two column values in file

I have a file with (with only one line) with multiple columns say 4 (in this case) : A B C D And I want the output as every possible combination of column values taking 2 at a time like: AB AC AD ...
2
votes
1answer
85 views

Why do newer version of awk print large integers instead of decimals?

I often use the awk (gawk) calculator: calc(){ awk "BEGIN{ print $* }" ;} In Ubuntu 14.04, it output big numbers in standard form, e.g. $ calc 56777654409823*4674990588446667952594759939814064128 ...
0
votes
1answer
34 views

Extract two lines following a line with a specific value

I have a text file and I would like to extract the two lines of data following a line containing only "Name" as a value. Name 032 Express intent to engage in diplomatic cooperation (such ...
1
vote
1answer
69 views

Break up large log files

I am trying to break a large log file into smaller files based on date. The file is of the following form, where some lines may not have a date. Those lines should be included with the previous ...
1
vote
2answers
62 views

Define a variable to awk pattern matching from bash

I am trying to grab 100 lines after the text "time: X" with X in {0,40,80,...,200}. Here is what I have so far: #!/bin/bash start=1 end=5 for i in $(seq $start $end);do j=$(($i*40)) awk '/time: ...
2
votes
1answer
47 views

Better way to run a series of seven identical commands in sed? [duplicate]

In the post Sed - Replace first k instances of a word in a file, a user suggests: sed 's/old/new/;s/old/new/;s/old/new/;s/old/new/;s/old/new/' filename as a method of replacing the first k ...
2
votes
1answer
59 views

How to convert a timezone in logs as they are being tailed

I would like to manipulate the format of a datestamp from a log, in realtime, as it is being tailed. The current format is 2016-04-06T23:19:20.878Z. I would like to convert it to our local timezone ...
3
votes
2answers
151 views

Filter files generated by `find` by parsed output of `file` command

I'm writing a quick tool to inspect the contents of a node.js node_modules folder or python virtualenv for native dependencies. As a quick first approximation to this I wrote the following command. ...
2
votes
1answer
42 views

Bash script to summarize “ last -a ”

I would like to make a shell script such that it runs the " last -a " command and summarizes it's data in the form : userID : number of login sessions Host or ip1 - number of ...
-3
votes
1answer
51 views

How do i create a sed script to prompt the user to replace a number in a file? [duplicate]

I'm new in Unix, and I'm learning about sed command. I need to write a very small script for searching a number in a file and replacing it. I tried doing grep, and than deleting the specific phone ...
3
votes
1answer
64 views

A common way of combining two files in Solaris and Linux

I am trying to write a script that would combine two files into one and should work in both Solaris 5.10 and a Linux box. File1: dit710 dit710 dit720 dit720 File2: FacA D0000000000000001 FacA ...
0
votes
2answers
45 views

return first instance of characters from a list

Lets say I have the output of this command saved to a file. cat /dev/urandom | tr -dc '[:graph:]' | fold -w 1000 | perl -pe 's/(.)(?=.*?\1)//g' | head -n 50 I would like to compare only the first n ...
1
vote
0answers
17 views

Find and redirect to capture pattern filename

I'm looking a way to do something like: find /path -name "201[2-6]*" | awk -F"/" '{print $5"/"$6"/"$7"/"$8",0,0,0"}' > archives_201\1.csv where archives_201\1.csv could be archives_201[2-6].csv ...
1
vote
1answer
87 views

Split large log file into pieces based on date

I am trying to work out a Linux command to split a large log file into pieces based on date. Using http://stackoverflow.com/questions/11713978/how-to-split-existing-apache-logfile-by-month/11714105#...
0
votes
1answer
48 views

Column comparison using awk [duplicate]

I would like to compare the first column of file1 with the second column of file2 and only if they match it should display any matching lines in file2 only as the output. The columns are separated by |...
0
votes
3answers
34 views

How can I designate range within an column when making an awk file?

I have this data file which I want to sort out by making an awk file. I want to find the average rate within certain range of ages. For example, from 10<age<20and 20<age<30. And I can't ...
0
votes
1answer
17 views

Separate values based on record

Using awk is it doable to separate values based on a certain record? Let's say we have following file: ACCJSH PREMIUM I5001 7 5976 7 @# 0.2 5195 5231 0.2 @# 39 5197 39 @# If we take the @# for ...
-1
votes
2answers
43 views

Need help in removing XML Tags with empty properties

Input File: <ServiceArea type="STATE" value="DC"> <DaylightSavingsUsed value="true"/> <FemtoSipProxyAddress value="10.13.123.100"/> <TermAuthAAAPSK value="secret1"/&...
4
votes
3answers
123 views

Replace multiple lines with a string when the line numbers are stored in a file

I have a file with few million lines, all the same. just for an example: Known Known Known Known Known Known ... I have another file with few thousand line numbers, for an example: 3 5 6 ... I ...
0
votes
1answer
39 views

How can i remove duplicates from the output of tshark output [duplicate]

sudo tshark -i ppp0 'tcp port 80 \ and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)' \ -R'http.request.method == "GET" && http.request.uri contains "/ABC/...
3
votes
3answers
149 views

Add mtime to grep -c output and sort the output by mtime

I have a directory full of logs named in the following style: info.log00001 info.log00002 info.log00003 ... info.log09999 info.log My current output (using grep -c) I need to analyze the ...
-1
votes
1answer
54 views

Organize a file [closed]

I want to thank the great help and support. Well, I have the following file: name id alello chr pos snp s1 215 a 1 11 a215 s1 216 a 1 11 a216 s1 217 ...
5
votes
2answers
320 views

Case-insensitive search in awk

I need to search for a keyword using awk, but I want to perform a case-insensitive (non case sensitive) search. I think the best approach is to capitalize both the search term ("key word") and the ...
1
vote
2answers
70 views

Tar files starting with the same pattern

I have some log files named : 2016-02-10_03-52.log 2016-02-10_04-43.log 2016-02-10_02-13.log ... 2016-03-15_07-03.log 2016-03-15_09-08.log Basically the pattern is : YYYY-MM-DD_.log I would ...
1
vote
2answers
45 views

Compare 1st column of the 1st file and 2nd column of the 2nd file and get the the output as following [duplicate]

File1: Excel file (.xls) UN ID St M1 M2 SE DOF PV PA FC 17127159 0 -5.9 297.3 765.7 0.22 4 0.003 0.00389231 2.57536 17127163 2 ...
0
votes
1answer
47 views

How to compare (using greater than and less than symbols) strings using awk [duplicate]

This is my input file: PA43171-7,1,3,/vobs/atlas-idc/src/utils/logger/IDCLogger/IDC.cpp,48,12:01:16.086‌​.660, Valid Sensor Reading,Date = 08-19-2014 , Time = 12:01:16 Voltage Sensor=11 , V$ $=11 , ...
2
votes
2answers
58 views

How To Use Awk To Print Between Two Positional Parameters

Is there a way to print everything after a positional parameters using awk? What I would like the following command to print is "p4 p5 p6 p7" echo p1 p2 p3 p4 p5 p6 p7 | awk '{print $4...}' ...
0
votes
2answers
51 views

Help in making multiple lines as single lines

I have below 4 lines: EXTRACT ETEST Last Started 2016-03-20 20:56 Status RUNNING Checkpoint Lag 00:00:02 (updated 00:00:04 ago) EXTRACT PTEST Initialized 2016-03-31 06:25 Status ...
0
votes
1answer
61 views

Compare 1st column of the 1st file and 2nd column of the 2nd file and matching

File1: Excel file (.xls) UN ID St M1 M2 SE DOF PV PA FC 17127159 0 -5.9 297.3 765.7 0.22 4 0.003 0.00389231 2.57536 17127163 2 ...
0
votes
1answer
53 views

How to compare the strings using < (Greater than symbol)

Below is my input file: PA43410-2,1,3,/vobs/atlas-idc/src/utils/logger/IDCLogger/IDC.cpp,48,19:16:46.523.177, 2 PS Sensor Value = -5.501000 , Min = -5.583000 , Max = -5.319000 PA43410-2,1,3,/vobs/...
4
votes
2answers
304 views

awk - combine two filters into one

I have this variable in a bash script on ubuntu 12.04 which is set to show the available MB on the root partition: AV_MB=$(df -m |awk NR==2 |awk '{print $4}') Is there an elegant way to combine ...
3
votes
1answer
148 views

how to use awk to do subtraction with numbers in a large file

I have a large file like this AT5G44030,TAACARA,1000,+,200,206 AT5G44030,TAACARA,1000,+,418,424 AT5G44030,TAACARA,1000,+,773,779 AT5G44030,NGATT,1000,+,114,118 AT5G44030,NGATT,1000,+,267,271 ............
0
votes
1answer
29 views

AWK weird output behavior when adding extra column

I have a file.csv that contains: ID,FN,LN,NUM 576,Pra,Mar,3 688,India,Desa,3 645,Ami,Path,5 245,Me,Yo,4 I want the output to be ID,FN,LN,NUM,R 576,Pra,Mar,3,R 688,India,Desa,3,R 645,Ami,Path,5,R ...
0
votes
2answers
111 views

Curl and Grep to Parsing Content

I need to parse the output of a curl command: curl --user test:test http://192.168.1.1/security/pkm.html | egrep '@company|config.pkm.password' This returns: <input type="text" id="config.pkm....
0
votes
1answer
33 views

get unique values from 1 column and sum up 2 next columns [closed]

I have a text file that looks like this: A 500 100 ABC B 200 100 BBB B 200 100 ACC D 150 235 CDD A 500 100 RER C 210 653 DDD C 210 653 DWR D 150 653 DDW D 150 653 DER What I try to accomplish is to ...
1
vote
2answers
37 views

awk if loop not able to print

I have a variable Test as below Test="Environment Dev" I want to compare col2 char 1 with others and print the whole name of env this is what i did echo $Test|awk '{ if ($2 == "^D*") print "DEV"; ...
1
vote
1answer
154 views

I need to merge multiple .csv files together side by side with unequal number of rows

I have anywhere from 3-5 .csv files and need to be able to merge them together while keeping everything in its respective columns below is an easy example with files with varying number of rows. file1 ...
0
votes
1answer
51 views

Awk and Grep to get data from CLI

I want to get output as below: Number Name Value ************************************* 889900 Staff 256000 990088 Staff 1048576 And my input would be as below: ...
0
votes
3answers
32 views

Perform regex search on a previously regex-matched line (one-liner)

I need to print a string which is in a unique line in a long text file. Example relevant line in file long.txt: Software version 4.0.10, 2002-03-28 The wanted string is exactly 4.0.10. The steps ...
0
votes
1answer
74 views

bash script to replace server short names in a file with fqdn

I have a rather large config file that has server short names in it like this: [host 1] host = server1 [host 2] host = server2 [host 3] host = server3 I need to replace all the server short names ...
1
vote
2answers
53 views

Count times an expression is true in awk.

In unix/awk code Essentially, I need to iterate through the file (data1.txt) and count the number of times the substr($0,29,2) of each line of the file == "04". data1.txt: 6597243042 ...
0
votes
1answer
34 views

Curly bracket confusion! [closed]

Curly brackets question, with this file content. one one one two two two three 098234 one one one two two two three 098234 one one one two two two three 098234 ... awk '{ a[NR]=$0 } END {...
1
vote
5answers
196 views

How to print nth line after match / empty lines

I'm trying to write a short script that finds empty lines and then prints the nth line after the empty lines. For foo1 foo2 foo3 bar1 bar2 bar3 spam1 spam2 spam3 eggs1 eggs2 eggs3 printing ...
1
vote
1answer
41 views

Only output value if x > y in awk

INPUT: 0:root@server:/root # vmstat 60 2 System configuration: lcpu=52 mem=131072MB ent=10.00 kthr memory page faults cpu ----- ----------- -----...
0
votes
1answer
50 views

Why does print work with awk but echo doesn't?

To sum it up.... This works: awk 'BEGIN { FS=":"; } { print $1 $3 $5; }' /etc/passwd But this doesn't: awk 'BEGIN { FS=":"; } { echo $1 $3 $5; }' /etc/passwd Being fairly new to learning ...
3
votes
1answer
42 views

comparing values in separate columns

I've got a csv that looks like this, (there are other columns but these are the pertinent ones) region, city, actual rating, predicted rating NSW, SYD, 0.765, 0.5 Victoria, Melbourne, 4.81225, 5.0 ...
0
votes
1answer
41 views

extract the data from 2 files

I have two files, the first file contain only 1 field, the second file has more lines than first file and also it has different field in each line. File 1: 5 15 20 File 2: 3 5 O 1.25 2.30 3.75 ...
0
votes
2answers
63 views

How to get only certain fields from a given file?

I have list of employees in following format Name: Phone No: Email: Name: Address: Phone No: Email: Name: Country: Address: Phone No: Email: Name: Address: Name : Email: Address: All I want ...
1
vote
4answers
116 views

Bash: How to print additional column matching IP Address to Name

I'm using netstat to collect information about connected IP's. And I have the following format: netstat -tn 2>/dev/null | awk '/:80 / {print $5}' | sed 's/.*::ffff://' | sed 's/:.*//' | sor t | ...