awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
1answer
16 views

Separate values based on record

Using awk is it doable to separate values based on a certain record? Let's say we have following file: ACCJSH PREMIUM I5001 7 5976 7 @# 0.2 5195 5231 0.2 @# 39 5197 39 @# If we take the @# for ...
-1
votes
2answers
25 views

Need help in removing XML Tags with empty properties

Input File: <ServiceArea type="STATE" value="DC"> <DaylightSavingsUsed value="true"/> <FemtoSipProxyAddress value="10.13.123.100"/> <TermAuthAAAPSK ...
3
votes
3answers
119 views

Add mtime to grep -c output and sort the output by mtime

I have a directory full of logs named in the following style: info.log00001 info.log00002 info.log00003 ... info.log09999 info.log My current output (using grep -c) I need to analyze the ...
1
vote
2answers
56 views

Tar files starting with the same pattern

I have some log files named : 2016-02-10_03-52.log 2016-02-10_04-43.log 2016-02-10_02-13.log ... 2016-03-15_07-03.log 2016-03-15_09-08.log Basically the pattern is : YYYY-MM-DD_.log I would ...
0
votes
1answer
32 views

How can i remove duplicates from the output of tshark output [duplicate]

sudo tshark -i ppp0 'tcp port 80 \ and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)' \ -R'http.request.method == "GET" && http.request.uri contains ...
0
votes
2answers
51 views

Help in making multiple lines as single lines

I have below 4 lines: EXTRACT ETEST Last Started 2016-03-20 20:56 Status RUNNING Checkpoint Lag 00:00:02 (updated 00:00:04 ago) EXTRACT PTEST Initialized 2016-03-31 06:25 Status ...
-1
votes
1answer
51 views

Organize a file [closed]

I want to thank the great help and support. Well, I have the following file: name id alello chr pos snp s1 215 a 1 11 a215 s1 216 a 1 11 a216 s1 217 ...
2
votes
2answers
36 views

How To Use Awk To Print Between Two Positional Parameters

Is there a way to print everything after a positional parameters using awk? What I would like the following command to print is "p4 p5 p6 p7" echo p1 p2 p3 p4 p5 p6 p7 | awk '{print $4...}' ...
5
votes
2answers
216 views

Case-insensitive search in awk

I need to search for a keyword using awk, but I want to perform a case-insensitive (non case sensitive) search. I think the best approach is to capitalize both the search term ("key word") and the ...
1
vote
2answers
44 views

Compare 1st column of the 1st file and 2nd column of the 2nd file and get the the output as following [duplicate]

File1: Excel file (.xls) UN ID St M1 M2 SE DOF PV PA FC 17127159 0 -5.9 297.3 765.7 0.22 4 0.003 0.00389231 2.57536 17127163 2 ...
0
votes
1answer
53 views

Compare 1st column of the 1st file and 2nd column of the 2nd file and matching

File1: Excel file (.xls) UN ID St M1 M2 SE DOF PV PA FC 17127159 0 -5.9 297.3 765.7 0.22 4 0.003 0.00389231 2.57536 17127163 2 ...
0
votes
1answer
36 views

How to compare (using greater than and less than symbols) strings using awk [duplicate]

This is my input file: PA43171-7,1,3,/vobs/atlas-idc/src/utils/logger/IDCLogger/IDC.cpp,48,12:01:16.086‌​.660, Valid Sensor Reading,Date = 08-19-2014 , Time = 12:01:16 Voltage Sensor=11 , V$ $=11 , ...
0
votes
2answers
83 views

how does this work in awk even without print?

bash-3.00# cat > b1.txt AAA,Apples,123 BBB,Bananas,124 CCC,Carrot,125 bash-3.00# cat > b2.txt Store1|AAA|123|11 Store2|BBB|124|23 Store3|CCC|125|57 Store4|DDD|126|38 bash-3.00# awk -F"[,|]" ...
3
votes
2answers
4k views

Merging 2 files with based on field match

I want to create a file that contains columns from two input files. File1 is like: aa 32 bb 15 cc 78 File2 is: fa 19 bc 23 cc 50 de 28 aa 45 bb 31 The task is, read through File1, if the 1st field ...
0
votes
2answers
78 views

Curl and Grep to Parsing Content

I need to parse the output of a curl command: curl --user test:test http://192.168.1.1/security/pkm.html | egrep '@company|config.pkm.password' This returns: <input type="text" ...
0
votes
1answer
43 views

How to compare the strings using < (Greater than symbol)

Below is my input file: PA43410-2,1,3,/vobs/atlas-idc/src/utils/logger/IDCLogger/IDC.cpp,48,19:16:46.523.177, 2 PS Sensor Value = -5.501000 , Min = -5.583000 , Max = -5.319000 ...
3
votes
1answer
121 views

how to use awk to do subtraction with numbers in a large file

I have a large file like this AT5G44030,TAACARA,1000,+,200,206 AT5G44030,TAACARA,1000,+,418,424 AT5G44030,TAACARA,1000,+,773,779 AT5G44030,NGATT,1000,+,114,118 AT5G44030,NGATT,1000,+,267,271 ...
4
votes
2answers
282 views

awk - combine two filters into one

I have this variable in a bash script on ubuntu 12.04 which is set to show the available MB on the root partition: AV_MB=$(df -m |awk NR==2 |awk '{print $4}') Is there an elegant way to combine ...
0
votes
1answer
28 views

AWK weird output behavior when adding extra column

I have a file.csv that contains: ID,FN,LN,NUM 576,Pra,Mar,3 688,India,Desa,3 645,Ami,Path,5 245,Me,Yo,4 I want the output to be ID,FN,LN,NUM,R 576,Pra,Mar,3,R 688,India,Desa,3,R 645,Ami,Path,5,R ...
-1
votes
4answers
88 views

How to break a string into smaller substrings with a condition on their lengths?

I have a file that contains some genetic sequences like this one: ...
1
vote
2answers
37 views

awk if loop not able to print

I have a variable Test as below Test="Environment Dev" I want to compare col2 char 1 with others and print the whole name of env this is what i did echo $Test|awk '{ if ($2 == "^D*") print "DEV"; ...
0
votes
1answer
27 views

get unique values from 1 column and sum up 2 next columns [closed]

I have a text file that looks like this: A 500 100 ABC B 200 100 BBB B 200 100 ACC D 150 235 CDD A 500 100 RER C 210 653 DDD C 210 653 DWR D 150 653 DDW D 150 653 DER What I try to accomplish is to ...
1
vote
1answer
67 views

I need to merge multiple .csv files together side by side with unequal number of rows

I have anywhere from 3-5 .csv files and need to be able to merge them together while keeping everything in its respective columns below is an easy example with files with varying number of rows. file1 ...
0
votes
3answers
30 views

Perform regex search on a previously regex-matched line (one-liner)

I need to print a string which is in a unique line in a long text file. Example relevant line in file long.txt: Software version 4.0.10, 2002-03-28 The wanted string is exactly 4.0.10. The steps ...
0
votes
1answer
46 views

Awk and Grep to get data from CLI

I want to get output as below: Number Name Value ************************************* 889900 Staff 256000 990088 Staff 1048576 And my input would be as below: ...
0
votes
1answer
54 views

bash script to replace server short names in a file with fqdn

I have a rather large config file that has server short names in it like this: [host 1] host = server1 [host 2] host = server2 [host 3] host = server3 I need to replace all the server short names ...
1
vote
2answers
51 views

Count times an expression is true in awk.

In unix/awk code Essentially, I need to iterate through the file (data1.txt) and count the number of times the substr($0,29,2) of each line of the file == "04". data1.txt: 6597243042 ...
0
votes
1answer
33 views

Curly bracket confusion! [closed]

Curly brackets question, with this file content. one one one two two two three 098234 one one one two two two three 098234 one one one two two two three 098234 ... awk '{ a[NR]=$0 } END ...
1
vote
1answer
38 views

Only output value if x > y in awk

INPUT: 0:root@server:/root # vmstat 60 2 System configuration: lcpu=52 mem=131072MB ent=10.00 kthr memory page faults cpu ----- ----------- ...
0
votes
1answer
34 views

Why does print work with awk but echo doesn't?

To sum it up.... This works: awk 'BEGIN { FS=":"; } { print $1 $3 $5; }' /etc/passwd But this doesn't: awk 'BEGIN { FS=":"; } { echo $1 $3 $5; }' /etc/passwd Being fairly new to learning ...
3
votes
1answer
40 views

comparing values in separate columns

I've got a csv that looks like this, (there are other columns but these are the pertinent ones) region, city, actual rating, predicted rating NSW, SYD, 0.765, 0.5 Victoria, Melbourne, 4.81225, 5.0 ...
0
votes
1answer
33 views

extract the data from 2 files

I have two files, the first file contain only 1 field, the second file has more lines than first file and also it has different field in each line. File 1: 5 15 20 File 2: 3 5 O 1.25 2.30 3.75 ...
1
vote
4answers
101 views

Bash: How to print additional column matching IP Address to Name

I'm using netstat to collect information about connected IP's. And I have the following format: netstat -tn 2>/dev/null | awk '/:80 / {print $5}' | sed 's/.*::ffff://' | sed 's/:.*//' | sor t | ...
0
votes
2answers
62 views

How to get only certain fields from a given file?

I have list of employees in following format Name: Phone No: Email: Name: Address: Phone No: Email: Name: Country: Address: Phone No: Email: Name: Address: Name : Email: Address: All I want ...
1
vote
2answers
32 views

Search for processes which are CPU consumming

I can track the process launched by the user4 with awk: ps aux | awk '$1~/user4/' And also to track what process consume more than 10% CPU: ps aux | awk '$3> 10' Hoe the question is not too ...
1
vote
2answers
80 views

Find files containing a set of words

I'm using grep to find files within a directory, containing a set of words. But grep search lines containing these words, what I want is grep to show me the files or file containing all those words ...
54
votes
16answers
71k views

Is there a way to get the min, max, median, and average of a list of numbers in a single command?

I have a list of numbers in a file, one per line. How can I get the minimum, maximum, median and average values? I want to use the results in a bash script. Although my immediate situation is for ...
3
votes
2answers
46 views

Sorting by two columns

I need help with this homework assignment: Write a shell script named sales1 using nawk and sort commands to do the following: 1. print the file in two sections, east and west, with lines alphabetized ...
2
votes
2answers
27 views

average of specific rows in a file and output in tab-delimited file

I have 6 rows in files. I need to find average only of specific rows in a file and the others should be left as they are. The average should be calculated for A1 and A2, B1 and B2, other lines should ...
1
vote
1answer
52 views

Extract random sample of N lines based on pattern

I have a file formatted like this: train/t/temple/east_asia/00000025.jpg 94 train/t/temple/east_asia/00000865.jpg 94 ... train/s/swamp/00000560.jpg 92 train/s/swamp/00000935.jpg 92 .... ...
3
votes
3answers
66 views

Replace strings in a file based on a list of strings and a list of corresponding replacements

I am trying to replace strings in a file A: Hello Peter, how is your dad? where is mom? where the strings to be replaced are in file B: Peter dad mom and their corresponding replacements are in ...
0
votes
1answer
32 views

Remove tab, new line, carriage return until encountered char

I have a CSV file that has record separator of ,\n. One of the columns, has the possibility of containing tabs and new lines, which I wish to remove. How do I remove these chars (\t\n\r) using sed ...
2
votes
0answers
48 views

Awk /sed extract information when a pattern match from a paragraph

I want to search a pattern "FROM" in paragraph that begins with CREATE VIEW and ends with ";" and save the result in a csv file. for example if I have the following file : CREATE VIEW view1 AS ...
0
votes
1answer
36 views

Monitoring Server Connections - Netstat formatting issue

I have had a few issues with a server recently. So i just wanted to leave a window showing the unique and IP's of connected devices. I have been using: watch -n 5 "netstat -tn 2>/dev/null | grep ...
2
votes
2answers
363 views

How do I add an if statement (regarding punctuation in a word) to this bash script

I have this bash script #!/bin/bash cat $@ | while read line do for word in $line do echo $word | circling-the-square # here's where i need to add the if statement: ...
0
votes
3answers
39 views

remove sequential lines that begins with the same pattern using awk or sed

I want to remove the sequential lines that begins with the same pattern "CREATE CURRENT" using sed or awk for example if have the folowwing file: CREATE CURRENT = 'aaaa' ; CREATE CURRENT = 'bbbb' ; ...
0
votes
1answer
42 views

subset a file just on one column

I have a tab delim file, I want to make a subset on the basis of column 2, choosing "A". 435 A 2 27.92464882 8.63E-07 1 752721 564 F 2 30.12532071 2.87E-07 1 798959 654 A 2 ...
0
votes
2answers
50 views

insert complex line before or after complex line

i have file with complex text print("ERROR: passwords don't match") password = hash_func(password.encode("UTF-8")).hexdigest() i need to insert this text between them with ...
1
vote
3answers
43 views

Count occurance of number and delete line of matches value

Could I get some help please with my problem? I have a file that looks like this (comma separated): A,250,13,820,0,80,201 B,0,0,0,0,3,0 C,0,2,1,0,0,0 D,0,1,105,0,0,10 I would like to ...
0
votes
2answers
53 views

How to output some data to different cells of an Excel File?

Example: I have a file that has following data: cvrta2100 Error 404 Fixed cvrta2111 Licensed Needs Fixing cvrta2123 Error 404 Fixed cvrta2333 Licensed Needs Fixing I am using awk to print ...