awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
2answers
37 views

awk if loop not able to print

I have a variable Test as below Test="Environment Dev" I want to compare col2 char 1 with others and print the whole name of env this is what i did echo $Test|awk '{ if ($2 == "^D*") print "DEV"; ...
0
votes
1answer
29 views

get unique values from 1 column and sum up 2 next columns [closed]

I have a text file that looks like this: A 500 100 ABC B 200 100 BBB B 200 100 ACC D 150 235 CDD A 500 100 RER C 210 653 DDD C 210 653 DWR D 150 653 DDW D 150 653 DER What I try to accomplish is to ...
1
vote
1answer
86 views

I need to merge multiple .csv files together side by side with unequal number of rows

I have anywhere from 3-5 .csv files and need to be able to merge them together while keeping everything in its respective columns below is an easy example with files with varying number of rows. file1 ...
0
votes
3answers
31 views

Perform regex search on a previously regex-matched line (one-liner)

I need to print a string which is in a unique line in a long text file. Example relevant line in file long.txt: Software version 4.0.10, 2002-03-28 The wanted string is exactly 4.0.10. The steps ...
0
votes
1answer
48 views

Awk and Grep to get data from CLI

I want to get output as below: Number Name Value ************************************* 889900 Staff 256000 990088 Staff 1048576 And my input would be as below: ...
0
votes
1answer
60 views

bash script to replace server short names in a file with fqdn

I have a rather large config file that has server short names in it like this: [host 1] host = server1 [host 2] host = server2 [host 3] host = server3 I need to replace all the server short names ...
1
vote
2answers
52 views

Count times an expression is true in awk.

In unix/awk code Essentially, I need to iterate through the file (data1.txt) and count the number of times the substr($0,29,2) of each line of the file == "04". data1.txt: 6597243042 ...
0
votes
1answer
33 views

Curly bracket confusion! [closed]

Curly brackets question, with this file content. one one one two two two three 098234 one one one two two two three 098234 one one one two two two three 098234 ... awk '{ a[NR]=$0 } END ...
1
vote
1answer
39 views

Only output value if x > y in awk

INPUT: 0:root@server:/root # vmstat 60 2 System configuration: lcpu=52 mem=131072MB ent=10.00 kthr memory page faults cpu ----- ----------- ...
0
votes
1answer
37 views

Why does print work with awk but echo doesn't?

To sum it up.... This works: awk 'BEGIN { FS=":"; } { print $1 $3 $5; }' /etc/passwd But this doesn't: awk 'BEGIN { FS=":"; } { echo $1 $3 $5; }' /etc/passwd Being fairly new to learning ...
3
votes
1answer
41 views

comparing values in separate columns

I've got a csv that looks like this, (there are other columns but these are the pertinent ones) region, city, actual rating, predicted rating NSW, SYD, 0.765, 0.5 Victoria, Melbourne, 4.81225, 5.0 ...
0
votes
1answer
36 views

extract the data from 2 files

I have two files, the first file contain only 1 field, the second file has more lines than first file and also it has different field in each line. File 1: 5 15 20 File 2: 3 5 O 1.25 2.30 3.75 ...
1
vote
4answers
105 views

Bash: How to print additional column matching IP Address to Name

I'm using netstat to collect information about connected IP's. And I have the following format: netstat -tn 2>/dev/null | awk '/:80 / {print $5}' | sed 's/.*::ffff://' | sed 's/:.*//' | sor t | ...
0
votes
2answers
62 views

How to get only certain fields from a given file?

I have list of employees in following format Name: Phone No: Email: Name: Address: Phone No: Email: Name: Country: Address: Phone No: Email: Name: Address: Name : Email: Address: All I want ...
1
vote
2answers
33 views

Search for processes which are CPU consumming

I can track the process launched by the user4 with awk: ps aux | awk '$1~/user4/' And also to track what process consume more than 10% CPU: ps aux | awk '$3> 10' Hoe the question is not too ...
1
vote
2answers
96 views

Find files containing a set of words

I'm using grep to find files within a directory, containing a set of words. But grep search lines containing these words, what I want is grep to show me the files or file containing all those words ...
57
votes
16answers
73k views

Is there a way to get the min, max, median, and average of a list of numbers in a single command?

I have a list of numbers in a file, one per line. How can I get the minimum, maximum, median and average values? I want to use the results in a bash script. Although my immediate situation is for ...
3
votes
2answers
47 views

Sorting by two columns

I need help with this homework assignment: Write a shell script named sales1 using nawk and sort commands to do the following: 1. print the file in two sections, east and west, with lines alphabetized ...
2
votes
2answers
28 views

average of specific rows in a file and output in tab-delimited file

I have 6 rows in files. I need to find average only of specific rows in a file and the others should be left as they are. The average should be calculated for A1 and A2, B1 and B2, other lines should ...
1
vote
1answer
57 views

Extract random sample of N lines based on pattern

I have a file formatted like this: train/t/temple/east_asia/00000025.jpg 94 train/t/temple/east_asia/00000865.jpg 94 ... train/s/swamp/00000560.jpg 92 train/s/swamp/00000935.jpg 92 .... ...
0
votes
1answer
33 views

Remove tab, new line, carriage return until encountered char

I have a CSV file that has record separator of ,\n. One of the columns, has the possibility of containing tabs and new lines, which I wish to remove. How do I remove these chars (\t\n\r) using sed ...
2
votes
0answers
52 views

Awk /sed extract information when a pattern match from a paragraph

I want to search a pattern "FROM" in paragraph that begins with CREATE VIEW and ends with ";" and save the result in a csv file. for example if I have the following file : CREATE VIEW view1 AS ...
0
votes
1answer
43 views

Monitoring Server Connections - Netstat formatting issue

I have had a few issues with a server recently. So i just wanted to leave a window showing the unique and IP's of connected devices. I have been using: watch -n 5 "netstat -tn 2>/dev/null | grep ...
2
votes
2answers
369 views

How do I add an if statement (regarding punctuation in a word) to this bash script

I have this bash script #!/bin/bash cat $@ | while read line do for word in $line do echo $word | circling-the-square # here's where i need to add the if statement: ...
0
votes
3answers
39 views

remove sequential lines that begins with the same pattern using awk or sed

I want to remove the sequential lines that begins with the same pattern "CREATE CURRENT" using sed or awk for example if have the folowwing file: CREATE CURRENT = 'aaaa' ; CREATE CURRENT = 'bbbb' ; ...
0
votes
1answer
42 views

subset a file just on one column

I have a tab delim file, I want to make a subset on the basis of column 2, choosing "A". 435 A 2 27.92464882 8.63E-07 1 752721 564 F 2 30.12532071 2.87E-07 1 798959 654 A 2 ...
0
votes
2answers
50 views

insert complex line before or after complex line

i have file with complex text print("ERROR: passwords don't match") password = hash_func(password.encode("UTF-8")).hexdigest() i need to insert this text between them with ...
1
vote
3answers
43 views

Count occurance of number and delete line of matches value

Could I get some help please with my problem? I have a file that looks like this (comma separated): A,250,13,820,0,80,201 B,0,0,0,0,3,0 C,0,2,1,0,0,0 D,0,1,105,0,0,10 I would like to ...
0
votes
2answers
73 views

How to output some data to different cells of an Excel File?

Example: I have a file that has following data: cvrta2100 Error 404 Fixed cvrta2111 Licensed Needs Fixing cvrta2123 Error 404 Fixed cvrta2333 Licensed Needs Fixing I am using awk to print ...
6
votes
4answers
8k views

Redirecting Output of print to multiples files in awk

I want to print same lines to 2 different files. What I am doing right now is: print ID[i]" "Value[i]" "TS1[i]" "TS2[i] > "file1.txt"; print ID[i]" "Value[i]" "TS1[i]" "TS2[i] > "file2.txt"; ...
1
vote
3answers
73 views

How to do following formatting using sed or awk or bash script on linux

Suppose, I have two files: file1 having below contents. message_aaa.wav test 1 of 1 test Label : aaa test Sentence : test aaa test String: test message Interpretation ...
1
vote
2answers
54 views

AWK: How to put field ($1) inside regular expression to select all prefixes?

Assume I have a file A: fileA fileB Suppose I have now a file named: fileA_someprefix_20160101.txt Now I want to match all lines from A which prefix this filename, so I thought: ...
3
votes
2answers
58 views

match columns and replace

I am newbie in linux. I have two very large text files with space-delimited fields: File1 527858 51 2 27.92464882 8.63E-07 570289 82 2 30.12532071 2.87E-07 571034 90 2 ...
0
votes
2answers
46 views

using awk to unzip files in directory

I have some zip files in directory when I need to unzip each in specific dir I used ls | awk '{ print "unzip " $1 " -d " $1} ' unzip p21286665_121020_Linux-x86-64.zip -d ...
0
votes
1answer
36 views

Merge files based on matching of first column

I have two files: File1 ARS-BFGL-BAC-10975 0.9303 688423261 1 01/04/2015 0.9983763305 ARS-BFGL-BAC-11025 0.9092 688423261 1 01/04/2015 0.9983763305 ARS-BFGL-BAC-11044 0.9626 688423261 2 01/04/2015 ...
4
votes
3answers
114 views

String replacement using a dictionary

What is a good way to do string replacements in a file using a dictionary with a lot of substituend-substituent pairs? And by a lot, I actually mean about 20 – not much, but many enough so that I want ...
-1
votes
1answer
44 views
2
votes
1answer
60 views

I can't find the process sending mail?

I did something stupid and ran an awk that runs a while loop that sends an email every 10 seconds. I ran this as a background process: egrep '^[1-9]' /etc/hosts | grep -v 'localhost' | awk '{ ...
-1
votes
1answer
28 views

Remove pattern from a line in file with Grep/sed/awk combo, pattern is another column

I have two columns (or could separate as two files) in a huge file and i need to remove the duplicate city name in the state/area column. los angeles los angeles ca, usa, west new york ...
2
votes
6answers
18k views

awk-Printing column value without new line and adding comma

input.txt EN1 EN2 EN3 EN4 EN5 output EN1,EN2,EN3,EN4,EN5 I have tried awk.But it is not printing with comma awk 'BEGIN { OFS = ","} { printf $1}' input.txt I have GNU Awk 4.0.0 version
6
votes
1answer
133 views

Make bash exit != 0 when called by AWK and interrupted with ^C

I'm having an issue with the following piece of (G)AWK script: do { ... } while (system("sleep 10")) My intention is to break the loop when the user presses ^C during the sleep, but it's not ...
1
vote
1answer
30 views

Calculate hits within a timespan from log file [closed]

I need to use UNIX utilities that implement out-of-core divide-and-conquer algorithms so to decrease the amount of virtual memory used (cap of 1 mb). I have a large data set that is an apache log ...
0
votes
3answers
55 views

Awk command is not working properly

I have a very simple problem, but for reasons it does not work properly.. I have these .txt files in the following format 2 250 1 4 250 1 5 250 1 I wanted to subtract 1 from the ...
3
votes
2answers
75 views

Passing a list of two PIDs to xargs only kills the first using ssh

I'm retrieving a list of two PIDs that I want to kill. My pipeline looks something like ps -ef | grep foo | grep -v grep | awk {'print $2'} | xargs kill -9 Both processes are killed when executing ...
10
votes
3answers
13k views

How to sed only that lines that contains given string?

INPUT: Select ASDF 325 sdfg sdflk lk Select TRG 46sdg rasdftz fsgs 45 Select ASDF 6ffg sdfg 4456 sdrg OUTPUT: Select ASDF 325 XXXX sdflk lk Select TRG 46sdg rasdftz fsgs 45 Select ASDF 6ffg XXXX ...
1
vote
3answers
72 views

Awk, print result

Why does awk '{print 3 + 3}' not work, but: echo 3|awk '{print $1 + 3}' does? Couldn't the first statement just output 6? What is the easiest way of printing the result of a calculation?
0
votes
1answer
16 views

UNIX AWK : Get Logs from 24 Hours

I am trying to extract data from the log files for last 24 hours . I am trying awk -v d="$(date -d'24 hours ago' +'%Y-%m-%d %H:%M')" '$1" "$2>=d &&/ERROR/' data.log This does find ...
2
votes
5answers
2k views

How let `sort | uniq -c` separate the number of occurrences by a tab charactor?

uniq -c separate the occurrence number by spaces, which is hard for to cut or awk to separate it out later. 1000_A1\tB1\n ___1_A2\tB2\n I can solve this problem using sed -r 's/^ *([0-9]+)/\1\t/' ...
3
votes
1answer
70 views

How to optimize 'grep'-and-save?

I want to know on which node my calculations were running on our cluster in case the node crashes and I get no files back to know on which node I have to search. For that case I wrote a little script ...
1
vote
2answers
37 views

Joining columns from files if they contain a match in another column

I have two files: File1.txt 30 40 A T match1 string1 45 65 G R match2 string2 50 78 C Y match3 string3 File2.txt match1 60 add1 50 add2 ...