awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
1answer
28 views

get unique values from 1 column and sum up 2 next columns [closed]

I have a text file that looks like this: A 500 100 ABC B 200 100 BBB B 200 100 ACC D 150 235 CDD A 500 100 RER C 210 653 DDD C 210 653 DWR D 150 653 DDW D 150 653 DER What I try to accomplish is to ...
1
vote
2answers
37 views

awk if loop not able to print

I have a variable Test as below Test="Environment Dev" I want to compare col2 char 1 with others and print the whole name of env this is what i did echo $Test|awk '{ if ($2 == "^D*") print "DEV"; ...
1
vote
1answer
70 views

I need to merge multiple .csv files together side by side with unequal number of rows

I have anywhere from 3-5 .csv files and need to be able to merge them together while keeping everything in its respective columns below is an easy example with files with varying number of rows. file1 ...
0
votes
1answer
46 views

Awk and Grep to get data from CLI

I want to get output as below: Number Name Value ************************************* 889900 Staff 256000 990088 Staff 1048576 And my input would be as below: ...
0
votes
3answers
31 views

Perform regex search on a previously regex-matched line (one-liner)

I need to print a string which is in a unique line in a long text file. Example relevant line in file long.txt: Software version 4.0.10, 2002-03-28 The wanted string is exactly 4.0.10. The steps ...
0
votes
1answer
54 views

bash script to replace server short names in a file with fqdn

I have a rather large config file that has server short names in it like this: [host 1] host = server1 [host 2] host = server2 [host 3] host = server3 I need to replace all the server short names ...
1
vote
2answers
51 views

Count times an expression is true in awk.

In unix/awk code Essentially, I need to iterate through the file (data1.txt) and count the number of times the substr($0,29,2) of each line of the file == "04". data1.txt: 6597243042 ...
0
votes
1answer
33 views

Curly bracket confusion! [closed]

Curly brackets question, with this file content. one one one two two two three 098234 one one one two two two three 098234 one one one two two two three 098234 ... awk '{ a[NR]=$0 } END ...
1
vote
5answers
125 views

How to print nth line after match / empty lines

I'm trying to write a short script that finds empty lines and then prints the nth line after the empty lines. For foo1 foo2 foo3 bar1 bar2 bar3 spam1 spam2 spam3 eggs1 eggs2 eggs3 printing ...
1
vote
1answer
38 views

Only output value if x > y in awk

INPUT: 0:root@server:/root # vmstat 60 2 System configuration: lcpu=52 mem=131072MB ent=10.00 kthr memory page faults cpu ----- ----------- ...
0
votes
1answer
35 views

Why does print work with awk but echo doesn't?

To sum it up.... This works: awk 'BEGIN { FS=":"; } { print $1 $3 $5; }' /etc/passwd But this doesn't: awk 'BEGIN { FS=":"; } { echo $1 $3 $5; }' /etc/passwd Being fairly new to learning ...
3
votes
1answer
41 views

comparing values in separate columns

I've got a csv that looks like this, (there are other columns but these are the pertinent ones) region, city, actual rating, predicted rating NSW, SYD, 0.765, 0.5 Victoria, Melbourne, 4.81225, 5.0 ...
0
votes
1answer
33 views

extract the data from 2 files

I have two files, the first file contain only 1 field, the second file has more lines than first file and also it has different field in each line. File 1: 5 15 20 File 2: 3 5 O 1.25 2.30 3.75 ...
0
votes
2answers
62 views

How to get only certain fields from a given file?

I have list of employees in following format Name: Phone No: Email: Name: Address: Phone No: Email: Name: Country: Address: Phone No: Email: Name: Address: Name : Email: Address: All I want ...
1
vote
4answers
102 views

Bash: How to print additional column matching IP Address to Name

I'm using netstat to collect information about connected IP's. And I have the following format: netstat -tn 2>/dev/null | awk '/:80 / {print $5}' | sed 's/.*::ffff://' | sed 's/:.*//' | sor t | ...
1
vote
2answers
32 views

Search for processes which are CPU consumming

I can track the process launched by the user4 with awk: ps aux | awk '$1~/user4/' And also to track what process consume more than 10% CPU: ps aux | awk '$3> 10' Hoe the question is not too ...
3
votes
2answers
46 views

Sorting by two columns

I need help with this homework assignment: Write a shell script named sales1 using nawk and sort commands to do the following: 1. print the file in two sections, east and west, with lines alphabetized ...
2
votes
2answers
27 views

average of specific rows in a file and output in tab-delimited file

I have 6 rows in files. I need to find average only of specific rows in a file and the others should be left as they are. The average should be calculated for A1 and A2, B1 and B2, other lines should ...
1
vote
2answers
80 views

Find files containing a set of words

I'm using grep to find files within a directory, containing a set of words. But grep search lines containing these words, what I want is grep to show me the files or file containing all those words ...
1
vote
1answer
52 views

Extract random sample of N lines based on pattern

I have a file formatted like this: train/t/temple/east_asia/00000025.jpg 94 train/t/temple/east_asia/00000865.jpg 94 ... train/s/swamp/00000560.jpg 92 train/s/swamp/00000935.jpg 92 .... ...
3
votes
3answers
69 views

Replace strings in a file based on a list of strings and a list of corresponding replacements

I am trying to replace strings in a file A: Hello Peter, how is your dad? where is mom? where the strings to be replaced are in file B: Peter dad mom and their corresponding replacements are in ...
0
votes
1answer
33 views

Remove tab, new line, carriage return until encountered char

I have a CSV file that has record separator of ,\n. One of the columns, has the possibility of containing tabs and new lines, which I wish to remove. How do I remove these chars (\t\n\r) using sed ...
2
votes
0answers
48 views

Awk /sed extract information when a pattern match from a paragraph

I want to search a pattern "FROM" in paragraph that begins with CREATE VIEW and ends with ";" and save the result in a csv file. for example if I have the following file : CREATE VIEW view1 AS ...
0
votes
1answer
36 views

Monitoring Server Connections - Netstat formatting issue

I have had a few issues with a server recently. So i just wanted to leave a window showing the unique and IP's of connected devices. I have been using: watch -n 5 "netstat -tn 2>/dev/null | grep ...
0
votes
3answers
39 views

remove sequential lines that begins with the same pattern using awk or sed

I want to remove the sequential lines that begins with the same pattern "CREATE CURRENT" using sed or awk for example if have the folowwing file: CREATE CURRENT = 'aaaa' ; CREATE CURRENT = 'bbbb' ; ...
0
votes
1answer
42 views

subset a file just on one column

I have a tab delim file, I want to make a subset on the basis of column 2, choosing "A". 435 A 2 27.92464882 8.63E-07 1 752721 564 F 2 30.12532071 2.87E-07 1 798959 654 A 2 ...
0
votes
2answers
50 views

insert complex line before or after complex line

i have file with complex text print("ERROR: passwords don't match") password = hash_func(password.encode("UTF-8")).hexdigest() i need to insert this text between them with ...
1
vote
3answers
43 views

Count occurance of number and delete line of matches value

Could I get some help please with my problem? I have a file that looks like this (comma separated): A,250,13,820,0,80,201 B,0,0,0,0,3,0 C,0,2,1,0,0,0 D,0,1,105,0,0,10 I would like to ...
0
votes
2answers
53 views

How to output some data to different cells of an Excel File?

Example: I have a file that has following data: cvrta2100 Error 404 Fixed cvrta2111 Licensed Needs Fixing cvrta2123 Error 404 Fixed cvrta2333 Licensed Needs Fixing I am using awk to print ...
3
votes
2answers
56 views

match columns and replace

I am newbie in linux. I have two very large text files with space-delimited fields: File1 527858 51 2 27.92464882 8.63E-07 570289 82 2 30.12532071 2.87E-07 571034 90 2 ...
1
vote
3answers
73 views

How to do following formatting using sed or awk or bash script on linux

Suppose, I have two files: file1 having below contents. message_aaa.wav test 1 of 1 test Label : aaa test Sentence : test aaa test String: test message Interpretation ...
1
vote
2answers
51 views

AWK: How to put field ($1) inside regular expression to select all prefixes?

Assume I have a file A: fileA fileB Suppose I have now a file named: fileA_someprefix_20160101.txt Now I want to match all lines from A which prefix this filename, so I thought: ...
0
votes
2answers
41 views

using awk to unzip files in directory

I have some zip files in directory when I need to unzip each in specific dir I used ls | awk '{ print "unzip " $1 " -d " $1} ' unzip p21286665_121020_Linux-x86-64.zip -d ...
0
votes
1answer
32 views

Merge files based on matching of first column

I have two files: File1 ARS-BFGL-BAC-10975 0.9303 688423261 1 01/04/2015 0.9983763305 ARS-BFGL-BAC-11025 0.9092 688423261 1 01/04/2015 0.9983763305 ARS-BFGL-BAC-11044 0.9626 688423261 2 01/04/2015 ...
-1
votes
1answer
41 views

how can i extract the second and third word of a line{that starts with 9} from a file where words are seperate with |? [closed]

case "$3" in -id) echo "Lines matches with the pattern $4 :" #grep -oP $4 $2 ;; esac
1
vote
1answer
48 views

I can't find the process sending mail?

I did something stupid and ran an awk that runs a while loop that sends an email every 10 seconds. I ran this as a background process: egrep '^[1-9]' /etc/hosts | grep -v 'localhost' | awk '{ ...
4
votes
3answers
86 views

String replacement using a dictionary

What is a good way to do string replacements in a file using a dictionary with a lot of substituend-substituent pairs? And by a lot, I actually mean about 20 – not much, but many enough so that I want ...
-1
votes
1answer
25 views

Remove pattern from a line in file with Grep/sed/awk combo, pattern is another column

I have two columns (or could separate as two files) in a huge file and i need to remove the duplicate city name in the state/area column. los angeles los angeles ca, usa, west new york ...
0
votes
3answers
54 views

Awk command is not working properly

I have a very simple problem, but for reasons it does not work properly.. I have these .txt files in the following format 2 250 1 4 250 1 5 250 1 I wanted to subtract 1 from the ...
1
vote
1answer
29 views

Calculate hits within a timespan from log file [closed]

I need to use UNIX utilities that implement out-of-core divide-and-conquer algorithms so to decrease the amount of virtual memory used (cap of 1 mb). I have a large data set that is an apache log ...
3
votes
2answers
61 views

Passing a list of two PIDs to xargs only kills the first using ssh

I'm retrieving a list of two PIDs that I want to kill. My pipeline looks something like ps -ef | grep foo | grep -v grep | awk {'print $2'} | xargs kill -9 Both processes are killed when executing ...
3
votes
2answers
297 views

How do I print the (numerical) ASCII values of each character in a file?

How can I print the numerical ASCII values of each character in a text file. Like cat, but showing the ASCII values only... (hex or decimal is fine). Example output for a file containing the word ...
1
vote
3answers
71 views

Awk, print result

Why does awk '{print 3 + 3}' not work, but: echo 3|awk '{print $1 + 3}' does? Couldn't the first statement just output 6? What is the easiest way of printing the result of a calculation?
1
vote
2answers
35 views

Joining columns from files if they contain a match in another column

I have two files: File1.txt 30 40 A T match1 string1 45 65 G R match2 string2 50 78 C Y match3 string3 File2.txt match1 60 add1 50 add2 ...
0
votes
1answer
12 views

UNIX AWK : Get Logs from 24 Hours

I am trying to extract data from the log files for last 24 hours . I am trying awk -v d="$(date -d'24 hours ago' +'%Y-%m-%d %H:%M')" '$1" "$2>=d &&/ERROR/' data.log This does find ...
3
votes
1answer
70 views

How to optimize 'grep'-and-save?

I want to know on which node my calculations were running on our cluster in case the node crashes and I get no files back to know on which node I have to search. For that case I wrote a little script ...
11
votes
2answers
525 views

awk memory leak?

Base on this I'm running the command < /dev/urandom hexdump -v -e '/1 "%u\n"' | awk '{ split("0,2,4,5,7,9,11,12",a,","); for (i = 0; i < 1; i+= 0.0001) printf("%08X\n", ...
1
vote
2answers
34 views

Reorganizing lines in a text file

So I have the following file: First Line Second Line Third Line 3 0.374 2 0.777 1 0.245 Fourth Line 4 0.687 ... And I want to reorganize it so the lines beginning with an int are appended to the ...
6
votes
1answer
129 views

Make bash exit != 0 when called by AWK and interrupted with ^C

I'm having an issue with the following piece of (G)AWK script: do { ... } while (system("sleep 10")) My intention is to break the loop when the user presses ^C during the sleep, but it's not ...
0
votes
1answer
32 views

Subset rows with awk [duplicate]

I am new to Linux, but my work requires me to do some specific task now, I hope you please help me for this. I have a data set as follow: Cow_ID Age DIM 1 1424 0 1 1425 1 1 1426 2 1 ...