awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

3
votes
1answer
39 views

grep output of cdrecord

I would like to parse the output of cdrecord/wodim using grep. cdrecord -v outputs lines like Track 01: 49 of 75 MB written (fifo 100%) [buf 100%] 10.2x. The problem is that cdrecord uses ...
1
vote
1answer
123 views

How to subtract rows from first row using awk?

My data file looks like: 10 -0.314690785295 20 -0.251967909317 30 -0.215271387106 40 -0.189228416217 The desired output is: 10 -0.0627229 20 -0.0994193 30 -0.1254623 I am really new in bash, so ...
0
votes
2answers
38 views

How to change the first column of data in a file?

I have .data file containing a table of numbers with the column format: m x y z vx vy vz separated by tabs, with a depth of 2000. The file name is initc.data. I want to change ...
0
votes
2answers
60 views

Collecting network information

I need to collect information about the network usage of each process. Nethogs presents the data I need in real time, I am trying to save the output to a file in order parse it and plot the data. ...
1
vote
4answers
67 views

Print only unique lines from file not the duplicates

I have a sorted words list, line by line file like so: apple apple grapes lime orange orange pear pear peach strawberry strawberry I only want to print out unique lines and drop duplicates: grapes ...
8
votes
1answer
169 views

Trap Ctrl-C in awk script

I believe Ctrl-C can be trapped in bash scripts. Is it also possible to trap it inside an Awk script in order to handle that event? For example, for aborting processing, but printing the results of ...
3
votes
3answers
306 views

Select the lines with exactly two columns in Linux

I have a file that is a mapping between filename and the corresponding transcripts. The file name and transcripts is separated by a TAB character. The transcription of the file may contain one or ...
0
votes
2answers
31 views

Joining certain lines of text together before modifying them

I have a cricket commentary like - 0.3 Richardson to RG Sharma, 2 runs, gentle, non-swinging length ball on off, Rohit seeks to pick it up. Hit high on the bat and spooned over the returning Mitch ...
1
vote
1answer
228 views

How do you list number of lines of every file in a directory in human readable format.

I have a list of directories and subdirectories that contain large csv files. There are about 500 million lines in these files, each is a record. I would like to know How many lines are in each ...
1
vote
1answer
49 views

Syntax error removing duplicate lines via awk '!x[$0]++' file

The file contains cat file a b c b d Trying to remove duplicate lines on SunOS via awk '!x[$0]++' file (as found in another postings) results in a syntax error awk: syntax error near line 1 awk:...
0
votes
2answers
50 views

Need help in data arrangement

Could you please help me to sort this out, I have data which i would like get output in below format using shell/perl script. Data: Virtual Machine Test status Running. Assigned Server OVS001....
0
votes
1answer
91 views

delete line that contains a case insensitive match

I have a file that contains information as so: 20 BaDDOg 31 baddog 42 badCAT 43 goodDoG 44 GOODcAT and I want to delete all lines that contain the word dog. This is my desired output:...
2
votes
3answers
311 views

awk : parse and write to another file

I have records in XML file like below. I need to search for <keyword>SEARCH</keyword> and if present then I need to take the entire record and write to another file.(starting from <...
1
vote
1answer
44 views

Linux + capture only the IP's from mount command

please advise how I can to print only the list of the IP's ( in the end of the lines ) , from mount output mount netapp64:/lunggg/OracleTeam/OracleCDs on /oracds type nfs (rw,fg,hard,nointr,nolock,...
0
votes
1answer
81 views

Add column to a CSV file as a function of other two columns, one from other file

Here is the situation: a.csv Contains values from some sensors captured every 1 second, the last column is the epoch (they actually looks like 1453998477.044).(*s are the sensors values). Example: ...
1
vote
5answers
916 views

Grep value from json file

my json file looks something like this [{"product":"Apple","id":"2134"},{"product":"Mango","id":"4567"}] I want to grep/sed/awk 'id' for given 'product'. Output: Enter product : Apple Your ...
2
votes
1answer
28 views

grep awk or sed a CSV Row containing a certain part of a web address

I am trying to clean up a Squid log file and I want to delete rows with a website containing "/0/" in the 11th column. An example: Row1: column1, column2, column3...column10, ht*p://blah.com/page/230/...
1
vote
2answers
57 views

awk is putting last column in a new line

I have this -77.199997 23.6343749630 39.9989584580 -80.000000 23.6364583430 39.9989584580 -81.599998 23.6385417220 39.9989584580 I want to reorder the columns so it reads column 2, ...
0
votes
1answer
167 views

Split string into array and print each element on a new line with commandline

I'm having a String which is seperated by commas like a,b,c,d,e,f that I want to split into an array with the comma as seperator. Then I want to print each element on a new line. The problem I'm ...
1
vote
1answer
64 views

splitting file based on values in specific column

I have a file that I would like to break up into multiple files with uniq values for the first column. For example, here is a file: fileA.txt 1 Cat 1 Dog 1 Frog 2 Boy 2 Girl 3 ...
1
vote
1answer
63 views

how can i comment out/in a section in source code with sed or awk

Currently i have files, which contain sections like this code statement1 code statement2 # BEGIN SOMENAME some code some other code # END SOMENAME code statement n +1 code statement n +1 What i ...
1
vote
2answers
139 views

Compare columns from two CSVs and merge on matches

I have two CSV files and am trying to merge them based on the first column in the first file matching the third column in the second file. They lines are not sorted. file1.csv: android,1,2 osx,2,5 ...
1
vote
3answers
78 views

Split a file based on required string

I have test file with the below contents: CREATE TABLE XYZ ( ABC INT, CDE CHAR(10), . . . ); Similarly, REPLACE VIEW XYZ ( ABC, CDE, EFG ); CREATE TABLE XYZ_1 ( ABC INT, CDE CHAR(10), . . . ); ...
0
votes
1answer
38 views

How to paste over column by separate matching column

I have these two files: file1.txt 0 rs3094315 0 0 C T 0 rs12562034 0 0 A G 0 rs3934834 0 0 C T file2.txt Chr,Pos,snp_name 1, 742429,rs3094315 Where the ...
1
vote
1answer
110 views

cut column range sed/awk/cat

I'd like to cut a certain range of text (i.e. characters 17-63) repeatedly from a *.txt file, which is several thousand lines long. One line looks like this: <script>addRow("...
1
vote
1answer
108 views

How to write a script to log in multiple server using for loop and ssh?

I want to write a script using a for loop and ssh, to log in several server. After logging in using awk command I want to get 7th column printed as output. I tried the script below but couldn't ...
5
votes
2answers
89 views

How to find one word and sequentially replace it?

I am running AIX 5.3(not by choice, but I cannot change it), and I have a text file: servers.txt Here is an example of the file's contents: apple port1 username password IPAddress TCP banana port2 ...
0
votes
3answers
83 views

How to find specific csv files and extract their second row into a single csv file

I'm trying to find out how to filter specific csv files by name and then extract their second row into a single csv file. I'm not sure whether it is a good idea to find files and then pipe it with ...
3
votes
3answers
141 views

Retrieving lines from a file depending on other lines

Imagine the following file structure: foo.bar.1 blabla moreblabla relevant=yes foo.bar.2 relevant=no foo.bar.3 blablabla foo.bar.4 relevant=yes I want to retrieve all foo.bar lines where within the ...
0
votes
3answers
111 views

How to group a bunch of rows based on information from the first row?

I have a data file looks like : 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 3 3 3 3 3 . . . 1 0 4 4 3 1 2 0 0 0 3 1 1 2 1 1 1 1 1 1 0 1 1 3 . . . 0 0 0 0 0 0 0 3 3 1 1 2 3 2 1 2 2 3 1 2 3 1 2 2 . . . ...
0
votes
2answers
137 views

How to cut every line from a particular string?

input.txt: john_Apple01_xyz_1 john_Fruit_Apple01_abc_c1 john_Apple21_trs_1 john_Fruit_Apple21_efg_1 john_Fruit_Apple21_tefg_1 Desired output: Apple01_xyz_1 Apple01_abc_c1 Apple21_trs_1 ...
1
vote
2answers
79 views

Add columns to a text file with set spacing in Bash

I am setting up a malware zone file in DNS which will point all hosts to a specific IP. The file is simply a list of hostnames. This is what I have now: googlle.in ...
0
votes
2answers
124 views

how to subtract first value from the last one within a column across a row in shell?

I have a data file looks like : 1 1 1 1 1 1 1 1 2 2 2 2 3 3 3 3 3 3 2 4 5 8 9 10 13 17 19 29 30 32 33 50 700 800 900 950 first I want to insert space among each 3 identical values keeping each ...
8
votes
7answers
596 views

Convert rows to columns

I have a file that includes details about VMs running in a hypervisor. We run some command and redirect the output to a file. And the is data available in the below format. Virtual Machine : OL6U5 ...
1
vote
2answers
48 views

awk alter the next line after a pattern match based on another pattern match

I have a file with lines such as: .... pattern1 100 200 300 pattern2 300 400 400 pattern1 300 900 700 pattern1 200 500 900 ... As shown in the above example, there are some lines where pattern2 ...
0
votes
4answers
46 views

Remove second line where first few columns identical in large file

I have a file that looks something like this: A 1 abc A 1 def A 2 ttt B 2 ppp B 2 qqq I'd like to keep the first line when the first two keys are identical, so something like this: A 1 abc A 2 ttt ...
1
vote
2answers
45 views

Using getline with NR in awk

I have a bash script which calls a awk script to print each line from the file name_list.txt #!/bin/bash awk -f parse_list.awk name_list.txt In the awk script, I am using getline with NR to print ...
1
vote
1answer
45 views

if condition in bash scripting

I have a text file that I want to check some line as below com1=`awk 'NF{a=b;b=$0} END{print (b ~ /\com/)? "Yes" : "No"}' $myfile` com2=`awk 'NF{a=b;b=$0} END{print (b ~ /\t/)? "Yes" : "No"}'...
0
votes
2answers
62 views

How can I add new character after each line?

I am trying to manipulate a file, by adding an A after each lines Name Test However the awk '{print $0"A"}' file_list.txt > file_list2.txt command gives the following result Name A Test A ...
1
vote
3answers
77 views

How to join two files by matching a column with an irregular number of columns?

file1: 0000002|SLM DEV CORP |PO 857 0000003|S TOPPING |APT 19 0000004|JD BROS LTD ...
0
votes
1answer
24 views

Insert Line in File When Group of Lines Changes

Let's say I have a text file like this: group1 hello group1 there group1 nice line group2 here group2 we group2 go What I want to do is insert a line when the group of lines changes, something like:...
0
votes
1answer
38 views

awk: arithmetic does not work on a particular field

I have several lines such as: N2O + M = N2 + O + M 1.300E+11 0.00 59620.0 I need to substitute the third from last field with a factor of itself which does not work ...
3
votes
2answers
209 views

AWK combine variable and regular expression in pattern match

I am looking to search a file for a column that is 3 letters followed by any 3 or 4 numerals. eg. if ( $1 ~ /^[A-Z][A-Z][A-Z][0-9][0-9][0-9]/) However I need the 3 letters to be a variable, so I ...
1
vote
0answers
223 views

How to redirect linux command outputs to specific row and column in csv?

I'm working on Unix Build post documentation process, I have a CSV file to be filled in and uploaded for every new build, to ensure the servers are build as per the original request. The CSV file ...
1
vote
3answers
57 views

Merging 2 files and keeping the one duplicate

I want to create a file that contains columns from two input files. File1 is like: 11111111 abc12345 Y 22222222 xyz23456 Y File2: 11111111 abc12345 33333333 ...
0
votes
2answers
225 views

Grep / awk on multiple files to single output

I have several txt-files containing data, where i use grep to search for a current string of text, and use awk to filter out the variable i need. The string is repeated through the file, so i ...
1
vote
2answers
34 views

Compare 2nd column of the 1st file and 1st column of the 2nd file and print the matched records of the 1st file

I have two files large files: f1.txt: 5020118359 |13ZJ24001218 |20141224|R 5020120475 |13ZJ38000813 |20141204|R 5020127431 |13ZJ38001569 ...
1
vote
3answers
55 views

vlookup work in awk on linux

Hi I have two files containing data: file1: 4,abc 3,xyz 5,fut 6,gfd file2: gfd,2.3 xyz,4.5 abc,6.7 fut,3.2 i wish to create a file3 as output file using linux scripting commands: 4,6.7 3,4.5 5,...
1
vote
0answers
63 views

Is there a reference for the assembler-like output produced by `awk -W dump`?

The command awk -W dump ... produces a dump of "assembler-like" code (as shown here [link], or by reading the man). Some of the instructions are easy to guess at their meaning such as jz, which is ...
1
vote
2answers
48 views

Copy plaintext lines from file and append them to same file with mid-line prefix or remove mid-line prefix

Making an ugly script that compiles a bunch of hosts files into one big list. Most lists will have two entries for a URL, one with www. and one without like this: 127.0.0.1 facebook.com 127.0.0.1 ...