awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
0answers
15 views

Loop through directory and sum value next to specific pattern producing an average

I want to loop through all the files in the directory. the files are set out like this: <Overall>4 other data <Overall>2 other data ...... I have the code: for file in .dat; do awk ...
1
vote
3answers
79 views

Parse file with Awk only when first row matches pattern

I need to interrogate the header of a CSV file, and if a column exists proceed with the data rows. Context is when the data contains columns depending on when and what emitted it. Hoping for a "pure" ...
2
votes
4answers
107 views

Sum value next to specific pattern

I have a file with multiple tags with a number next to it e.g. <Overall>4 other <tags> and data <Overall>2 other <tags> and data <Overall>3 How would I search through ...
0
votes
1answer
28 views

How to add a condition within a for loop during substitution if string is null

I am trying to add a condition in this code, that if there is null string in translation file for either string or repl[string] for example, my file input_chk.txt has following details: input_chk.txt ...
1
vote
1answer
27 views

insert string for missing value

I've run into some problems with editing a text file I've received due to some missing values in a specific column for some rows. For example, here is what the file looks like (mostly): ID10 dad ...
2
votes
2answers
51 views

Reformat date to unix time stamp in csv table

I have a .csv file containing the date and time in the format 01/20/2016 23:53:01 in the first column. My columns are separated by semicolons, i.e. 01/21/2016 03:03:01;18616;0;1 01/21/2016 ...
3
votes
2answers
56 views

How can I do a binary search of a text file for a specific character string?

I have a text file that contains a large number of records, each on a single line. Some of the records have special characters that have been corrupted, and I'm attempting to find those by looking for ...
0
votes
1answer
60 views

replace long text string (script with MANY special characters). sed, awk, grep

My site was hacked / infected. I replaced the url of the malicious link, but other elements in the malicious script are still making my site get blocked. Without inserting a hundred or so "escapes", ...
2
votes
1answer
44 views

Regex matching on awk in command line

I'm trying to get the filename from unrar command verbose and deleting them all through command line. The unrar command output is similar to this: Extracting BucketSort.cpp ...
1
vote
2answers
72 views

How to replace multiple blank lines with a specific non-blank line?

The sample input is 1 2 3 4 5 6 7 8 9 The expected output is 1 2 3 --- 4 5 --- 6 7 8 --- 9
0
votes
3answers
134 views

Filtering content from df output

I have the command below to get the filesystem usage on a server: df -k | awk '{print $5 " " $6}' | grep % When executed, the following are the results: 97% / 0% /proc 0% /etc/mnttab 0% /dev/fd ...
3
votes
4answers
72 views

display lines with not more than 4 words in the second column

I have a symbol table of the form M07UP49A0870I422.wav <s> haraa keelaa <bn> </s> M07UP49A0870I423.wav <s> <horn> jau <babble> </s> ...
0
votes
1answer
29 views

Delete Non numeric row from a csv

I have used sed to allow only numeric and csv format ,I have used sed '/[^0-9, -, .]/d' as regex. It has still "_","1-0","1-1" character as o/p which worries me. sed 's/'"$(printf '\015')"'//g' ...
0
votes
1answer
42 views

Convert a column of epochs into a time column starting from 0

What I have Hi, imagine a lot of this files, where first column is epoch, and the other(s) column(s) are some data: 1000333,34,1 1001456,56,0 1005356,34,2 What I need I need to transform them ...
0
votes
3answers
38 views

List out the original IP Address of the host

I have following content in the file. I would like to print the IP address which is Under Name. Server: 17.128.100.12 Address: 17.128.100.12#53 Name: Testserver1 Address: 17.146.118.156 ...
3
votes
1answer
99 views

Print some lines before and after pattern match excluding lines matching another patterns

I use awk on Solaris to print 5 lines before and after a pattern match. Currently, the following one-liner does the job: /usr/xpg4/bin/awk 'c-->0;$0~s{if(b)for(c=b+1;c>1;c--)print ...
3
votes
1answer
37 views

grep output of cdrecord

I would like to parse the output of cdrecord/wodim using grep. cdrecord -v outputs lines like Track 01: 49 of 75 MB written (fifo 100%) [buf 100%] 10.2x. The problem is that cdrecord uses ...
1
vote
1answer
75 views

How to subtract rows from first row using awk?

My data file looks like: 10 -0.314690785295 20 -0.251967909317 30 -0.215271387106 40 -0.189228416217 The desired output is: 10 -0.0627229 20 -0.0994193 30 -0.1254623 I am really new in bash, so ...
0
votes
2answers
36 views

How to change the first column of data in a file?

I have .data file containing a table of numbers with the column format: m x y z vx vy vz separated by tabs, with a depth of 2000. The file name is initc.data. I want to change ...
0
votes
2answers
50 views

Collecting network information

I need to collect information about the network usage of each process. Nethogs presents the data I need in real time, I am trying to save the output to a file in order parse it and plot the data. ...
0
votes
4answers
54 views

Print only unique lines from file not the duplicates

I have a sorted words list, line by line file like so: apple apple grapes lime orange orange pear pear peach strawberry strawberry I only want to print out unique lines and drop duplicates: grapes ...
8
votes
1answer
138 views

Trap Ctrl-C in awk script

I believe Ctrl-C can be trapped in bash scripts. Is it also possible to trap it inside an Awk script in order to handle that event? For example, for aborting processing, but printing the results of ...
3
votes
3answers
254 views

Select the lines with exactly two columns in Linux

I have a file that is a mapping between filename and the corresponding transcripts. The file name and transcripts is separated by a TAB character. The transcription of the file may contain one or ...
0
votes
2answers
30 views

Joining certain lines of text together before modifying them

I have a cricket commentary like - 0.3 Richardson to RG Sharma, 2 runs, gentle, non-swinging length ball on off, Rohit seeks to pick it up. Hit high on the bat and spooned over the returning Mitch ...
1
vote
1answer
95 views

How do you list number of lines of every file in a directory in human readable format.

I have a list of directories and subdirectories that contain large csv files. There are about 500 million lines in these files, each is a record. I would like to know How many lines are in each ...
1
vote
1answer
41 views

Syntax error removing duplicate lines via awk '!x[$0]++' file

The file contains cat file a b c b d Trying to remove duplicate lines on SunOS via awk '!x[$0]++' file (as found in another postings) results in a syntax error awk: syntax error near line 1 ...
0
votes
2answers
50 views

Need help in data arrangement

Could you please help me to sort this out, I have data which i would like get output in below format using shell/perl script. Data: Virtual Machine Test status Running. Assigned Server ...
0
votes
1answer
42 views

delete line that contains a case insensitive match

I have a file that contains information as so: 20 BaDDOg 31 baddog 42 badCAT 43 goodDoG 44 GOODcAT and I want to delete all lines that contain the word dog. This is my desired ...
2
votes
3answers
269 views

awk : parse and write to another file

I have records in XML file like below. I need to search for <keyword>SEARCH</keyword> and if present then I need to take the entire record and write to another file.(starting from ...
1
vote
1answer
42 views

Linux + capture only the IP's from mount command

please advise how I can to print only the list of the IP's ( in the end of the lines ) , from mount output mount netapp64:/lunggg/OracleTeam/OracleCDs on /oracds type nfs ...
0
votes
1answer
67 views

Add column to a CSV file as a function of other two columns, one from other file

Here is the situation: a.csv Contains values from some sensors captured every 1 second, the last column is the epoch (they actually looks like 1453998477.044).(*s are the sensors values). Example: ...
1
vote
5answers
627 views

Grep value from json file

my json file looks something like this [{"product":"Apple","id":"2134"},{"product":"Mango","id":"4567"}] I want to grep/sed/awk 'id' for given 'product'. Output: Enter product : Apple Your ...
2
votes
1answer
26 views

grep awk or sed a CSV Row containing a certain part of a web address

I am trying to clean up a Squid log file and I want to delete rows with a website containing "/0/" in the 11th column. An example: Row1: column1, column2, column3...column10, ...
1
vote
2answers
54 views

awk is putting last column in a new line

I have this -77.199997 23.6343749630 39.9989584580 -80.000000 23.6364583430 39.9989584580 -81.599998 23.6385417220 39.9989584580 I want to reorder the columns so it reads column 2, ...
0
votes
1answer
89 views

Split string into array and print each element on a new line with commandline

I'm having a String which is seperated by commas like a,b,c,d,e,f that I want to split into an array with the comma as seperator. Then I want to print each element on a new line. The problem I'm ...
1
vote
1answer
48 views

splitting file based on values in specific column

I have a file that I would like to break up into multiple files with uniq values for the first column. For example, here is a file: fileA.txt 1 Cat 1 Dog 1 Frog 2 Boy 2 Girl 3 ...
1
vote
1answer
44 views

how can i comment out/in a section in source code with sed or awk

Currently i have files, which contain sections like this code statement1 code statement2 # BEGIN SOMENAME some code some other code # END SOMENAME code statement n +1 code statement n +1 What i ...
1
vote
2answers
108 views

Compare columns from two CSVs and merge on matches

I have two CSV files and am trying to merge them based on the first column in the first file matching the third column in the second file. They lines are not sorted. file1.csv: android,1,2 osx,2,5 ...
1
vote
3answers
77 views

Split a file based on required string

I have test file with the below contents: CREATE TABLE XYZ ( ABC INT, CDE CHAR(10), . . . ); Similarly, REPLACE VIEW XYZ ( ABC, CDE, EFG ); CREATE TABLE XYZ_1 ( ABC INT, CDE CHAR(10), . . . ); ...
0
votes
1answer
36 views

How to paste over column by separate matching column

I have these two files: file1.txt 0 rs3094315 0 0 C T 0 rs12562034 0 0 A G 0 rs3934834 0 0 C T file2.txt Chr,Pos,snp_name 1, 742429,rs3094315 Where the ...
1
vote
1answer
67 views

cut column range sed/awk/cat

I'd like to cut a certain range of text (i.e. characters 17-63) repeatedly from a *.txt file, which is several thousand lines long. One line looks like this: ...
1
vote
1answer
86 views

How to write a script to log in multiple server using for loop and ssh?

I want to write a script using a for loop and ssh, to log in several server. After logging in using awk command I want to get 7th column printed as output. I tried the script below but couldn't ...
5
votes
2answers
83 views

How to find one word and sequentially replace it?

I am running AIX 5.3(not by choice, but I cannot change it), and I have a text file: servers.txt Here is an example of the file's contents: apple port1 username password IPAddress TCP banana port2 ...
0
votes
3answers
71 views

How to find specific csv files and extract their second row into a single csv file

I'm trying to find out how to filter specific csv files by name and then extract their second row into a single csv file. I'm not sure whether it is a good idea to find files and then pipe it with ...
3
votes
3answers
138 views

Retrieving lines from a file depending on other lines

Imagine the following file structure: foo.bar.1 blabla moreblabla relevant=yes foo.bar.2 relevant=no foo.bar.3 blablabla foo.bar.4 relevant=yes I want to retrieve all foo.bar lines where within the ...
0
votes
3answers
96 views

How to group a bunch of rows based on information from the first row?

I have a data file looks like : 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 3 3 3 3 3 . . . 1 0 4 4 3 1 2 0 0 0 3 1 1 2 1 1 1 1 1 1 0 1 1 3 . . . 0 0 0 0 0 0 0 3 3 1 1 2 3 2 1 2 2 3 1 2 3 1 2 2 . . . ...
0
votes
2answers
109 views

How to cut every line from a particular string?

input.txt: john_Apple01_xyz_1 john_Fruit_Apple01_abc_c1 john_Apple21_trs_1 john_Fruit_Apple21_efg_1 john_Fruit_Apple21_tefg_1 Desired output: Apple01_xyz_1 Apple01_abc_c1 Apple21_trs_1 ...
1
vote
2answers
65 views

Add columns to a text file with set spacing in Bash

I am setting up a malware zone file in DNS which will point all hosts to a specific IP. The file is simply a list of hostnames. This is what I have now: googlle.in ...
0
votes
2answers
112 views

how to subtract first value from the last one within a column across a row in shell?

I have a data file looks like : 1 1 1 1 1 1 1 1 2 2 2 2 3 3 3 3 3 3 2 4 5 8 9 10 13 17 19 29 30 32 33 50 700 800 900 950 first I want to insert space among each 3 identical values keeping each ...
8
votes
7answers
455 views

Convert rows to columns

I have a file that includes details about VMs running in a hypervisor. We run some command and redirect the output to a file. And the is data available in the below format. Virtual Machine : OL6U5 ...