awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
0answers
7 views

How to change the first column of data in a file?

I have .data file containing a table of numbers with the column format: m x y z vx vy vz separated by tabs, with a depth of 2000. The file name is initc.data. I want to change ...
0
votes
0answers
10 views

Collecting network information

I need to collect information about the network usage of each process. Nethogs presents the data I need in real time, I am trying to save the output to a file in order parse it and plot the data. ...
0
votes
4answers
25 views

Print only unique lines from file not the duplicates

I have a sorted words list, line by line file like so: apple apple grapes lime orange orange pear pear peach strawberry strawberry I only want to print out unique lines and drop duplicates: grapes ...
3
votes
5answers
175 views

Command line method to find repeat-word typos, with line numbers

Updated: Clarify line number requirement, some verbosity reductions From the command line, is there a way to: check a file of English text to find repeat-word typos, along with line numbers where ...
5
votes
1answer
50 views

Trap Ctrl-C in awk script

I believe Ctrl-C can be trapped in bash scripts. Is it also possible to trap it inside an awk script in order to handle that event? E.g. for aborting processing, but printing the results of what has ...
-2
votes
1answer
27 views

Read columns from file and substitute values in another file using bash script [on hold]

I have following file : input.txt b73_chr10 w22_chr9 w22_chr7 w22_chr10 w22_chr8 w22_chr8 I have written the following code(given below) to read the first and second column and substitute ...
1
vote
1answer
29 views

Syntax error removing duplicate lines via awk '!x[$0]++' file

The file contains cat file a b c b d Trying to remove duplicate lines on SunOS via awk '!x[$0]++' file (as found in another postings) results in a syntax error awk: syntax error near line 1 ...
0
votes
2answers
25 views

Joining certain lines of text together before modifying them

I have a cricket commentary like - 0.3 Richardson to RG Sharma, 2 runs, gentle, non-swinging length ball on off, Rohit seeks to pick it up. Hit high on the bat and spooned over the returning Mitch ...
3
votes
3answers
186 views

Select the lines with exactly two columns in Linux

I have a file that is a mapping between filename and the corresponding transcripts. The file name and transcripts is separated by a TAB character. The transcription of the file may contain one or ...
1
vote
1answer
34 views

How do you list number of lines of every file in a directory in human readable format.

I have a list of directories and subdirectories that contain large csv files. There are about 500 million lines in these files, each is a record. I would like to know How many lines are in each ...
0
votes
2answers
45 views

Need help in data arrangement

Could you please help me to sort this out, I have data which i would like get output in below format using shell/perl script. Data: Virtual Machine Test status Running. Assigned Server ...
1
vote
1answer
25 views

how can i comment out/in a section in source code with sed or awk

Currently i have files, which contain sections like this code statement1 code statement2 # BEGIN SOMENAME some code some other code # END SOMENAME code statement n +1 code statement n +1 What i ...
1
vote
6answers
20k views

How can I output the difference between 2 files?

My file consists of the the following; roughly: username:username:username:username:username The above line continues to about 600 characters. I use the awk command in order to use it as an ...
0
votes
1answer
19 views

delete line that contains a case insensitive match

I have a file that contains information as so: 20 BaDDOg 31 baddog 42 badCAT 43 goodDoG 44 GOODcAT and I want to delete all lines that contain the word dog. This is my desired ...
1
vote
3answers
72 views

Split a file based on required string

I have test file with the below contents: CREATE TABLE XYZ ( ABC INT, CDE CHAR(10), . . . ); Similarly, REPLACE VIEW XYZ ( ABC, CDE, EFG ); CREATE TABLE XYZ_1 ( ABC INT, CDE CHAR(10), . . . ); ...
2
votes
3answers
195 views

awk : parse and write to another file

I have records in XML file like below. I need to search for <keyword>SEARCH</keyword> and if present then I need to take the entire record and write to another file.(starting from ...
1
vote
5answers
236 views

Grep value from json file

my json file looks something like this [{"product":"Apple","id":"2134"},{"product":"Mango","id":"4567"}] I want to grep/sed/awk 'id' for given 'product'. Output: Enter product : Apple Your ...
1
vote
1answer
35 views

Linux + capture only the IP's from mount command

please advise how I can to print only the list of the IP's ( in the end of the lines ) , from mount output mount netapp64:/lunggg/OracleTeam/OracleCDs on /oracds type nfs ...
0
votes
1answer
43 views

Add column to a CSV file as a function of other two columns, one from other file

Here is the situation: a.csv Contains values from some sensors captured every 1 second, the last column is the epoch (they actually looks like 1453998477.044).(*s are the sensors values). Example: ...
1
vote
2answers
48 views

awk is putting last column in a new line

I have this -77.199997 23.6343749630 39.9989584580 -80.000000 23.6364583430 39.9989584580 -81.599998 23.6385417220 39.9989584580 I want to reorder the columns so it reads column 2, ...
1
vote
1answer
34 views

splitting file based on values in specific column

I have a file that I would like to break up into multiple files with uniq values for the first column. For example, here is a file: fileA.txt 1 Cat 1 Dog 1 Frog 2 Boy 2 Girl 3 ...
2
votes
1answer
19 views

grep awk or sed a CSV Row containing a certain part of a web address

I am trying to clean up a Squid log file and I want to delete rows with a website containing "/0/" in the 11th column. An example: Row1: column1, column2, column3...column10, ...
0
votes
1answer
38 views

Split string into array and print each element on a new line with commandline

I'm having a String which is seperated by commas like a,b,c,d,e,f that I want to split into an array with the comma as seperator. Then I want to print each element on a new line. The problem I'm ...
5
votes
2answers
65 views

How to find one word and sequentially replace it?

I am running AIX 5.3(not by choice, but I cannot change it), and I have a text file: servers.txt Here is an example of the file's contents: apple port1 username password IPAddress TCP banana port2 ...
1
vote
2answers
74 views

Compare columns from two CSVs and merge on matches

I have two CSV files and am trying to merge them based on the first column in the first file matching the third column in the second file. They lines are not sorted. file1.csv: android,1,2 osx,2,5 ...
0
votes
1answer
34 views

How to paste over column by separate matching column

I have these two files: file1.txt 0 rs3094315 0 0 C T 0 rs12562034 0 0 A G 0 rs3934834 0 0 C T file2.txt Chr,Pos,snp_name 1, 742429,rs3094315 Where the ...
1
vote
1answer
37 views

cut column range sed/awk/cat

I'd like to cut a certain range of text (i.e. characters 17-63) repeatedly from a *.txt file, which is several thousand lines long. One line looks like this: ...
3
votes
3answers
135 views

Retrieving lines from a file depending on other lines

Imagine the following file structure: foo.bar.1 blabla moreblabla relevant=yes foo.bar.2 relevant=no foo.bar.3 blablabla foo.bar.4 relevant=yes I want to retrieve all foo.bar lines where within the ...
7
votes
7answers
353 views

Convert rows to columns

I have a file that includes details about VMs running in a hypervisor. We run some command and redirect the output to a file. And the is data available in the below format. Virtual Machine : OL6U5 ...
1
vote
1answer
45 views

How to write a script to log in multiple server using for loop and ssh?

I want to write a script using a for loop and ssh, to log in several server. After logging in using awk command I want to get 7th column printed as output. I tried the script below but couldn't ...
0
votes
3answers
39 views

How to find specific csv files and extract their second row into a single csv file

I'm trying to find out how to filter specific csv files by name and then extract their second row into a single csv file. I'm not sure whether it is a good idea to find files and then pipe it with ...
-1
votes
0answers
41 views

dump unstructured content of file to a csv or excel sheet [duplicate]

I believe it is not duplicate of other thread which we discussed in past. The reason being is: Here we have some rows (Virtual Disk, Physical Disk, device and shareddisk) which are repeating several ...
0
votes
2answers
1k views

Sum two columns from different files

I have two files like this: File1: -54.005 -67.405 0.1 -54.05 -68.7833 0.2 -54.05 -68.7333 0.3 File2: -54.005 -67.4050 70 12.7 -54.05 -68.7833 167 12.5 -54.05 -68.7333 205 12.6 ...
0
votes
3answers
75 views

How to group a bunch of rows based on information from the first row?

I have a data file looks like : 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 3 3 3 3 3 . . . 1 0 4 4 3 1 2 0 0 0 3 1 1 2 1 1 1 1 1 1 0 1 1 3 . . . 0 0 0 0 0 0 0 3 3 1 1 2 3 2 1 2 2 3 1 2 3 1 2 2 . . . ...
0
votes
2answers
67 views

How to cut every line from a particular string?

input.txt: john_Apple01_xyz_1 john_Fruit_Apple01_abc_c1 john_Apple21_trs_1 john_Fruit_Apple21_efg_1 john_Fruit_Apple21_tefg_1 Desired output: Apple01_xyz_1 Apple01_abc_c1 Apple21_trs_1 ...
0
votes
2answers
77 views

how to subtract first value from the last one within a column across a row in shell?

I have a data file looks like : 1 1 1 1 1 1 1 1 2 2 2 2 3 3 3 3 3 3 2 4 5 8 9 10 13 17 19 29 30 32 33 50 700 800 900 950 first I want to insert space among each 3 identical values keeping each ...
1
vote
2answers
47 views

Add columns to a text file with set spacing in Bash

I am setting up a malware zone file in DNS which will point all hosts to a specific IP. The file is simply a list of hostnames. This is what I have now: googlle.in ...
0
votes
4answers
33 views

Remove second line where first few columns identical in large file

I have a file that looks something like this: A 1 abc A 1 def A 2 ttt B 2 ppp B 2 qqq I'd like to keep the first line when the first two keys are identical, so something like this: A 1 abc A 2 ttt ...
0
votes
0answers
74 views

How to redirect linux command outputs to specific row and column in csv?

I'm working on Unix Build post documentation process, I have a CSV file to be filled in and uploaded for every new build, to ensure the servers are build as per the original request. The CSV file ...
1
vote
3answers
38 views

Transpose paramter values from a file and Generate parameter files from it

There's a file containing a table with the parameter names as the first line, values in the following lines: How to perform the following using awk, sed or any shell scripting utility: generate a ...
-1
votes
0answers
27 views

dump non formated output of commands to csv file

I have some commands with which i can get list of VMs running on hypervisors. I redirect that output to some file.txt and use while loop to read each line from file.txt and execute two more commands ...
1
vote
2answers
31 views

awk alter the next line after a pattern match based on another pattern match

I have a file with lines such as: .... pattern1 100 200 300 pattern2 300 400 400 pattern1 300 900 700 pattern1 200 500 900 ... As shown in the above example, there are some lines where pattern2 ...
1
vote
2answers
30 views

Using getline with NR in awk

I have a bash script which calls a awk script to print each line from the file name_list.txt #!/bin/bash awk -f parse_list.awk name_list.txt In the awk script, I am using getline with NR to print ...
1
vote
1answer
40 views

if condition in bash scripting

I have a text file that I want to check some line as below com1=`awk 'NF{a=b;b=$0} END{print (b ~ /\com/)? "Yes" : "No"}' $myfile` com2=`awk 'NF{a=b;b=$0} END{print (b ~ /\t/)? "Yes" : ...
0
votes
2answers
48 views

How can I add new character after each line?

I am trying to manipulate a file, by adding an A after each lines Name Test However the awk '{print $0"A"}' file_list.txt > file_list2.txt command gives the following result Name A Test A ...
3
votes
2answers
73 views

AWK combine variable and regular expression in pattern match

I am looking to search a file for a column that is 3 letters followed by any 3 or 4 numerals. eg. if ( $1 ~ /^[A-Z][A-Z][A-Z][0-9][0-9][0-9]/) However I need the 3 letters to be a variable, so I ...
2
votes
4answers
7k views

Match multiple regular expressions from a single file using awk

I'm trying to parse a HTML file using shell scripting. There are 4 different regular expressions that i need to catch: name= , age= , class= , marks=. Using grep "name=\|age=\|class=\|marks=" ...
0
votes
1answer
18 views

Insert Line in File When Group of Lines Changes

Let's say I have a text file like this: group1 hello group1 there group1 nice line group2 here group2 we group2 go What I want to do is insert a line when the group of lines changes, something ...
1
vote
3answers
36 views

How to join two files by matching a column with an irregular number of columns?

file1: 0000002|SLM DEV CORP |PO 857 0000003|S TOPPING |APT 19 0000004|JD BROS LTD ...
0
votes
2answers
762 views

put rows of numbers into a column with shell script

I have a file with many lines, like 1 jfkdajfd 1 2 3 5 2 fkldfjld 3 fdkfloaf 9 10 4 fldfldkf 5 fdskf;ak 12 1 4 I want to get all the numbers and put them in a column in a file, like 1 ...