awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
1answer
89 views

How to write a script to log in multiple server using for loop and ssh?

I want to write a script using a for loop and ssh, to log in several server. After logging in using awk command I want to get 7th column printed as output. I tried the script below but couldn't ...
5
votes
2answers
84 views

How to find one word and sequentially replace it?

I am running AIX 5.3(not by choice, but I cannot change it), and I have a text file: servers.txt Here is an example of the file's contents: apple port1 username password IPAddress TCP banana port2 ...
0
votes
3answers
71 views

How to find specific csv files and extract their second row into a single csv file

I'm trying to find out how to filter specific csv files by name and then extract their second row into a single csv file. I'm not sure whether it is a good idea to find files and then pipe it with ...
3
votes
3answers
138 views

Retrieving lines from a file depending on other lines

Imagine the following file structure: foo.bar.1 blabla moreblabla relevant=yes foo.bar.2 relevant=no foo.bar.3 blablabla foo.bar.4 relevant=yes I want to retrieve all foo.bar lines where within the ...
0
votes
3answers
98 views

How to group a bunch of rows based on information from the first row?

I have a data file looks like : 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 3 3 3 3 3 . . . 1 0 4 4 3 1 2 0 0 0 3 1 1 2 1 1 1 1 1 1 0 1 1 3 . . . 0 0 0 0 0 0 0 3 3 1 1 2 3 2 1 2 2 3 1 2 3 1 2 2 . . . ...
0
votes
2answers
110 views

How to cut every line from a particular string?

input.txt: john_Apple01_xyz_1 john_Fruit_Apple01_abc_c1 john_Apple21_trs_1 john_Fruit_Apple21_efg_1 john_Fruit_Apple21_tefg_1 Desired output: Apple01_xyz_1 Apple01_abc_c1 Apple21_trs_1 ...
1
vote
2answers
65 views

Add columns to a text file with set spacing in Bash

I am setting up a malware zone file in DNS which will point all hosts to a specific IP. The file is simply a list of hostnames. This is what I have now: googlle.in ...
0
votes
2answers
113 views

how to subtract first value from the last one within a column across a row in shell?

I have a data file looks like : 1 1 1 1 1 1 1 1 2 2 2 2 3 3 3 3 3 3 2 4 5 8 9 10 13 17 19 29 30 32 33 50 700 800 900 950 first I want to insert space among each 3 identical values keeping each ...
8
votes
7answers
461 views

Convert rows to columns

I have a file that includes details about VMs running in a hypervisor. We run some command and redirect the output to a file. And the is data available in the below format. Virtual Machine : OL6U5 ...
1
vote
2answers
39 views

awk alter the next line after a pattern match based on another pattern match

I have a file with lines such as: .... pattern1 100 200 300 pattern2 300 400 400 pattern1 300 900 700 pattern1 200 500 900 ... As shown in the above example, there are some lines where pattern2 ...
0
votes
4answers
44 views

Remove second line where first few columns identical in large file

I have a file that looks something like this: A 1 abc A 1 def A 2 ttt B 2 ppp B 2 qqq I'd like to keep the first line when the first two keys are identical, so something like this: A 1 abc A 2 ttt ...
1
vote
2answers
39 views

Using getline with NR in awk

I have a bash script which calls a awk script to print each line from the file name_list.txt #!/bin/bash awk -f parse_list.awk name_list.txt In the awk script, I am using getline with NR to print ...
1
vote
1answer
44 views

if condition in bash scripting

I have a text file that I want to check some line as below com1=`awk 'NF{a=b;b=$0} END{print (b ~ /\com/)? "Yes" : "No"}' $myfile` com2=`awk 'NF{a=b;b=$0} END{print (b ~ /\t/)? "Yes" : ...
0
votes
2answers
51 views

How can I add new character after each line?

I am trying to manipulate a file, by adding an A after each lines Name Test However the awk '{print $0"A"}' file_list.txt > file_list2.txt command gives the following result Name A Test A ...
1
vote
3answers
62 views

How to join two files by matching a column with an irregular number of columns?

file1: 0000002|SLM DEV CORP |PO 857 0000003|S TOPPING |APT 19 0000004|JD BROS LTD ...
0
votes
1answer
24 views

Insert Line in File When Group of Lines Changes

Let's say I have a text file like this: group1 hello group1 there group1 nice line group2 here group2 we group2 go What I want to do is insert a line when the group of lines changes, something ...
0
votes
1answer
36 views

awk: arithmetic does not work on a particular field

I have several lines such as: N2O + M = N2 + O + M 1.300E+11 0.00 59620.0 I need to substitute the third from last field with a factor of itself which does not work ...
3
votes
2answers
124 views

AWK combine variable and regular expression in pattern match

I am looking to search a file for a column that is 3 letters followed by any 3 or 4 numerals. eg. if ( $1 ~ /^[A-Z][A-Z][A-Z][0-9][0-9][0-9]/) However I need the 3 letters to be a variable, so I ...
1
vote
0answers
178 views

How to redirect linux command outputs to specific row and column in csv?

I'm working on Unix Build post documentation process, I have a CSV file to be filled in and uploaded for every new build, to ensure the servers are build as per the original request. The CSV file ...
1
vote
3answers
56 views

Merging 2 files and keeping the one duplicate

I want to create a file that contains columns from two input files. File1 is like: 11111111 abc12345 Y 22222222 xyz23456 Y File2: 11111111 abc12345 33333333 ...
0
votes
2answers
172 views

Grep / awk on multiple files to single output

I have several txt-files containing data, where i use grep to search for a current string of text, and use awk to filter out the variable i need. The string is repeated through the file, so i ...
1
vote
2answers
28 views

Compare 2nd column of the 1st file and 1st column of the 2nd file and print the matched records of the 1st file

I have two files large files: f1.txt: 5020118359 |13ZJ24001218 |20141224|R 5020120475 |13ZJ38000813 |20141204|R 5020127431 |13ZJ38001569 ...
1
vote
3answers
46 views

vlookup work in awk on linux

Hi I have two files containing data: file1: 4,abc 3,xyz 5,fut 6,gfd file2: gfd,2.3 xyz,4.5 abc,6.7 fut,3.2 i wish to create a file3 as output file using linux scripting commands: 4,6.7 3,4.5 ...
1
vote
0answers
61 views

Is there a reference for the assembler-like output produced by `awk -W dump`?

The command awk -W dump ... produces a dump of "assembler-like" code (as shown here [link], or by reading the man). Some of the instructions are easy to guess at their meaning such as jz, which is ...
1
vote
2answers
39 views

Copy plaintext lines from file and append them to same file with mid-line prefix or remove mid-line prefix

Making an ugly script that compiles a bunch of hosts files into one big list. Most lists will have two entries for a URL, one with www. and one without like this: 127.0.0.1 facebook.com 127.0.0.1 ...
1
vote
5answers
269 views

To remove extra space from line using shell command

This is my file, TLRUIDA CBdms Status DP 6/1/1 DC 6/1/5 0 Y TLRUIDA CBdms Status DP 6/2/1 DC 6/2/5 0 Y TLRUIDA CBdms Status ...
4
votes
5answers
394 views

Command line method to find repeat-word typos, with line numbers

Updated: Clarify line number requirement, some verbosity reductions From the command line, is there a way to: check a file of English text to find repeat-word typos, along with line numbers where ...
0
votes
1answer
41 views

How to select lines in a file that contain a number from a list of numbers

I have a text file, file.dat, with a number of columns separated by a space. There are some numbers in the second column. I have another text file, select.dat with a list of numbers. I'd like to ...
1
vote
1answer
46 views

grep content between two starting bracket

Below is file output that I want to access [1] RsyncCommand: 0 Number of files: 18 Number of files transferred: 0 Total file size: 24.01M bytes Total transferred file size: 0 bytes Literal data: 0 ...
0
votes
1answer
82 views

Identifying duplicate fields and REMOVE both with awk

Previously, I asked the question: Identifying duplicate fields and print both with awk. I have a file with multiple columns and wanted to identify lines where specific column values (cols 3-6) had ...
4
votes
5answers
199 views

How do I print all lines of a file with duplicate values in a certain column

Given a file like this: 1,768,12,46576457,7898 1,123,435,134,146 2,345,6756856,12312,1311 5,234,567465,12341,1341 1,3245,4356345,2442,13 9,423,2342,121,463 9,989,342,121,1212 I would like to list ...
1
vote
1answer
67 views

how concatenate two commands?

Please can someone help me to concatenate two commands on shell, I'd have the path result and concatenate with the number of column in the first field. EX: #this gives me the path of my path ...
1
vote
3answers
97 views

grep a awk string from a file and write to the other part of string file

My file, aaa,has 5 lines. 01_AAA-11-ID22 02_BBB-33-ID44 03_CCC-55-ID66 . . I tried to grep anything after underline from a file bigfile.txt for i in $(cat aaa) do A= $(awk -F '_' '{print $1}' $i) ...
5
votes
6answers
157 views

How to remove symbols from a column using awk

I have data like this: chr1 134901 139379 - "ENSG00000237683.5"; chr1 860260 879955 + "ENSG00000187634.6"; chr1 861264 866445 - "ENSG00000268179.1"; chr1 879584 894689 - ...
0
votes
1answer
21 views

How to remove a row if the difference between two columns is less than 2000

I have a dataset that looks as follows: chr1 HAVANA gene 69091 70008 . + . gene_id "ENSG00000186092.4"; transcript_id "ENSG00000186092.4"; gene_type "protein_coding"; gene_status ...
1
vote
1answer
80 views

split file into multiple pieces

Problem: given a file samplein, it can be split up into multiple pieces as follows: $ cat samplein START Unix Linux START Solaris Aix SCO $ awk '/START/{x="F"++i;}{print > x}' samplein $ ls F* ...
3
votes
2answers
237 views

Conditionally delete a line from file

In a file that looks like: 549.432086168 0.0000E+00 9.6988e-04 2.0580E-02 1.0000E+01 9.6988e-04 2.0580E-02 2.0000E+01 9.6988e-04 2.0580E-02 . . . 5.6000E+02 7.0997e-06 -3.7538E-04 Delete the last ...
6
votes
1answer
489 views

What is a *.sorted command and why doesn't it work in my bash?

In a Bash Guide for Beginners (sec. 3.2.1.2. Local variables) there is an example command which is supposed to list all of the local variables, that are not in enviromental variables, or so I think: ...
0
votes
1answer
42 views

Format the output of awk command

I have to format the output of the awk command which gets the input from a file Sample Input File 123 ABC 12 23 34 45 56 67 78 13 234 DEF 23 34 45 56 67 78 89 345 GHI 34 45 56 67 78 90 10 01 99 456 ...
0
votes
1answer
36 views

Get line after match using awk

I want MAths asad 2323 dfd to show line after asad I do awk '{if($1=="asad")next};1' test get results MAths 2323 dfd why is next not working? I only want to match the Maths not the rest of ...
3
votes
2answers
64 views

Set awk array on command line?

I know the -v switch can be used to awk on the command line to set the value for a variable. Is there any way to set values for awk associative array elements on the command line? Something like: ...
2
votes
2answers
74 views

How to sort data stream by 2 column names (column number may vary)?

I get a stream of data from an API with which look like this: redID blueID whiteID 1 22 2 44 15 41 2 15 15 31 2 14 What I need to do, is to sort ...
2
votes
2answers
33 views

How to alter a column in a row in a tab delimited text file if it's duplicated

Lets say I have these many rows chr1 66999638 66999638 SGIP1 1 + chr1 66999251 66999251 SGIP1 1 + chr1 33545778 33549778 AZIN2 1 + chr1 8376144 8380144 ...
0
votes
2answers
59 views

How to Pivot the data by awk in unix shell script?

I'm trying to pivot a file using awk. This is an example of my input File: P2P,P2P,MP,DL,AIR,UP,HR,UPE,GSM,S,PP,1,NA,0.2,0.2 P2P,P2P,MP,DL,AI,UP,HR,UPE,GSM,F,PP,4,NA,0.2,0.8 ...
2
votes
2answers
63 views

Selective extraction of data

I have the following block of data in a txt file : Informatica(r) PMCMD, version [9.5.1 HotFix4], build [313.0217], SOLARIS 64-bit Copyright (c) Informatica Corporation 1994 - 2014 All Rights ...
0
votes
1answer
29 views

awk for loop with leading zeros in variable

I want to run a for loop in awk with leading zeros in the index variable. This isn't for printing a number with leading zeros, which I could easily handle with a printf statement. It's for checking ...
1
vote
2answers
64 views

Adding “:” time-formatting using awk

For an input file named Lab1: 034023 052030 034023 022130 044023 012030 034223 022030 034123 152030 024023 152030 AWK command awk 'gsub(/[0-9][0-9]/,"&:",$1) ...
0
votes
1answer
28 views

Removing “,” for calculation and adding it back using awk

I have a file as January 124,567.89 Feburary 234,567.78 March 349,123.77 April 112,248.00 May 107,345.22 June 180,670.23 July 122,457.03 August 345,789.99 September ...
0
votes
2answers
39 views

Use a shell variable with Awk

I have this: if [[ $1 = "-c" ]] && [[ $2 =~ ^[a-z A-Z]+$ ]] then awk -v var="$2" '{print $2}' something.txt fi What I want is with a comand like -c abc to show the lines in something.txt ...
1
vote
2answers
54 views

Print smallest integer from file using awk custom function?

awk function looks like this in a file name fun.awk: { print small() } function small() { a[NR]=$0 smal=0 for(i=1;i<=3;i++) { if( a[i]<a[i+1]) smal=a[i] else smal=a[i+1] } return smal } ...