awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
0answers
193 views

How to redirect linux command outputs to specific row and column in csv?

I'm working on Unix Build post documentation process, I have a CSV file to be filled in and uploaded for every new build, to ensure the servers are build as per the original request. The CSV file ...
1
vote
3answers
44 views

Transpose paramter values from a file and Generate parameter files from it

There's a file containing a table with the parameter names as the first line, values in the following lines: How to perform the following using awk, sed or any shell scripting utility: generate a ...
1
vote
2answers
41 views

awk alter the next line after a pattern match based on another pattern match

I have a file with lines such as: .... pattern1 100 200 300 pattern2 300 400 400 pattern1 300 900 700 pattern1 200 500 900 ... As shown in the above example, there are some lines where pattern2 ...
1
vote
2answers
39 views

Using getline with NR in awk

I have a bash script which calls a awk script to print each line from the file name_list.txt #!/bin/bash awk -f parse_list.awk name_list.txt In the awk script, I am using getline with NR to print ...
1
vote
1answer
44 views

if condition in bash scripting

I have a text file that I want to check some line as below com1=`awk 'NF{a=b;b=$0} END{print (b ~ /\com/)? "Yes" : "No"}' $myfile` com2=`awk 'NF{a=b;b=$0} END{print (b ~ /\t/)? "Yes" : "No"}'...
0
votes
2answers
53 views

How can I add new character after each line?

I am trying to manipulate a file, by adding an A after each lines Name Test However the awk '{print $0"A"}' file_list.txt > file_list2.txt command gives the following result Name A Test A ...
3
votes
2answers
151 views

AWK combine variable and regular expression in pattern match

I am looking to search a file for a column that is 3 letters followed by any 3 or 4 numerals. eg. if ( $1 ~ /^[A-Z][A-Z][A-Z][0-9][0-9][0-9]/) However I need the 3 letters to be a variable, so I ...
2
votes
4answers
8k views

Match multiple regular expressions from a single file using awk

I'm trying to parse a HTML file using shell scripting. There are 4 different regular expressions that i need to catch: name= , age= , class= , marks=. Using grep "name=\|age=\|class=\|marks=" ...
0
votes
1answer
24 views

Insert Line in File When Group of Lines Changes

Let's say I have a text file like this: group1 hello group1 there group1 nice line group2 here group2 we group2 go What I want to do is insert a line when the group of lines changes, something like:...
1
vote
3answers
71 views

How to join two files by matching a column with an irregular number of columns?

file1: 0000002|SLM DEV CORP |PO 857 0000003|S TOPPING |APT 19 0000004|JD BROS LTD ...
0
votes
2answers
801 views

put rows of numbers into a column with shell script

I have a file with many lines, like 1 jfkdajfd 1 2 3 5 2 fkldfjld 3 fdkfloaf 9 10 4 fldfldkf 5 fdskf;ak 12 1 4 I want to get all the numbers and put them in a column in a file, like 1 ...
1
vote
1answer
71 views

Replace awk and tr with just awk for converting to lowercase

Not a duplicate question, as I am asking for awk specifically. I have the following command to convert everything to lowercase, and then reprint out substrings in a certain order. cat words |tr '[:...
0
votes
2answers
197 views

Grep / awk on multiple files to single output

I have several txt-files containing data, where i use grep to search for a current string of text, and use awk to filter out the variable i need. The string is repeated through the file, so i ...
1
vote
3answers
51 views

vlookup work in awk on linux

Hi I have two files containing data: file1: 4,abc 3,xyz 5,fut 6,gfd file2: gfd,2.3 xyz,4.5 abc,6.7 fut,3.2 i wish to create a file3 as output file using linux scripting commands: 4,6.7 3,4.5 5,...
0
votes
1answer
36 views

awk: arithmetic does not work on a particular field

I have several lines such as: N2O + M = N2 + O + M 1.300E+11 0.00 59620.0 I need to substitute the third from last field with a factor of itself which does not work ...
1
vote
3answers
104 views

Using bash variable substitution instead of cut/awk

Can I use bash variable substitution to extract a piece of a variable based on a delimeter? I'm trying to get the immediate directory name of a filename (in this case, foo). $ filename=./foo/bar/baz....
1
vote
3answers
57 views

Merging 2 files and keeping the one duplicate

I want to create a file that contains columns from two input files. File1 is like: 11111111 abc12345 Y 22222222 xyz23456 Y File2: 11111111 abc12345 33333333 ...
1
vote
2answers
33 views

Compare 2nd column of the 1st file and 1st column of the 2nd file and print the matched records of the 1st file

I have two files large files: f1.txt: 5020118359 |13ZJ24001218 |20141224|R 5020120475 |13ZJ38000813 |20141204|R 5020127431 |13ZJ38001569 ...
12
votes
9answers
3k views

How can I delete the 5th word of every line in a file?

I want to delete the 5th word of each line in a file. The current content of the file: File is not updated or and will be removed System will shut down f within 10 seconds Please save your work ...
1
vote
0answers
61 views

Is there a reference for the assembler-like output produced by `awk -W dump`?

The command awk -W dump ... produces a dump of "assembler-like" code (as shown here [link], or by reading the man). Some of the instructions are easy to guess at their meaning such as jz, which is ...
1
vote
2answers
44 views

Copy plaintext lines from file and append them to same file with mid-line prefix or remove mid-line prefix

Making an ugly script that compiles a bunch of hosts files into one big list. Most lists will have two entries for a URL, one with www. and one without like this: 127.0.0.1 facebook.com 127.0.0.1 ...
1
vote
5answers
270 views

To remove extra space from line using shell command

This is my file, TLRUIDA CBdms Status DP 6/1/1 DC 6/1/5 0 Y TLRUIDA CBdms Status DP 6/2/1 DC 6/2/5 0 Y TLRUIDA CBdms Status ...
0
votes
1answer
96 views

Identifying duplicate fields and REMOVE both with awk

Previously, I asked the question: Identifying duplicate fields and print both with awk. I have a file with multiple columns and wanted to identify lines where specific column values (cols 3-6) had ...
0
votes
1answer
42 views

How to select lines in a file that contain a number from a list of numbers

I have a text file, file.dat, with a number of columns separated by a space. There are some numbers in the second column. I have another text file, select.dat with a list of numbers. I'd like to ...
1
vote
1answer
46 views

grep content between two starting bracket

Below is file output that I want to access [1] RsyncCommand: 0 Number of files: 18 Number of files transferred: 0 Total file size: 24.01M bytes Total transferred file size: 0 bytes Literal data: 0 ...
5
votes
6answers
165 views

How to remove symbols from a column using awk

I have data like this: chr1 134901 139379 - "ENSG00000237683.5"; chr1 860260 879955 + "ENSG00000187634.6"; chr1 861264 866445 - "ENSG00000268179.1"; chr1 879584 894689 - "...
4
votes
5answers
215 views

How do I print all lines of a file with duplicate values in a certain column

Given a file like this: 1,768,12,46576457,7898 1,123,435,134,146 2,345,6756856,12312,1311 5,234,567465,12341,1341 1,3245,4356345,2442,13 9,423,2342,121,463 9,989,342,121,1212 I would like to list ...
1
vote
1answer
87 views

split file into multiple pieces

Problem: given a file samplein, it can be split up into multiple pieces as follows: $ cat samplein START Unix Linux START Solaris Aix SCO $ awk '/START/{x="F"++i;}{print > x}' samplein $ ls F* ...
1
vote
1answer
71 views

how concatenate two commands?

Please can someone help me to concatenate two commands on shell, I'd have the path result and concatenate with the number of column in the first field. EX: #this gives me the path of my path ...
3
votes
1answer
699 views

extracting data from a log file and plotting it using Gnuplot

I have a logfile with the following data in it: Sat Sep 20 11:25:01 BST 2014 -- temp=53.0'C -- message from script /usr/src/scripts/wifi_test_2.sh Sat Sep 20 11:26:01 BST 2014 -- temp=53.0'C -- ...
1
vote
3answers
107 views

grep a awk string from a file and write to the other part of string file

My file, aaa,has 5 lines. 01_AAA-11-ID22 02_BBB-33-ID44 03_CCC-55-ID66 . . I tried to grep anything after underline from a file bigfile.txt for i in $(cat aaa) do A= $(awk -F '_' '{print $1}' $i) ...
0
votes
1answer
22 views

How to remove a row if the difference between two columns is less than 2000

I have a dataset that looks as follows: chr1 HAVANA gene 69091 70008 . + . gene_id "ENSG00000186092.4"; transcript_id "ENSG00000186092.4"; gene_type "protein_coding"; gene_status "...
4
votes
3answers
774 views

Removal of lines with no more or fewer than 'N' fields?

I am working on mac with sed, perl, awk, bash.. I have a large-ish (10GB) text file which has 13 fields (columns) of TAB delimited data. Unfortunately some of these lines have extraneous TABs , so I ...
2
votes
2answers
64 views

Selective extraction of data

I have the following block of data in a txt file : Informatica(r) PMCMD, version [9.5.1 HotFix4], build [313.0217], SOLARIS 64-bit Copyright (c) Informatica Corporation 1994 - 2014 All Rights ...
3
votes
2answers
243 views

Conditionally delete a line from file

In a file that looks like: 549.432086168 0.0000E+00 9.6988e-04 2.0580E-02 1.0000E+01 9.6988e-04 2.0580E-02 2.0000E+01 9.6988e-04 2.0580E-02 . . . 5.6000E+02 7.0997e-06 -3.7538E-04 Delete the last ...
2
votes
2answers
77 views

How to sort data stream by 2 column names (column number may vary)?

I get a stream of data from an API with which look like this: redID blueID whiteID 1 22 2 44 15 41 2 15 15 31 2 14 What I need to do, is to sort ...
6
votes
1answer
498 views

What is a *.sorted command and why doesn't it work in my bash?

In a Bash Guide for Beginners (sec. 3.2.1.2. Local variables) there is an example command which is supposed to list all of the local variables, that are not in enviromental variables, or so I think: ...
0
votes
1answer
47 views

Format the output of awk command

I have to format the output of the awk command which gets the input from a file Sample Input File 123 ABC 12 23 34 45 56 67 78 13 234 DEF 23 34 45 56 67 78 89 345 GHI 34 45 56 67 78 90 10 01 99 456 ...
0
votes
1answer
37 views

Get line after match using awk

I want MAths asad 2323 dfd to show line after asad I do awk '{if($1=="asad")next};1' test get results MAths 2323 dfd why is next not working? I only want to match the Maths not the rest of ...
0
votes
2answers
74 views

How to Pivot the data by awk in unix shell script?

I'm trying to pivot a file using awk. This is an example of my input File: P2P,P2P,MP,DL,AIR,UP,HR,UPE,GSM,S,PP,1,NA,0.2,0.2 P2P,P2P,MP,DL,AI,UP,HR,UPE,GSM,F,PP,4,NA,0.2,0.8 P2P,P2P,MP,DL,AIR,UP,HR,...
3
votes
2answers
70 views

Set awk array on command line?

I know the -v switch can be used to awk on the command line to set the value for a variable. Is there any way to set values for awk associative array elements on the command line? Something like: ...
2
votes
2answers
33 views

How to alter a column in a row in a tab delimited text file if it's duplicated

Lets say I have these many rows chr1 66999638 66999638 SGIP1 1 + chr1 66999251 66999251 SGIP1 1 + chr1 33545778 33549778 AZIN2 1 + chr1 8376144 8380144 ...
0
votes
1answer
29 views

awk for loop with leading zeros in variable

I want to run a for loop in awk with leading zeros in the index variable. This isn't for printing a number with leading zeros, which I could easily handle with a printf statement. It's for checking ...
1
vote
2answers
64 views

Adding “:” time-formatting using awk

For an input file named Lab1: 034023 052030 034023 022130 044023 012030 034223 022030 034123 152030 024023 152030 AWK command awk 'gsub(/[0-9][0-9]/,"&:",$1) gsub(/[0-9][0-9]/,"&:",$2)...
1
vote
4answers
2k views

awk a specific column with unexpected line break

I can't seem to find a solution for this and it's driving me crazy. I know I can use awk to print a column(s). I'm having trouble printing a specific column though because of the way my file system is ...
0
votes
1answer
28 views

Removing “,” for calculation and adding it back using awk

I have a file as January 124,567.89 Feburary 234,567.78 March 349,123.77 April 112,248.00 May 107,345.22 June 180,670.23 July 122,457.03 August 345,789.99 September 234,...
0
votes
2answers
39 views

Use a shell variable with Awk

I have this: if [[ $1 = "-c" ]] && [[ $2 =~ ^[a-z A-Z]+$ ]] then awk -v var="$2" '{print $2}' something.txt fi What I want is with a comand like -c abc to show the lines in something.txt ...
1
vote
2answers
57 views

Print smallest integer from file using awk custom function?

awk function looks like this in a file name fun.awk: { print small() } function small() { a[NR]=$0 smal=0 for(i=1;i<=3;i++) { if( a[i]<a[i+1]) smal=a[i] else smal=a[i+1] } return smal } ...
2
votes
4answers
2k views

How to subtract two times in shell

I nee to subtract two lines which is in the format of time in shell. The time format is hh:mm:ss I used the code below to get the time. cat /var/log/kern.log |grep usb |tail -2| awk '{print $3}' ...
1
vote
4answers
202 views

Move lines 2 3 after line 5 using awk

I have file content as: this line 1 no un1x this lines 22 0 butbutbut this 33 22 has unix but not 1 THIS is not butbutbut ffff second line awk 'NR==2 && NR==3 {f=$0;next} NR ==5 &&...