awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
2answers
63 views

Problem with data comparison in if loop

I have 2 variables in bash. Using test i found out that one of them is integer and other one is string. I want to compare them and see if they are equal. I tried the following options. if [ "$a" -eq "...
6
votes
4answers
519 views

Delete column that includes a specific text

I was looking for an option to delete a column with a specific text like "ddd" aaa bbb ccc ddd eee fff 1 2 3 4 5 6 2 3 4 5 6 0 So that the output looks likes this: aaa bbb ccc ...
3
votes
2answers
70 views

How to format a file while moving?

I have 2 directories. SOURCE and DESTINATION. I am moving the .csv files from source to destination as mv -f SOURCE/*.csv DESTINATION/ While moving, I want to remove the first and last line from ...
2
votes
1answer
44 views

How to change trailing numbers to incrementing number

I'm using ArchLinux 4.1.6-1 and bash 4.3. I need to tidy up a pile of programs in 200+ industrial robots and so trying to automate the leg work. I can strip a file down to a just a list of ...
1
vote
1answer
27 views

Modify string if original does not match particular string

I have a file with millions of rows and want to modify values in column A only if the string is '.'. The modification would be to add "chr:"$2":"$3 to the start of the string. All other rows would be ...
0
votes
1answer
91 views

Wanted: nonassociative array in AWK

I am looking at speeding up some code in AWK by serially processing an internal string representation instead of serially processing an array representation. (For Examples 1 and 2 below assume ...
4
votes
2answers
394 views

Keeping first instance of duplicates

I have a file with multiple columns and have identified lines where specific column values (cols 3-6) have been duplicated using a bash script. Example input: A B C D E F G 1 2 T TACA A 3 2 Q 3 4 I ...
2
votes
1answer
257 views

replace nth column of a csv file by the help of lookup table

I have a csv file (a.csv) as below (with thousands of lines): NMS_NE,,,,,,,,,,,,,,,,,,,,,,, D,NE Name,IP Address,Site Name,Product UPEI,Product Name,Product Model,NE Status,Serial Number,SW Major ...
-1
votes
1answer
38 views

Get the respective tablenames if it matches particular pattern

I need to list the table names if it finds particular pattern from the below file. CREATE TABLE IMS.ACTIVITY ( "ID" INTEGER NOT NULL , OWNING_TYPE FOR COLUMN OWNIN00001 VARCHAR(20) ...
0
votes
1answer
58 views

Split a set of data to massives and find maximum for each

I have a set of data looking like this: 1 0.1 2 0.23 3 0.05 5 0.73 6 0.40 7 0.34 10 0.11 11 0.45 12 0.68 I need to split the set of data to massives using the rule that the next ...
1
vote
3answers
314 views

how to extract a value from a string in xml using grep or awk in Linux

I have a file containing: DEVICE="7" PGMNR="24" VCONF="800" In this file, I want to find PGMNR="24" and I need the value 24 without the ". The PGMNR="24" may be located anywhere in the line. The ...
2
votes
3answers
215 views

How to substite a " in awk?

For example, I need to change a " into the word quote in to change a string with a " at some point into a string with a quote at some point I have tried: $ echo 'a string with a " at some ...
1
vote
2answers
71 views

use an export variable as part of filename in awk program

I have set an export X25DATAF=/home/Bob/X25/Datafiles in my profile and that's what is outputted when I do a echo $X25DATAF. I want to replace /home/Bob/X25/Datafiles in the awk below with the $...
0
votes
1answer
61 views

Replace a string with “string|string”

I have a pretty weird problem. I did some calculations for bioinformatics, but my downstream apps won't accept my header information in its current format. In order to circumvent this I modified my ...
0
votes
1answer
29 views

create a structured table

I have a file which is as follows Tran No Date Vchr No Debit Credit 1239 05/06/2015 115 750.00 ....
1
vote
2answers
38 views

Reading a variable-sized line and splitting its content into separate lines with text

I have a file with the header (first line) with a variable length; and it comes as: "One"|"Two"|"Three" or "One"|"Two"|"Three"|...|"One Hundred" and want to get do a loop that transforms it ...
0
votes
1answer
58 views

Fetch second and fourth column from a file

Problem Statement I am not able to get the values in second and fourth column. Tried grep -i "AUGC\|AACN\|AACY" output.csv | cut -d ',' 2,4 DATA success,1234,["madhu",{'first',[{'last',18}]},"I0"...
0
votes
1answer
50 views

How to remove the first instance of something in a file with awk?

If I have a file with multiple lines with ... end ... end ... end ... How could I remove only the first instance? So far I have tried awk 'BEGIN {found=false} / end/ { d; found=true} else ...
4
votes
3answers
292 views

Convert genotypes to 0/1

I have a file that looks like: 1 rs6687776 1020428 T C T C T C C C T C C C T C The 4th and 5th column are the two different possible alleles at that ...
1
vote
2answers
121 views

Print the first and last match of a field with awk

I have a tab delimited file: qrs John tuv abcd Sam efgh ijk Sam lmnp abcd Sam efgh ijk Sam lmnp qrs John tuv I am trying to print the line in which the ...
3
votes
5answers
779 views

Find text between tab (\t) as a delimiter

I thought this will be simple, but can't find out how to do it. Scenario I have a single .csv file with id_user,text,id_group columns where each column is delimited by tabs such like: "123456789" ...
4
votes
1answer
130 views

Thousand separator in awk

I would like to print thousand separator with awk's printf. In bash, it is pretty straightforward: > printf "a %'d b \n" 1234567 a 1,234,567 b > LC_NUMERIC=fr_FR.utf8 > printf "a %'d b \n" ...
1
vote
2answers
53 views

Split file in two on line before pattern

This is an almost identical question to split file into two parts, at a pattern but instead of having the lines up to XYZ in file1 in the example file.txt: ABC EFG XYZ HIJ KNL I want to have the ...
6
votes
1answer
173 views

Handling 3 Files using awk

Consider following files: file1: boo,8,1024 foo,7,2048 file2: foo,0,24,154 noo,0,10,561 file3: 24,154,7,1024,0 What I need is to go to File1 and check if $2==7; if true, take $1, $2 and $3 ...
0
votes
1answer
32 views

Print specific FULL column only by range

I have a file which have contents like this. Column1 Column2 Column3 Column4 USER1 25 Years xyz zyz 123 abc ...
10
votes
5answers
611 views

How to print own script name in mawk?

In bash $0 contains the name of the script, but in awk if I make a script named myscript.awk with the following content: #!/usr/bin/awk -f BEGIN{ print ARGV[0] } and run it, it will only print "awk"...
1
vote
2answers
62 views

Remove last comma within a pattern

If I have the following text within a file (somefile.txt): CREATE TABLE "table_name" ( "id" int(11) unsigned NOT NULL, "some_field" varchar(10), ); CREATE TABLE "another_table" ( "id" int(...
0
votes
2answers
28 views

Delete multiple lines with text and curly braces

I have a multiple files that contains something like: this is a test1 { test 123 test 456 test 789 } this is a test2 { test 123 test 456 test 789 } this is a test3 { test 123 test 456 test 789 } ...
0
votes
2answers
33 views

Merge labels from lines with duplicate fields

I have a rather large text file(thousands of lines) in following format: 123.123 5 123.123 91 156.456 45 233.185 85 233.185 4 257.448 1 455.456 60 455.456 2 The first column are ...
0
votes
3answers
133 views

How to replace contents of each line depending on some formula?

I've been working with bash to get some data output to a multiline file with the data stored in this format for each row: text1 "text2" text3 integer1,integer2 A "space" separates the first four ...
8
votes
8answers
1k views

Split text file into lines with fixed number of words

Related, but no satisfactory answers: How can I split a large text file into chunks of 500 words or so? I'm trying to take a text file (http://mattmahoney.net/dc/text8.zip) with > 10^7 words all in ...
1
vote
2answers
52 views

With awk, how to Insert a number only for lines starting with “Toto” when they are between a specific couple of patterns

With awk, I would like to insert a numbering like Record n°i# in front of each line starting with Toto only when they are located in-between two specific patterns start=ABCD and stop=EFGH. Input file ...
2
votes
4answers
492 views

extract url between 2 strings in a file

I have a file in which each line is like this "372"^""^"2015-09-03 06:59:44.475"^"NEW"^"N/A"^""^0^"105592"^"https://example-url.com"^"example-domain < MEN'S ULTRA < UltraSeriesViewAll (18)"^"...
2
votes
3answers
77 views

How to add “wrappers” around methods in source code based on a pattern, using sed, awk, grep and friends

How can I add wrappers to a file based on a pattern? For instance I have the following: ... find(css_policy_form_stage3).click with_ajax_wait expect(css_coverage_type_everquote).to ...
0
votes
2answers
70 views

Replace values on the basis of matching ID variable

I have two files. file1 has 6 columns, file2 has 2 columns. I want to replace the values in the 6th column of file1 by the values in column 2 of file2, based on matching ID variables (column 1 in both ...
1
vote
4answers
175 views

Replacing a “time” column with corresponding values

I have a file in the following format: "2004-04-19 12:25:57" 44 44 "2004-04-19 13:39:32" 36 36 "2004-04-19 14:00:53" 34 34 I need 2 new files: a) A file which will replace "time" values of the ...
5
votes
8answers
746 views

awk or sed command to match regex at specific line, exit true if success, false otherwise

I need to determine if a file contains a certain regex at a certain line and to return true (exit 0) if found, and otherwise false. Maybe I'm overthinking this, but my attempts proved a tad unwieldy. ...
0
votes
1answer
229 views

Replace the first word of each line with its MD5

I have file with two space separated word in each line. I am looking for shell scrip in Unix to replace the first word of each row with its md5 hash representation. I have tried to adjust what is ...
-1
votes
3answers
66 views

How to delete certain line from file?

I have a text file with line containing only comma. How can I delete this particular line? Consider following example: CREATE TABLE ENTERPRISE.TSDAILYACCTSUMMARY ( TSDAILYACCTID FOR COLUMN ...
-1
votes
1answer
87 views

How to add double quotes to a various strings in a same pattern in a file

I have a file like this CREATE TABLE ENTDTA.$$BAPWD ( PERSID NUMBER(10) DEFAULT 0 NOT NULL , "PASSWORD" CHAR(10) DEFAULT '' NOT NULL , UPDATE_IDENT NUMBER(7, 0) DEFAULT 0 NOT NULL ) ...
1
vote
1answer
238 views

OSX: Delete all matching certificates by command-line

Using shell script to delete all matching certificates. I found a list of certificates using: security find-certificate -c "certificatename" -a -Z|grep SHA-1|awk '{ print $NF }' Output is like ...
1
vote
1answer
33 views

How can I use awk to create a variable that is used in the next line edit

I have a file with a list of info. Each line has either 12 or 13 fields. For example I will use a file will 3 and 4 fields. e.g. myfile.txt user password location date john passwd1 new_york today ...
2
votes
1answer
93 views

Merge lines with matching first column

I've been trying to use awk to merge two files when they have the same first column. Here are my example files: FileA.txt A2M 1 A4GALT 11 AAAS 35 AAGAB 7 FileB.txt A4GALT 2 AAAS ...
3
votes
2answers
196 views

Why awk says “syntax error” for the comma I placed between the two patterns?

I run the following script under Linux (Mint on a USB key) with GNU Awk 4.0.1 installed by default. Error returned by awk is the following : awk: cmd. line:7: /^Observation/,/^@@@/ {...
-1
votes
1answer
31 views

Look for a particular pattern and do a swap on that line

I have a set of files containing VIEW_NAME FOR COLUMN DNAME , VIEW_OWNER FOR COLUMN DCREATOR , OBJECT_NAME FOR COLUMN ONAME , OBJECT_SCHEMA FOR COLUMN OSCHEMA So, ...
0
votes
1answer
45 views

replacing first link of a text “awk equivalent of sed command”

I wanted to replace the first line of a text with sed it should be like sed -i.bak "1 s/^.*$/"CompoundState\tMethod\tApproach\tS^2\tEnergy\tPath"/" awk.xls but I also need to run it on OSX but it ...
0
votes
1answer
31 views

sum over floating points of the form 3.016e-06 using awk

I want to sum over floats of the form: 3.016e-06 The floats are saved in a files called S_3x3.txt S_4x4.txt ... S_8x8.txt My code sums only the first integer. Instead of 3.016e-06 + 4.022e-06 it ...
6
votes
4answers
1k views

How to add a header and/or footer to a sed or awk stream?

I have a bunch of output going through sed and awk. How can I prefix the output with START and suffix the answer with END? For example, if I have All this code on all these lines and all these ...
4
votes
1answer
102 views

How to use awk to indent a source file based on simple rules?

How can I indent source code based on a couple of simple rules? As an example, I've used sed and ask to transform a selenium HTML source table to the following rspec like code. How could I ...
0
votes
1answer
41 views

Print concat vars inside loop

I have a code like this: cat input | while read p; do lynx -dump "www.xyz.com/p?=$p" | grep -m 1 -A 1 "Hello" | awk -F: '{print $2}' >> output done Where the content looks like this: [$p ...