awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
1answer
99 views

Reading a string till a key word and replacing from there with another string

I am new to UNIX and am practising shell scripting. I have an application where I need string searching and replacement. There are some lines in a file 1 and file 2. I have a search key word. If the ...
1
vote
1answer
82 views

When will awk treat newline character as `;` and when not?

I'm reading the famous awk tutorial from http://www.grymoire.com/Unix/Awk.html. In which it mentioned To be precise, in the original AWK you can insert a new line character after the curly braces, ...
1
vote
1answer
188 views

grep (/sed/awk) month range

I've looked around for solutions to this, but it's sort of difficult to search for the solution to such a specific problem (lot of words to put in Google). So, I have a file with data that looks like ...
2
votes
2answers
113 views

how to run awk twice across the same file

I have a log file with user_id's and another column with contest results. I would like to: find all the user_ids of the users who won given those user_id's, return ALL the log entries for those ...
3
votes
3answers
199 views

How to view all the content in an awk array?

In my understanding, awk array is something like python dict. So I write down the code bellow to explore it: awk '{my_dict[$1] = $2} END { print my_dict}' zen And I got: awk: can't read value of ...
1
vote
1answer
52 views

Comparing two files using awk language

I have two files named file1 and file2. I want to compare with the NR of first files with the first field of file2 i.e. $1. Like this: if(file1.NR==file2.$1){ then do some calculations}. Please help ...
3
votes
3answers
299 views

Why awk understand FS=“*” but not for FS=“-*-”?

I got a test file, its content is: a -*- b I used awk 'BEGIN {FS="*"} {print $2}' test, it prints out - b Correct! But when I use awk 'BEGIN {FS="-*-"} {print $2}' test, I got: * I know that ...
1
vote
1answer
60 views

setenv from awk in tcsh

I am trying to set environment variables one per line from awk: $ <source command> awk '/inst/ { n = split($0,a,"/"); cmd = "setenv " a[n-1] "_ver" $0; system(cmd) }' However the out I get is ...
4
votes
1answer
165 views

Why does awk '{print “Hi, Judy”}' print “Hi, Judy” repeatedly when pressing Return?

For my understanding, usually awk follows a file or an output. Then it will exec its code on everyline of that file or output. In my case, my command awk '{print "Hi, Judy"}' didn't follow by any ...
-1
votes
2answers
28 views

Need to merge mulitple unix txt lines [closed]

Hi I need to merge multiple lines too one line in txt using awk,have tried different things but can't get the result I'm after! An example; need it to go from this: 11 01 30 31 33 37 39 39 30 30 30 ...
2
votes
1answer
34 views

How to remove duplicate lines while keeping the order AND ignoring case?

[user@notebook ~]$ printf '1\n5\n3\n3\n4\n2\nA\nB\na\n' 1 5 3 3 4 2 A B a [user@notebook ~]$ [user@notebook ~]$ printf '1\n5\n3\n3\n4\n2\nA\nB\na\n' | awk '!seen[$0]++' 1 5 3 4 2 A B a [user@notebook ...
0
votes
2answers
29 views

How to select specific sets or sturture from a file, if struture contains specific string?

I want to extract the sets with having specific string, for example: file abc.txt contains abc 123 abcd 1234 abcde 12345 abc 123 abcd 123 abcde 12345 abc 123 abcd 1234 abcde 12345 Now from the ...
0
votes
1answer
177 views

Sum two columns from different files

I have two files like this: File1: -54.005 -67.405 0.1 -54.05 -68.7833 0.2 -54.05 -68.7333 0.3 File2: -54.005 -67.4050 70 12.7 -54.05 -68.7833 167 12.5 -54.05 -68.7333 205 12.6 ...
1
vote
1answer
54 views

Merge columns with the same value with awk

It is possible merge columns with the same value in awk? Something like if awk '$1 = $2' then merge I got this AZ14962 AZ14962 ctg_ 104 AZ14962 AZ14962 ctg_ 4 AZ14962 AZ14962 ctg_ 61 ...
0
votes
1answer
63 views

How to use awk to read data between all frequent time intervals

I have a log file in following format [30/Jan/2015:10:10:30 +0000] 12.30.30.204 xff=- reqId=[-] status_check len=- GET /api/getstatus HTTP/1.1 mi=- ec=- 200 425 [30/Jan/2015:10:11:00 +0000] ...
-1
votes
1answer
28 views

how to print certain text in a line

How can I get the result in the below format? Below is the line on which action needs to be performed. Empty Results found in BusListingAction/buslisting.htm for: DepartureCity: BARMER ArrivalCity ...
0
votes
2answers
83 views

Using AWK to Print Output of Combination of Unspecified Number of Files

My objective is to move into a subdirectory, call it SUB, which has many text files with 5 columns each and always the same number of rows. I want one large 'sum' file that adds all columns from 2-5, ...
1
vote
2answers
67 views

Use awk to replace a column without changing format [closed]

I have a PDB file (see below) and I need to change the A in column 5 to a B, AND I need to change it without changing the format of the original file. I was able to change the A to a B using this awk ...
3
votes
3answers
83 views

Append and write out variable-named file from list of variables

I would like to create multiple 'wget' shell scripts to download a batch of considerably large (~3GB) files using our HPC cluster. The names of the files are stored in filenames.txt, like so: $cat ...
1
vote
2answers
171 views

read first element in array

defaults read -g AppleLanguages produces something like this: ( en, de ) I'd like to extract just the first element, in this case "en", defaults read -g AppleLanguages | awk '/\(/ ...
1
vote
1answer
62 views

split line based on space and delete the second part

I have a big file >fid|29290408|locus|VBIEntCas2262_0001| Phosphoglycolate phosphatase (EC 3.1.3.18) [Enterococcus casseliflavus EC20] ...
-1
votes
1answer
196 views

multiple input in a awk file

Please guide me to get out of it. Thanks in advance. I am having a situation that I have to write a awk script which takes the input of two .out files and generate a single .txt file. $ cat ...
3
votes
7answers
1k views

can we print the last word of each line in linux using sed command?

suppose , if there is a file consisting of following lines , if they are 12345 567 7878 66 er3 t45t y6y46y 4y6 y656y y5y 46y6 65y7 y66uyuy yy46y6y The output has to look like: 66 ...
0
votes
1answer
34 views

Insert space at specified position in a file

I have this sample file: 234234|111||J1X0H8|SERP||R|D|666|M 234235|222||J1X0N5|SERP||C||555|M For each line, I need to insert a space after the 3rd character of the 4th column, like so: ...
0
votes
2answers
331 views

How to convert Log file to CSV using bash

Is it possible to turn this log entry into a properly formatted csv file using sed/awk? I've tried multiple ways to do this using sed/awk. Alas I couldn't do it. Before I do it in perl. is this ...
0
votes
0answers
32 views

Awk - Take $4$3$2 from Line 01 and $4 from Line 07 and Create a matching key

Any help is greatly appriciated. Desired Matching Key: $4$3$2 from line 01 and $4 from line 07 of A.txt Example: In A.txt $4=5000, $3=68, $2=89 in line 01 and $4=RT0429 or RT0428 or RT0588 in line ...
2
votes
3answers
259 views

Replace multiline string in files

I have a number of files I want to update by replacing one multi-line string with another multi-line string. Something along the lines of: * Some text, * something else * another thing And I want ...
0
votes
2answers
54 views

Get all the records from a file with Date collumn less than a month

Input file ID,Name,join_date 21433432,rds,2014_02_01 2131321,ABCDS,2014-10-20 2432745314,ASRER, 2015-01-20 2132432423,safdsad, 2015-12-30 Need two files created one for records with the date column ...
1
vote
2answers
52 views

awk manipulation

I have a file like this SRR959756.1081725 1 RNU2-54P:112:133:hsa-miR-1246 SRR959756.1162547 1 RNU2-54P:112:133:hsa-miR-1246 SRR959756.128602 2 ...
4
votes
3answers
65 views

How to select first occurrence between two patterns including them

How can I select first occurrence between two patterns including them. Preferably using sed or awk. I have: text something P1 something content1 content2 something P2 something text something P1 ...
-3
votes
1answer
35 views

Empty columns which were seen before

I have a large CSV file and want to empty certain columns if they were seen before. So I have (to illustrate my problem): Category | Subcategory ---------+------------ foo | bar foo | bar ...
0
votes
2answers
41 views

Multiply a column using awk

How do I multiply a column by an integer (say 3) and replace the older value by new one, using awk? Input: Data 9390.900391 10573.089844 80.000000 200.000000 2.700000 Data ...
1
vote
2answers
92 views

Formatting ifconfig using sed/awk

I'm using ifconfig on OpenSUSE. When I run ifconfig eth0 I get eth0 Link encap:Ethernet HWaddr CE:FD:75:DF:A5:6D inet addr:172.16.4.177 Bcast:172.16.5.255 Mask:255.255.254.0 inet6 ...
1
vote
1answer
113 views

Searching file based on data range

I previously posted requesting help with counting occurrences of a string. I'm now hoping to search for the occurrence of a string within a range of values and print out a similarly formatted file ...
2
votes
3answers
217 views

How can I insert the contents of a file into a string in bash

I would like to append the contents of a multi-line text file after a particular line in a string. For example, if the file file.txt contains line 1 line 2 I'd like do so something like printf ...
1
vote
1answer
272 views

Compare two Arrays in KSH and output the difference

I am not extremely familiar with KSH (Actually just started using it) and I am having problems with trying to create a script that will essentially compare two arrays that have been stored and then ...
1
vote
2answers
165 views

Multiple File Redirection with awk and find

I have some files in a directory. And I want to add some lines at top and end of file using awk. Example: My awk command: awk 'BEGIN { print "line1\nline2" } { print $0 } END { print "line3\nline4" ...
3
votes
2answers
136 views

Assign specific line from a variable to another variable

I would like to take a specific line from a variable to another variable. I tried this but it doesn't work: c="1.apple 2.banna 3.peach" read "Please choose fruit [1-3]:" t a=$c | awk "NR==$t" echo ...
-4
votes
1answer
213 views

Merge two csvfiles by column header

I have two csv files: success.csv "ID","Legacy ID","Field1", "Field2" 1,1111,google,news 2,2222,yahoo,news error.csv "Legacy ID","Field1","Field2","Message" 3333,aol,news,"Failed to upload ...
2
votes
1answer
60 views

Monitor and alert user when stty settings change?

A particularly large (~10^6 LOC) program causes my stty settings to change from echo ixon icanon to -echo -ixon -icanon and I would like to find the function in this massive program that causes this ...
4
votes
2answers
68 views

How to print matching value of patterned field separator in AWK

In the below command, cat abc.txt|awk -F"[0-9]+." '{print FS $3}' Here FS prints [0-9]+.. However, I want the awk to print the matched value by that field separator e.g. "99." . Any idea how can ...
3
votes
5answers
540 views

Awk - output the second line of a number of .dat files to one file

I have searched through previously asked questions and can't find an answer to what I am trying to do. note, I am new to linux. I have multiple files something like: (in reality i have 80) file1.dat ...
2
votes
2answers
124 views

Extracting values from a file keyed by multiple keys

Consider a file with key=value pairs, and each key is optionally a concatenation of multiple keys. In other words, many keys can map to one value. The reason behind this is that each key is a ...
0
votes
1answer
41 views

For value falls within a range get corresponding value

I am thinking to use awk to search a value from input file in a reference file and get a corresponding value from reference file. Both reference and input files can be sorted based on $3 and $2 ...
1
vote
1answer
335 views

Awk compare 2 files, print match together with non match lines filled with 0

I have two files. I want to match the 1st column of first file to 1st column of second file. If there is a match I'd like to print the 2st column of second file, if not 0. I have tried awk (awk ...
1
vote
2answers
57 views

splitting a file by column number

I have a file with 10,671 columns. I am trying to split it into 10 smaller files, each with 1000 columns until all columns have been printed, named file_transposed01, file_transposed02, ... ...
3
votes
1answer
45 views

Filter out inconsistent data blocks

I would like to filter out rows that has inconsistent geno values in line-part blocks. In case of duplicate consistent values , I would just keep the first row. As example, 'gf345 part1' has more ...
0
votes
2answers
44 views

Rearranging data in file with awk

I have a table ID Comment Statement 1 hi hello 2 hi hello 1 cya bye 1 k ok 2 kk ok How would one go about producing a result ...
2
votes
3answers
79 views

Rename mp3 file names to a pattern

I have a list of like names likes so: /A/B/C/D/E/F/filename_album1-003.mp3 its more like album\d+ I would like to change it to something like /A/B/C/D/E/F/filename_song-3.mp3 song is just a ...
0
votes
1answer
32 views

putting headers back after awk command

I have file like this. Its called a sam file @HD VN:1.0 SO:unsorted @SQ SN:RNU2-1 LN:388 @SQ SN:RNU6-1100P LN:304 SRR959756.23 0 RNU2-1 56 0 6S32M12S * 0 0 ...