awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

2
votes
6answers
124 views

Printing specific section of a line when a trigger value is present

I'm parsing data in the following format: prop1=value1:prop2=value2:prop3=value3+prop1=value4:prop2=value5 parts of the string are delimited by + properties can appear in any order the desired ...
0
votes
4answers
507 views

Adding a character to every other text line

I have a huge (ca. 20G) text file which contains millions of passages (a.k.a. paragraphs) with headers. Headers and paragraphs are always one line each, e.g., Sunshine This is a sunny day. ...
0
votes
3answers
123 views

Finding and replacing text inside awk block

I want to find and replace a value inside awk block. Here is a sample input file to my script P1 10,9:11/18013013582 ,10:1 ,167:0 ,487:5/E.164 11,9:15/310410532169026 ,10:60 ,167:0 ,487:4/IMSI ...
3
votes
2answers
827 views

Split log file by time range

I have a log file and would like to cut it into copies of 15min. In the log file the lines start with: dd mmm yyyy hh:mm:ss,xxx e.g. 12 Feb 2013 16:05:02,xxx log text... and the file rotates ...
1
vote
2answers
37 views

Using Awk to Search Bro Log for IPs/Keywords Found in Online Text File

I am examining Bro logs and would like to find a way to use AWK to grab IPs from a tracker link and search the Bro logs for those IPs automatically, printing the results of each match into ...
1
vote
2answers
47 views

Is there a way to limit the range of the range pattern in awk?

I am trying to use the awk range pattern to find all SQL select statements in a group of files, inspired by this stackoverflow answer. From the awk manual: The pattern1, pattern2 form of an ...
1
vote
2answers
50 views

Using sed to replace javascript code in multiple files

I need to replace an old google analytics code with the new universal tracking code in lots of static html files. The old code looks like this: _uacct = "UA-XXXXXX-X"; urchinTracker(); The new code ...
1
vote
2answers
86 views

Compare and print data from different text files to one in Shell

How can I compare and print data from different text files to one in Shell. I have captures NAS details of three different boxes using SSH, now I need to combine all the three text files to one file ...
1
vote
2answers
359 views

merge csv files, escape comma inside quotes

I have 3 csv files I want to join by first column (id column) Each file has the same 3 columns Row example : id | timestamp | Name 3792318, 2014-07-15 00:00:00, "A, B" When I join ...
1
vote
2answers
150 views

Compare set of values from two files and print the output in third file in awk

Hi friends i have problem with comparing two files.I'll try my best to explain.I have two file f1 and f2 which i have to compare on basis of two sets Parameter(p) and Value(v). Basically f1 contains ...
1
vote
2answers
136 views

Merging two different files with conditions

I have two different files as shown below. Content of a.txt: HDR|1|||||||||| DTL|@||||||||||| TLR||||||||||||| HDR|1|||||||||||| ...
0
votes
2answers
26 views

Variable in for loop to be used as an array for a csv output

I've been struggling to come up with a solution for this, the goal is to organize the results into comma separated columns as we have to recable our entire datacenter from cat6 to cat6a. $DBFILE ...
0
votes
2answers
34 views

Select lines and format

Is there a way to format the following output so that only lines 1 and the respective lines that state the "rule" name print out? Depending on the configuation, the "rule" line could vary. It could be ...
0
votes
2answers
120 views

delete multiple users

I am the root user and I am setting up a menu for another user to use. This other user will only get this menu. There are two options that are interlinked: the first option is to search users. The ...
0
votes
2answers
81 views

Co Record Verification

I have a task requiring some awk to verify a few records quickly. Let' say: A1,A2 B1,B2 C1,C2 C2,C1 A1,C1 A1,B1 B1,A1 Which would be the best way to check for reciprocity only between the A# the ...
3
votes
1answer
38 views

Solaris: PKG - Script To Verify All Packages Except for a Few

I'm trying to verify the all packages except for a pre-defined list of packages that I know are going to fail for known reasons. This script is going to be run on all Solaris systems within our ...
3
votes
1answer
45 views

Filter out inconsistent data blocks

I would like to filter out rows that has inconsistent geno values in line-part blocks. In case of duplicate consistent values , I would just keep the first row. As example, 'gf345 part1' has more ...
1
vote
1answer
42 views

sort file based one colum and group by another column

I have a file of three columns: 1 A 0.5 2 B 0.7 3 A 10 4 C 4 5 B 4 I want to sort the file by increasing order of column 3 and group by column 2 1 A 0.5 3 A 10 ...
1
vote
1answer
60 views

Shell intervals using awk

I have a file a 10:10 10:30 b 12:15 13:15 c 16:54 17:54 and I need to enter interval for example 10:15 so my output should be a 10:10 10:30 How I need to do that using awk?
1
vote
1answer
25 views

Unable to replace variable in shell script using awk

I have been struggling to replace every occurence of "<KEY>" in a file with ${encryptedkeyValue} I wrote this line but it is not doing ...
1
vote
1answer
46 views

delete non important characters from command output

the version command in my Linux machines display the following output version Red.Hat.OS. release..........5.5 APP.Release.............4.2.0.0-02 how to remove the middle "." dot character ...
1
vote
1answer
60 views

Single record of a file getting splitted over multiple lines

We are using flat text files to store in teradata table after transformations from Informatica. The file contains 14 fields (~ separated). But as the records are getting scattered over multiple lines ...
1
vote
1answer
182 views

grep (/sed/awk) month range

I've looked around for solutions to this, but it's sort of difficult to search for the solution to such a specific problem (lot of words to put in Google). So, I have a file with data that looks like ...
1
vote
1answer
96 views

awk array using number as value did not work

I have a reference file: Refrence File Dpse\GA30012 FBgn0000447 chr2 26607738 26607962 -1 Dpse\GA19764 FBgn0085819 chrX 28571020 28571736 -1 Dpse\ttk FBgn0000100 chr2 ...
1
vote
1answer
68 views

Multiply rows of a column in different chunk of data, add them together and get the average?

I have a chunk of data that looks like this: 924 777 777 777 8445.65349 P 9.44657 -0.840283 -0.394467 C 4.0555 -0.683104 -1.70917 P 7.5169 3.54172 2.77681 C 3.21536 2.67146 2.20242 P 4.33641 7.62012 ...
0
votes
1answer
48 views

Cutting logs (and not copying) from a log file between two time stamps

I have a very huge access.log file of 2 Gb containing apache access logs. The date time format in that is as below : 04/May/2015:02:15:25 I want to cut (and not just copy) the logs from this ...
0
votes
1answer
45 views

Search and replacing strings in a numerical data file

I have a data file I need to edit it is in the form: ...
0
votes
1answer
69 views

How to parse an awk variable

I'm trying to pull some information out of a log file, looking back over a specific period of time (e.g. the past hour). I've found the following awk construct that works, capturing the data as a ...
0
votes
1answer
85 views

Avoid conversion to scientific notation in awk

I have a csv file that contains hundreds of columns. I would like to subtract the value of a specific column from a static number and then print the entire line. The values of that column are long ...
0
votes
1answer
64 views

Space removal after a pattern in SED

I have a file instrument.xml bash-4.1$ more instrument.xml <instr instr_id="CPT-2380950-011-001-826" valor_ch="" instr_type="CurrentAccount" instr_name="CPT-2380950-011-001-826" qn_type="1" ...
0
votes
1answer
48 views

How do I add a \n to the end of the record on file using AWK

I have a file that can contain 1 or many lines oracle1.txt oracle2.txt oracle3.txt How do I using AWK put a \n after the last line only so it will show oracle1.txt oracle2.txt oracle3.txt\n I ...
0
votes
1answer
98 views

Reading a string till a key word and replacing from there with another string

I am new to UNIX and am practising shell scripting. I have an application where I need string searching and replacement. There are some lines in a file 1 and file 2. I have a search key word. If the ...
0
votes
1answer
174 views

Sum two columns from different files

I have two files like this: File1: -54.005 -67.405 0.1 -54.05 -68.7833 0.2 -54.05 -68.7333 0.3 File2: -54.005 -67.4050 70 12.7 -54.05 -68.7833 167 12.5 -54.05 -68.7333 205 12.6 ...
0
votes
1answer
31 views

putting headers back after awk command

I have file like this. Its called a sam file @HD VN:1.0 SO:unsorted @SQ SN:RNU2-1 LN:388 @SQ SN:RNU6-1100P LN:304 SRR959756.23 0 RNU2-1 56 0 6S32M12S * 0 0 ...
0
votes
1answer
67 views

Replacing code between two JavaScript comments

I need to set a value of JavaScript variable from bash. Variable lives in index.html file and I'd like to use unix SED command to do that. Inside index.html I have it lake this: <script> ...
0
votes
1answer
80 views

Finding words with one vowel group

I am trying to use a command to find all words in a list of words with one group vowels. Like four, for and far, but also words like ore and or. What I got so far: grep -i ...
0
votes
1answer
71 views

How can I use gsub with a pattern? And how to tell it which file to awk?

I've read here that I can replace the values of all columns containing a particular text using awk -F'\t' -vOFS='\t' '{ gsub("CC", "C", $1) ; gsub("AA", "A", $1) ; print }' But where do I tell awk ...
-1
votes
1answer
31 views

Webpage content - search for pattern & save to file

I have a web page, say https://test.salesforce.com. I would like to search for a pattern, e.g. "src" at the beginning of a line in the entire webpage content. If the pattern is found, I need to ...
-2
votes
1answer
59 views

Extract/print every 3rd occurence of pattern

I need help with extracting/printing lines until a stop pattern. This needs to happen after every 3rd occurrence of start pattern, i.e ERROR_FILE_NOT_FOUND and continue until the end of the file. ...
-4
votes
1answer
213 views

Merge two csvfiles by column header

I have two csv files: success.csv "ID","Legacy ID","Field1", "Field2" 1,1111,google,news 2,2222,yahoo,news error.csv "Legacy ID","Field1","Field2","Message" 3333,aol,news,"Failed to upload ...
2
votes
0answers
279 views

extracting data from a log file and plotting it using Gnuplot

I have a logfile with the following data in it: Sat Sep 20 11:25:01 BST 2014 -- temp=53.0'C -- message from script /usr/src/scripts/wifi_test_2.sh Sat Sep 20 11:26:01 BST 2014 -- temp=53.0'C -- ...
1
vote
0answers
24 views

How to use awk-it in Emacs?

I'm a pretty new Emacs user and I want to call awk from Emacs. After googling around I found awk-it at http://www.emacswiki.org/emacs/AWK_It. However I still can't understand how to use it after I ...
0
votes
0answers
68 views

How can I combine information of three files to create a fourth one?

I have three csv files, each of them containing the information I need to get a fourth one. I'm describing toy files but the actual ones contain much more columns and rows, still on the same models. ...
0
votes
0answers
37 views

Output of wc | awk is different in shell script than from cli

When issued via command line, this returns "2": pgrep someprocess | wc | awk '{print $1}' When run in a script like this, it returns "4" #!/bin/bash SOME_PROCESS=`pgrep someprocess | wc | awk ...
0
votes
0answers
32 views

Awk - Take $4$3$2 from Line 01 and $4 from Line 07 and Create a matching key

Any help is greatly appriciated. Desired Matching Key: $4$3$2 from line 01 and $4 from line 07 of A.txt Example: In A.txt $4=5000, $3=68, $2=89 in line 01 and $4=RT0429 or RT0428 or RT0588 in line ...
0
votes
0answers
47 views

Extracting trace information using gawk

I am new to awk programming and am using gawk to extract trace information from a mrt file in order to use it further for analysis. I have successfully been able to extract trace information from a ...
0
votes
0answers
39 views

jumbled character set

In my Linux env, for some unknown! reason when I process a raw unformatted file which I copied from Win7prof (was encoded in utf-16), the resulting file is all jumbled. It will print OK in Linux ...
0
votes
0answers
33 views

to check fields in configuration file

i have configuration file in which first field is filename, second field is the column if we find the value of that column is null in the filename that is given in first column. if any of value of ...
0
votes
0answers
230 views

wget: Retrieving a list of URLs when modifying input data file on the fly

This issue is currently driving me up the wall. It just does not work as it should. I have a file inp with audio samples to download where I preserved the internal ID number by parsing some other ...