awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

4
votes
2answers
65 views

Awk inplace option with print statement

In recent version of awk there is a inplace option that can be used to do in place operations similar to sed's -i option. However I can't make it work with print statement. Let's see my example. The ...
1
vote
1answer
60 views

Shell intervals using awk

I have a file a 10:10 10:30 b 12:15 13:15 c 16:54 17:54 and I need to enter interval for example 10:15 so my output should be a 10:10 10:30 How I need to do that using awk?
1
vote
1answer
46 views

delete non important characters from command output

the version command in my Linux machines display the following output version Red.Hat.OS. release..........5.5 APP.Release.............4.2.0.0-02 how to remove the middle "." dot character ...
2
votes
1answer
34 views

How to rsync only a specific list of files to the same location on another server

I have a list of files: /location/file.txt Content of file: /location1/file2.abc /location2/location3/file1.abc /location4/location5/fileX.cde I'd like to copy files to another sever like this: ...
5
votes
6answers
306 views

split file into two parts, at a pattern

How to split a large file into two parts, at a pattern? Given an example file.txt: ABC EFG XYZ HIJ KNL I want to split this file at XYZ such that file1 contains lines up-to XYZ and rest of the ...
1
vote
3answers
37 views

How can I swap the order of each two character pair in a string?

I am trying to automate the swapping of characters 1 and 2, 3 and 4, etc, in a an arbitrary-length string of hex characters. Example: A627E39B becomes: 6A723EB9 It seems like this should be ...
2
votes
2answers
41 views

Replace between patterns that start and end in different lines

I have a file with the following format: Whatever1 Whatever2 Whatever3 binaries=( text1 bin2 ohhh3 ) Whatever4 Whatever5 Whatever6 I need to replace whatever inside lines binaries=( and ) with a ...
2
votes
2answers
90 views

Extract substrings from each line. These substrings are in comma separated, “name=value” format

I have lines in a file that are comma separated. There are no column headers, it's mostly comma separated 'name=value' pairs. Here is some test data: ...
3
votes
4answers
206 views

How to truncate the second column to given length

Given input of the form XY981743 foobarlkasdf saflkas asfZR!sgfad asdSAD asdsadf SAdfasdf46lk lksad bar foolkasjfdrte how can I truncate only the second column? The delimiter is TAB and the ...
0
votes
3answers
58 views

Printing column value in the next line

My input is in TSV format: C1 r1 r2 r3 C2 r1 r3 Desired output is: C1 r1 r2 r3 C2 r1 r3 So, basically what I want is printing value of column 1 in next line, is it ...
1
vote
1answer
50 views

Pattern matching inside awk

I intend to modify the output of ls -la to display the output of file command. The below command works well for the purpose find /opt -type f | xargs ls -let | awk 'BEGIN { OFS="\t" } { sprintf("file ...
0
votes
1answer
43 views

Solaris: How to run a system command on top of awk and combine all output in a single line

Have this peculiar requirement to print files in a given directory in the below mentioned format filename1 <type of file> owner group filename2 <type of file> owner ...
3
votes
2answers
838 views

Split log file by time range

I have a log file and would like to cut it into copies of 15min. In the log file the lines start with: dd mmm yyyy hh:mm:ss,xxx e.g. 12 Feb 2013 16:05:02,xxx log text... and the file rotates ...
0
votes
1answer
86 views

Is there a way to use sed or awk to filter data inside a zipped file? [duplicate]

Is there a way to use sed or awk to filter data inside a zipped file? When I run a sed or awk command and send to stdout the file created is 0 bytes.
1
vote
2answers
32 views

Using Unix, when detecting a pattern, we need to detect its corresponding other pattern

When detecting the line containing the pattern night, we need to detect its corresponding line containing the other pattern === which is just occurring at a higher line as shown below. File content: ...
3
votes
3answers
78 views

awk ifs and variables - cannot pass a variable from one line towards subsequent lines

First of all I'm new to awk so please excuse if it's something simple. I'm trying to generate a file that contains paths. I'm using for this an ls -LT listing as well as an awk script: This is an ...
1
vote
1answer
38 views

Insert second file contents using awk

I am trying to append the whole of second file contents (it is a log) in XML <log> tag using awk. I cannot combine both input files into one since I am using a FS="\n" RS="\n\n" which is suited ...
0
votes
1answer
55 views

Help in manipulating a test file

I have a file look like aaa/bbb/ccc.bbb aaa/bbb/ddd.bbb aaa/eee/fff.eee aaa/eee/ggg.eee I would like to skip lines ending similarly (where ending is defined by the extension, e.g. .bbb & .bbb) ...
36
votes
4answers
3k views

Reverse grepping

Let's say, I have a really big text file (about 10.000.000 lines). I need to grep it from the end and save result to a file. What's the most efficient way to accomplish task?
0
votes
1answer
24 views

zsh alias with linefeeds, commas and quotes

In my zshrc I have an alias like this: alias sl='screen -list' It fits my need (seeing what screens I have running), but the output is rather ugly: [pts/7]~% sl There are screens on: ...
1
vote
2answers
70 views

AWK script to compare content of 2 files

I need experts help in resolving my following problem ... I have 2 files .. file1: abc|123|check def|456|map ijk|789|globe lmn|101112|equator file2: check map equator globe AWK function ...
3
votes
1answer
38 views

Solaris: PKG - Script To Verify All Packages Except for a Few

I'm trying to verify the all packages except for a pre-defined list of packages that I know are going to fail for known reasons. This script is going to be run on all Solaris systems within our ...
0
votes
3answers
45 views

Read line from file and add that to the same line of another file

I have one file like this: aaa bbbb c And another one like this: 1 1241 2 301094209 3 432423 I want this: 1 1241 aaa 2 301094209 bbbb 3 432423 c so I can plot it with Gnuplot. If you ...
0
votes
4answers
69 views

What the equivalent of “grep | cut” using sed or awk?

Say I had a config file /etc/emails.conf email1 = user@dinkum.dorg email2 = user@winkum.worg email3 = user@stinkum.storg and I wanted to get email2 I could do a: grep email2 /etc/emails.conf | ...
1
vote
2answers
144 views

How would I use sed or awk to extract certain lines in a file

If I wanted to search for all lines in a file that start with a date, such as "May 1", how would I do that with sed or awk? I need to extract this data and either send to screen or a file. Thanks.
-1
votes
2answers
32 views

Extracting certain data within a script output file

We have a script that runs several commands and rolls up all the output and inserts in into a log file. One of the commands it runs is "netstat -anp". I'd like to be able to search this file for all ...
2
votes
3answers
4k views

Awk printf number in width and round it up

I need to printf a number out, but with given width and rounded (with awk!) %10s I have this and somehow I need to connect the %d but everything I do, ends up with too much parametres for awk ...
1
vote
2answers
37 views

Using Awk to Search Bro Log for IPs/Keywords Found in Online Text File

I am examining Bro logs and would like to find a way to use AWK to grab IPs from a tracker link and search the Bro logs for those IPs automatically, printing the results of each match into ...
1
vote
1answer
36 views

Passing variables to range patterns in awk

I'm trying to use range patterns in awk: grep -a volume somefile | awk '/^Apr 25 23:44:04*/,/^Apr 26 12:44:01*/ {print}' This works fine, but when I try to execute it by putting the date and time ...
2
votes
1answer
62 views

Filter a log file by month, date and time in bash

I'd like to fetch the data between Apr 24 10:00:00.000000 and Apr 25 24:00:00.999999 in the following log but I am not sure how to get this work: files/file1:Apr 22 02:47:00.663117 somedata ...
0
votes
1answer
25 views

Filtering Log Output on Separate Lines

I did a recursive grep in a directory to find all files that would match as follows grep -ER "match_string1|match_string2" /path/to/dir/ What I get as output is: ...
1
vote
2answers
47 views

Is there a way to limit the range of the range pattern in awk?

I am trying to use the awk range pattern to find all SQL select statements in a group of files, inspired by this stackoverflow answer. From the awk manual: The pattern1, pattern2 form of an ...
1
vote
2answers
50 views

awk/grep to match word in two files [duplicate]

I have two files: file1: 905894 1197693 3703749 file2: 905894 Test1 1197693 Test2 3703749 Test3 I want to get the values in the second column if elements in file 1 column 1 matches any of ...
1
vote
1answer
37 views

Synchronize columns of two matrices

Please guide on how to solve this problem. I have pairs of files which I need to modify so that they have the same common columns in the same sequence. If my files are File1 and File2 as follows ...
6
votes
5answers
1k views

Joining two files with unique identifier

I have two files with approximately 12900 and 4400 entries respectively, that I want to join. The files contain location information for all landbased weather observing stations around the globe. The ...
5
votes
2answers
2k views

Fast elimination of duplicate lines across multiple files

I have a huge amount of data in which each (data-)line should be unique. There are a lot of files in one folder in which this is already true. It is about 15GB splitted into roughly 170 files with ...
0
votes
1answer
65 views

AWK is throwing an error “unexpected newline or end of string”

My Bash script: declare -a lang=('english' 'spainsh') sms="Free msg: Due to upgrades on apps you’ll need a new version by yyyy/mm/dd to app running. Visit a store or go to" url="google.com/UA2392 ...
1
vote
1answer
61 views

Awk Command vs Grep

I had a question regarding the awk command. I want to take a bunch of files and find a single line from each and extract them into a comma seperated text file such that I can import it into excel for ...
6
votes
6answers
236 views

How can I avoid escaped delimiter characters in a Unix command?

I need to take the sum of a field which is tilde (~) delimited. Problem I have is that my data is also having delimiter escaped. Example 1~CEO~ashok\~kumar~1000 As we see in 3rd field above we ...
1
vote
2answers
297 views

Compare two file and return a true or false value

I have two files with one column. If the first rows in the two files are same the print 1 otherwise print 0. file1: M A A M file2: M M A A output: 1 0 1 0 Can anybody tell me the awk script? ...
1
vote
1answer
482 views

Match first fields of two tab separated files and print matching values

I have two files. Match the 1st column of first file to 1st column of second file and print entire line of the second file. Second file is a <tab> separated file. I have tried many awk one ...
2
votes
2answers
130 views

How can I compare two files based on the value of the first column?

I want to compare two files based on the value of their first column: file1 aaa 23 3 bbb 56 6 ccc 45 2 file2 bbb m a aaa a m ccc m m The output file should be: aaa a m bbb m a ccc m m ...
8
votes
2answers
4k views

Conditional block vs conditional statement (if)

Say I have a file: PRO 1 GLN 5.55112e-17 ILE -6.245e-17 THR 5.55112e-17 I want every line that has a number unequal to 1 in the second column to change it to 0 and keep the rest. If I use if (i.e ...
2
votes
2answers
752 views

Merging columns from two separate files

How to create a new file merging selective columns from two separate files using awk? Without messing up the elements orders of BOTH files. Exemple: File 3 may contain column 1,2,3 from File 1 and ...
4
votes
4answers
320 views

Printing selective fields following 1st column instructions

Suppose I have a file in which I'd like to simultaneously print different awk commands following the first column instuctions, without messing up the original file (as it would happen with two ...
1
vote
2answers
101 views

Search for a string in a line and append related lines

I have a file containing some ID (identifier) and DS (description) lines. I'd like to keep the rows containing the string value in their ID line, as well as all the following (related) DS lines before ...
4
votes
3answers
586 views

pipe line with non-deterministic output

I have happened upon a command that sometimes works and sometimes does not, even when executed multiple times in rapid succession in a bash shell (I have not tested the behavior in other shells). The ...
2
votes
4answers
454 views

Insert new line after non matching pattern

I have following output from script host1:101 host1:102 host2:101 host2:102 host2:103 host4:101 host5:101 host5:102 host5:103 host5:104 etc.. I wanted to insert a new line after each host group. ...
3
votes
4answers
306 views

Count how many elements in the nth column in a whole file

Input file: A 1,2,3,4 #length($2)=4 B 1,2 #length($2)=2 C 9,8,7,6,5,4 #length($2)=6 Expected output: 12 #4+2+6 A method like: awk -F '[\t,]' '{print length($2)}' but ...
1
vote
3answers
43 views

Concatenate Sequences under a Heading

I have a file like File1. With a heading for example ">Contig1" and I want to concatenate all the sequences under the respective heading. File1: >Contig1 ACCCCATACATCCAAA NNNNNNNNNNCATACA ...