awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
3answers
47 views

How to use sed or awk to obtain just a directory

3:ââsdb1 8:17 1 984M 0 part /media/(NAMEOFDRIVEHERE) I want to just have the /media/(NAMEOFDRIVEHERE) location so I can store it in a variable. I've tried some sed commands and have ...
0
votes
1answer
66 views

Cutting logs (and not copying) from a log file between two time stamps

I have a very huge access.log file of 2 Gb containing apache access logs. The date time format in that is as below : 04/May/2015:02:15:25 I want to cut (and not just copy) the logs from this ...
0
votes
2answers
35 views

awk replace specific column in specific line

My data file is sdfsdfsdf 1 0.5000 0.5000 0.5000 8 7 90 135 90 135 1 2 0.5000 0.5000 0.0000 8 7 90 -45 90 -45 1 3 0.0000 0.5000 0.5000 8 7 90 225 90 225 1 4 0.5000 0.0000 0.5000 8 7 90 45 90 ...
1
vote
2answers
49 views

Add string to list using either AWK or SED?

Hi I want to be able to edit the list of name like this using either AWK or SED. Sample input list file: john paul rose lily Desired output: I am john of earth; I am paul of earth; I am rose of ...
3
votes
1answer
58 views

Counting number of entries in column 2 where column 1 is similar

I am trying to count the number of entries in column 2 for the same column1 and also number of times the first entry in column2 for similar column1 occur. For example my input TSV file is: C1 NC01 ...
8
votes
5answers
637 views

Remove duplicate lines from a file that contains a timestamp

This question/answer has some good solutions for deleting identical lines in a file, but won't work in my case since the otherwise duplicate lines have a timestamp. Is it possible to tell awk to ...
1
vote
1answer
37 views

Conditional awk across 4 columns

I have a tab-delimited file with 9 columns as follows: 1 25555818 T 14 0 0 0 14 0:0:0:14 2 25555819 T 14 7 0 0 7 7:0:0:7 3 25555820 G 13 0 0 13 0 ...
0
votes
2answers
54 views

Awk Using NR to output iteration number

I am looping through a list of files, extracting the final line, and printing out columns 8, 9, and 10. I need to also print to the output the 'event number', which is essentially the total number of ...
4
votes
2answers
76 views

Awk inplace option with print statement

In recent version of awk there is a inplace option that can be used to do in place operations similar to sed's -i option. However I can't make it work with print statement. Let's see my example. The ...
1
vote
1answer
70 views

Shell intervals using awk

I have a file a 10:10 10:30 b 12:15 13:15 c 16:54 17:54 and I need to enter interval for example 10:15 so my output should be a 10:10 10:30 How I need to do that using awk?
1
vote
1answer
49 views

delete non important characters from command output

the version command in my Linux machines display the following output version Red.Hat.OS. release..........5.5 APP.Release.............4.2.0.0-02 how to remove the middle "." dot character ...
2
votes
1answer
44 views

How to rsync only a specific list of files to the same location on another server

I have a list of files: /location/file.txt Content of file: /location1/file2.abc /location2/location3/file1.abc /location4/location5/fileX.cde I'd like to copy files to another sever like this: ...
5
votes
6answers
416 views

split file into two parts, at a pattern

How to split a large file into two parts, at a pattern? Given an example file.txt: ABC EFG XYZ HIJ KNL I want to split this file at XYZ such that file1 contains lines up-to XYZ and rest of the ...
1
vote
3answers
41 views

How can I swap the order of each two character pair in a string?

I am trying to automate the swapping of characters 1 and 2, 3 and 4, etc, in a an arbitrary-length string of hex characters. Example: A627E39B becomes: 6A723EB9 It seems like this should be ...
2
votes
2answers
47 views

Replace between patterns that start and end in different lines

I have a file with the following format: Whatever1 Whatever2 Whatever3 binaries=( text1 bin2 ohhh3 ) Whatever4 Whatever5 Whatever6 I need to replace whatever inside lines binaries=( and ) with a ...
2
votes
2answers
107 views

Extract substrings from each line. These substrings are in comma separated, “name=value” format

I have lines in a file that are comma separated. There are no column headers, it's mostly comma separated 'name=value' pairs. Here is some test data: ...
3
votes
4answers
223 views

How to truncate the second column to given length

Given input of the form XY981743 foobarlkasdf saflkas asfZR!sgfad asdSAD asdsadf SAdfasdf46lk lksad bar foolkasjfdrte how can I truncate only the second column? The delimiter is TAB and the ...
0
votes
3answers
62 views

Printing column value in the next line

My input is in TSV format: C1 r1 r2 r3 C2 r1 r3 Desired output is: C1 r1 r2 r3 C2 r1 r3 So, basically what I want is printing value of column 1 in next line, is it ...
1
vote
1answer
60 views

Pattern matching inside awk

I intend to modify the output of ls -la to display the output of file command. The below command works well for the purpose find /opt -type f | xargs ls -let | awk 'BEGIN { OFS="\t" } { sprintf("file ...
0
votes
1answer
59 views

Solaris: How to run a system command on top of awk and combine all output in a single line

Have this peculiar requirement to print files in a given directory in the below mentioned format filename1 <type of file> owner group filename2 <type of file> owner ...
0
votes
1answer
126 views

Is there a way to use sed or awk to filter data inside a zipped file? [duplicate]

Is there a way to use sed or awk to filter data inside a zipped file? When I run a sed or awk command and send to stdout the file created is 0 bytes.
1
vote
2answers
33 views

Using Unix, when detecting a pattern, we need to detect its corresponding other pattern

When detecting the line containing the pattern night, we need to detect its corresponding line containing the other pattern === which is just occurring at a higher line as shown below. File content: ...
1
vote
1answer
40 views

Insert second file contents using awk

I am trying to append the whole of second file contents (it is a log) in XML <log> tag using awk. I cannot combine both input files into one since I am using a FS="\n" RS="\n\n" which is suited ...
0
votes
1answer
58 views

Help in manipulating a test file

I have a file look like aaa/bbb/ccc.bbb aaa/bbb/ddd.bbb aaa/eee/fff.eee aaa/eee/ggg.eee I would like to skip lines ending similarly (where ending is defined by the extension, e.g. .bbb & .bbb) ...
36
votes
4answers
4k views

Reverse grepping

Let's say, I have a really big text file (about 10.000.000 lines). I need to grep it from the end and save result to a file. What's the most efficient way to accomplish task?
0
votes
1answer
30 views

zsh alias with linefeeds, commas and quotes

In my zshrc I have an alias like this: alias sl='screen -list' It fits my need (seeing what screens I have running), but the output is rather ugly: [pts/7]~% sl There are screens on: ...
1
vote
2answers
79 views

AWK script to compare content of 2 files

I need experts help in resolving my following problem ... I have 2 files .. file1: abc|123|check def|456|map ijk|789|globe lmn|101112|equator file2: check map equator globe AWK function ...
3
votes
1answer
44 views

Solaris: PKG - Script To Verify All Packages Except for a Few

I'm trying to verify the all packages except for a pre-defined list of packages that I know are going to fail for known reasons. This script is going to be run on all Solaris systems within our ...
0
votes
3answers
46 views

Read line from file and add that to the same line of another file

I have one file like this: aaa bbbb c And another one like this: 1 1241 2 301094209 3 432423 I want this: 1 1241 aaa 2 301094209 bbbb 3 432423 c so I can plot it with Gnuplot. If you ...
0
votes
4answers
109 views

What the equivalent of “grep | cut” using sed or awk?

Say I had a config file /etc/emails.conf email1 = user@dinkum.dorg email2 = user@winkum.worg email3 = user@stinkum.storg and I wanted to get email2 I could do a: grep email2 /etc/emails.conf | ...
1
vote
2answers
166 views

How would I use sed or awk to extract certain lines in a file

If I wanted to search for all lines in a file that start with a date, such as "May 1", how would I do that with sed or awk? I need to extract this data and either send to screen or a file. Thanks.
-1
votes
2answers
49 views

Extracting certain data within a script output file

We have a script that runs several commands and rolls up all the output and inserts in into a log file. One of the commands it runs is "netstat -anp". I'd like to be able to search this file for all ...
2
votes
3answers
5k views

Awk printf number in width and round it up

I need to printf a number out, but with given width and rounded (with awk!) %10s I have this and somehow I need to connect the %d but everything I do, ends up with too much parametres for awk ...
1
vote
2answers
51 views

Using Awk to Search Bro Log for IPs/Keywords Found in Online Text File

I am examining Bro logs and would like to find a way to use AWK to grab IPs from a tracker link and search the Bro logs for those IPs automatically, printing the results of each match into ...
1
vote
1answer
51 views

Passing variables to range patterns in awk

I'm trying to use range patterns in awk: grep -a volume somefile | awk '/^Apr 25 23:44:04*/,/^Apr 26 12:44:01*/ {print}' This works fine, but when I try to execute it by putting the date and time ...
2
votes
1answer
80 views

Filter a log file by month, date and time in bash

I'd like to fetch the data between Apr 24 10:00:00.000000 and Apr 25 24:00:00.999999 in the following log but I am not sure how to get this work: files/file1:Apr 22 02:47:00.663117 somedata ...
0
votes
1answer
37 views

Filtering Log Output on Separate Lines

I did a recursive grep in a directory to find all files that would match as follows grep -ER "match_string1|match_string2" /path/to/dir/ What I get as output is: ...
1
vote
2answers
52 views

Is there a way to limit the range of the range pattern in awk?

I am trying to use the awk range pattern to find all SQL select statements in a group of files, inspired by this stackoverflow answer. From the awk manual: The pattern1, pattern2 form of an ...
1
vote
2answers
66 views

awk/grep to match word in two files [duplicate]

I have two files: file1: 905894 1197693 3703749 file2: 905894 Test1 1197693 Test2 3703749 Test3 I want to get the values in the second column if elements in file 1 column 1 matches any of ...
1
vote
1answer
39 views

Synchronize columns of two matrices

Please guide on how to solve this problem. I have pairs of files which I need to modify so that they have the same common columns in the same sequence. If my files are File1 and File2 as follows ...
7
votes
5answers
1k views

Joining two files with unique identifier

I have two files with approximately 12900 and 4400 entries respectively, that I want to join. The files contain location information for all landbased weather observing stations around the globe. The ...
5
votes
2answers
3k views

Fast elimination of duplicate lines across multiple files

I have a huge amount of data in which each (data-)line should be unique. There are a lot of files in one folder in which this is already true. It is about 15GB splitted into roughly 170 files with ...
0
votes
1answer
154 views

AWK is throwing an error “unexpected newline or end of string”

My Bash script: declare -a lang=('english' 'spainsh') sms="Free msg: Due to upgrades on apps you’ll need a new version by yyyy/mm/dd to app running. Visit a store or go to" url="google.com/UA2392 ...
1
vote
1answer
76 views

Awk Command vs Grep

I had a question regarding the awk command. I want to take a bunch of files and find a single line from each and extract them into a comma seperated text file such that I can import it into excel for ...
6
votes
6answers
256 views

How can I avoid escaped delimiter characters in a Unix command?

I need to take the sum of a field which is tilde (~) delimited. Problem I have is that my data is also having delimiter escaped. Example 1~CEO~ashok\~kumar~1000 As we see in 3rd field above we ...
1
vote
2answers
308 views

Compare two file and return a true or false value

I have two files with one column. If the first rows in the two files are same the print 1 otherwise print 0. file1: M A A M file2: M M A A output: 1 0 1 0 Can anybody tell me the awk script? ...
1
vote
1answer
493 views

Match first fields of two tab separated files and print matching values

I have two files. Match the 1st column of first file to 1st column of second file and print entire line of the second file. Second file is a <tab> separated file. I have tried many awk one ...
3
votes
2answers
141 views

How can I compare two files based on the value of the first column?

I want to compare two files based on the value of their first column: file1 aaa 23 3 bbb 56 6 ccc 45 2 file2 bbb m a aaa a m ccc m m The output file should be: aaa a m bbb m a ccc m m ...
8
votes
2answers
5k views

Conditional block vs conditional statement (if)

Say I have a file: PRO 1 GLN 5.55112e-17 ILE -6.245e-17 THR 5.55112e-17 I want every line that has a number unequal to 1 in the second column to change it to 0 and keep the rest. If I use if (i.e ...
4
votes
4answers
335 views

Printing selective fields following 1st column instructions

Suppose I have a file in which I'd like to simultaneously print different awk commands following the first column instuctions, without messing up the original file (as it would happen with two ...