awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
1answer
78 views

Splitting Apache Vhosts into Individual Files

I have several vhosts in one file that I am looking to break each into a file of their own corresponding to the ServerName. I'm looking to take this file <VirtualHost *:80> ServerName ...
3
votes
2answers
1k views

Removing unwanted lines in text file

All I need is the sed or awk code to remove the even numbered lines in a text file. Nothing fancy I just need to remove irrelevant data on the even lines of a text file.
3
votes
7answers
1k views

Problem in splitting a string using awk or cut command

I have a properties file which has a password field (key value pair) PASSWORD=NDhhHcsOBofXUdUzGw5B0Q== I am trying to get the value in my shell script using the awk command password=`awk -F "=" ...
2
votes
2answers
2k views

Comparing two files using Unix and Awk

I need to compare two files, File1 and File2 (Separated by space) using 4 fields (Field 1, 2, 4 and 5 of File 1 with field1, 2, 4 and 5 of File2). Logic: If column 1, 2 and 4 of File 1 matches with ...
2
votes
6answers
165 views

Bash - Split the previous commands parameters

How would one parse the previous command in bash? Example: root$ ssh foo@1.2.3.4 root$ echo !$ &>/tmp/foo.txt root$ cat /tmp/foo.txt foo@1.2.3.4 Goal: Send just the ipaddress 1.2.3.4 to ...
1
vote
3answers
150 views

Cat 2 files and avoid duplication of lines

I have the file1 with the following content {"name":"clio5", "value":"13"} {"name":"citroen_c4", "value":"23"} {"name":"citroen_c3", "value":"12"} {"name":"golf4", "value":"16"} {"name":"golf3", ...
1
vote
1answer
63 views

How to design the table in awk?

My input file is file1 count month 505 Apr-2010 252 Mar-2010 426 Feb-2010 201 Jan-2010 211 Dec-2009 Same file I want the output like this below +-------+----------+ | count | month ...
1
vote
1answer
86 views

Compare the similar file and not similar files display below output in awk?

I'd like to compare the following 2 files and display a comparison of the 2 files with each other. File #1: DATE DS 2012-08-02 1 2013-06-23 1 2013-06-27 2 2013-06-28 2 2013-06-29 779 File ...
1
vote
2answers
69 views

How to use AWK record separator?

Here is the input: * true, false, yes, no, always, never. * ******************************************************************************* default: su = true ... maxexpired = ...
1
vote
3answers
2k views

Count number of lines with a certain value in a column for all files in directory recursively

I have 40 files in a directory and I want to count the number of times there is a line with a "2" in the first column in each file individually. I am trying something like this, but it prints out ...
1
vote
2answers
206 views

Are there versions of Unix that don't have awk in default install?

Are there versions of Unix that don't have awk installed by default? I am fine with an answer containing some really old outdated Unix variant which predates awk's creation; but would ideally prefer ...
-1
votes
2answers
52 views

Line match to trigger specific argument

I have a file with 2 types of contents as below. QJKLJFLHLKJDGFIUD 03:12:23 JHKJFDGDSFGDFGHDF 03:15:45 HJIK ASDSdSADSADSSADSA 03:30:34 DFDF KWQA I have 2 binaries. One is on the local ...
4
votes
3answers
190 views

Check for, and add, missing timestamps to individual lines in a file

I have a file in the following format: ABCD 01206001022T01YA022T01YA022T07SO 09:20:38 ABCD 01206001022ACION 09:24:40 ABCD 04006001021S01UK 09:24:42 ABCD 7878696621321312 23213213213213 ABCD ...
1
vote
6answers
2k views

Repeat each line multiple times

Would like to have each line in a file repeated a fixed number of times. e.g. have each line repeated four times: a b c becomes: a a a a b b b b c c c c I've done some searching, and there are ...
1
vote
5answers
399 views

delete lines in a csv file older than 7 days

I have a csv that I need to remove all lines that are older than 7 days. This is the format of the csv Person ID VIP CS SS LT FTLT PS Modified Datestamp T001028 1 1 1 0 0 0 ...
2
votes
4answers
1k views

Split long output into two columns

Is there a simple utility or script to columnate the output from one of my scripts? I have data in some form: A aldkhasdfljhaf B klajsdfhalsdfh C salkjsdjkladdag D lseuiorlhisnflkc E sdjklfhnslkdfhn ...
9
votes
8answers
1k views

How to remove multiple newlines at EOF?

I have files that end in one or more newlines and should end in only one newline. How can I do that with Bash/Unix/GNU tools? Example bad file: 1\n \n 2\n \n \n 3\n \n \n \n Example corrected ...
-1
votes
2answers
1k views

How can I print unique line base on particular column value using awk? [closed]

I have a comma separated file with multiple fields. Example: candidate1,12,56,ATTG candidate2,45,90,ATTG candidate3,90,140,ATTG candidate4,2,36,AGCT So, does anyone know the answer?
0
votes
4answers
3k views

Extract substring using regular expression on a Unix file

I have file with below contents . /ABC/RTE/AD_900_VOP_123/OPP /ABC/RTE/TRE/AD_900_VOP_145/BBB /ABC/RTE/AN_900_VFP_124/FBF /ABC/RTE/HD_900_FOP_153/WEW /ABD/RDV/AD_900_VOP_123/OPP ...
1
vote
2answers
631 views

Use FOR Loop inside END block of awk command

I'm trying to find the average number of files per day. The following script works: #!/bin/sh ls -l | grep "^-" | awk '{ key=$6$7 freq[key]++ } END { for (date in freq) printf ...
4
votes
3answers
811 views

Change the text on a single line number in multiple text files

Suppose that I have ten bash shell scripts: script1.sh, script2.sh, ..., script10.sh. Initially, all ten files are identical. Now, I would like to make two changes in each script: In each file, ...
1
vote
1answer
308 views

awk can't print array element

Lets sat my file1 is: A S B Q file2 is: B 2 I want to match $1 from file2 with $1 file1 and print file1 $2 && file2 $2, for the output like this: Q 2 My awk solution ...
2
votes
2answers
86 views

Print line n-1 (and n) if line n fulfills condition

I have a fasta file looking something like this: >19 dna:chromosome chromosome:NCBIM37:19:1:61342430:1 37330562 ID13406 U - TACATGGGTGACAAGGAGGTCAGGTGTTA >19 dna:chromosome ...
5
votes
1answer
2k views

Removing leading zeros from DATE

I made an alias of the date command to display date in the following format: 2013.06.14.12.10.02 using this command: alias date = date +"%Y.%m.%d.%H.%M.%S" Everything works great, except I ...
1
vote
2answers
89 views

Grep and sort several unsorted things in several lines (in one document)

I have a file with several unsorted things in one line which I want to put into a new file This is an example of part of the file I have: X1314448: SaMi|SM_g2554.t1 SaMi|SM_g5072.t1 Des|Des_g3808.t1 ...
4
votes
2answers
353 views

Use sed or awk to join lines in a certain pattern?

Not sure on how to turn this input: A B C apartner bpartner cpartner into this output: A apartner B bpartner c cpartner I have this pattern in a 80 line file that needs to be 40 line. Also, how ...
4
votes
2answers
172 views

AWK - a question about columns

I have a question. I was trying to deal with it by myself, but it seems like I am too new in awk to make it work. Let's assume that we have a file (eg. database.txt) (values are tab-separated): ...
2
votes
3answers
134 views

Delete lines containing repeated text

I have a file containing two paths on each line. I want to remove the lines containing the same path twice. I work on Linux and Solaris. I would like a one-liner in sed or awk or perl. Example input ...
2
votes
3answers
859 views

I want to find duplicates in a file and add a character to the end of the line on the 1st match

I am trying to find duplicates in a file and once a match is found mark the 1st match with a character or word on the end of the line. eg my file (test.html) contains the following entries host= ...
2
votes
2answers
2k views

Replacing text between two HTML Comments

I am new to the world of sed/awk and regex in general and have been studying their usage, but have been floundering trying to meet my need: I have an htm page that has a single line notice that needs ...
2
votes
3answers
185 views

Text to yaml conversion

I have a tab separated text file in this format: your-email your-order-id PayPal-transaction-id your-first-name your-second-name s@ess.com 12345 54321 sooky spooky t@tee.com 23456 ...
4
votes
2answers
792 views

Inotifywait for large number of files in a directory

What I want to do is, to monitor a directory (not recursive, just one) for new files created and append those files to one single big file as they are being written. The number of files that are ...
2
votes
3answers
2k views

Easy way to parse syslog date format

I'd like to write a script that analyses the last week's syslog. But my syslog logs in a stupid date format, e.g. "May 22". If it logged as 2013-05-22, this task would be trivial. Is there a way ...
1
vote
1answer
40 views

Matching hexadecimals with gawk

I have a binary file with the following content (printed in hexadecimal): $ xxd -p virus.com 5669727573b440bb0100b90500ba0000cd21 I can match the first 5 letters (which are the alphanumeric string ...
5
votes
2answers
203 views

awk + paste for cleaning up PATH?

I have seen this code in .cshrc init files on a few machines. I went through a few awk tutorials in trying to understand how it works, but I am still unable to decrypt it. setenv PATH `echo $PATH | ...
1
vote
3answers
470 views

awk a specific column with unexpected line break

I can't seem to find a solution for this and it's driving me crazy. I know I can use awk to print a column(s). I'm having trouble printing a specific column though because of the way my file system is ...
2
votes
3answers
62 views

Replace certain strings with value from former line

I have the following text: Source IP,URL 1.1.1.1,example1.com Blank,example2.com 2.2.2.2,example3.com Blank,example4.com Blank,example5.com Blank,example6.com Blank,example7.com 7.7.7.7,example8.com ...
3
votes
3answers
158 views

sed + how to remove character/s that start or ended on each number

How do I remove the . character(s) that start in the beginning of each number or end on each number? Remark – perl one liner also good alternative for sed. Example input: .23.12.44.5. .233.3.3.3 ...
3
votes
3answers
165 views

checking data in columns when a data or some may be missing or present?

I am not sure if this possible. say i have columns like : Team Colour Game Rainfall PlayerName XYZ Blue Cricket Yes Kapil suppose i need to search ...
2
votes
2answers
92 views

confuse with awk user define function

I have solved Projecteuler Problem 3, but the thing is that I am not able to create the function in awk. I have tried this working code (without function): #!/usr/bin/awk -f BEGIN{ ...
1
vote
1answer
551 views

Using awk to parse AIX's lsuser output in stanza format

I'm creating a simple awk script to parse AIX's lsuser -f ALL output and return information in one line. A sample of the output is something like this. root: id=0 pgrp=something ...
2
votes
6answers
172 views

How can I separate these two outputs using awk command

Here's my output: MessageID=3990592283244651750-30192b51.13df831d93a.7eb4;EsbTiming=2013-04-12T01:07:46.099 I want only the time. How can I separate them?
0
votes
1answer
177 views

moving columns across rows

I am dealing with a txt file and trying to keep the first column/field in place, but shift the 2nd column down by 1 row. I tried this, but it gave the error unmatched .'s. awk '{for(i=1; i<NR; ...
3
votes
3answers
4k views

How can I use SED or AWK to replace placeholders in a template file with variable content that contains special characters?

CentOS 6.3 I'm trying to get a small script to send an email containing a copy of email headers in the body (for the purpose of internal reporting). The template file contains the following: ...
4
votes
3answers
2k views

Remove string from a particular field using awk/sed

I have a file (>80,000 lines) that looks likes this: chr1 GTF2GFF chromosome 1 249213345 . . . ID=chr1;Name=chr1 chr1 GTF2GFF gene 11874 14408 . + . ...
0
votes
1answer
741 views

comparing lines with awk vs while read line

I have two files one with 17k lines and another one with 4k lines. I wanted to compare position 115 to position 125 with each line in the second file and if there is a match, write the entire line ...
3
votes
2answers
235 views

Automated partition selection in bash script

I'm trying to write a bash script to run from Linux on a CD or flashdrive that can mount Windows and change stuff, but I don't know how to go about detecting which partition/drive is Windows. My best ...
2
votes
2answers
1k views

Using bash shell function inside AWK

Is it possible to use bash function inside AWK somehow? Example file (string, int, int, int) Mike 247808 247809 247810 Trying to convert values from decimal to hexadecimal. Function defined ...
4
votes
1answer
764 views

Pass colors from ls through pipe to awk 'print' statement

This is a follow-up to my question from yesterday, Show sum of file sizes in directory listing. Thanks to Zero Piraeus and a point in the right direction by Mauritz Hansen, I now have function ...
3
votes
1answer
451 views

Output of `java -version` not matched by grep or awk

On my Linux machine, it isn't clear to me why if I do the following then I don't get only the version string ("1.5.0_32"). # java -version | grep version | awk '{print $NF}' java version "1.5.0_32" ...