awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

2
votes
1answer
2k views

Diff the output of two `awk` commands

I'm trying to compute the difference between the output of two awk commands but my simple attempts at it seem to be failing. Here is what I'm trying: diff $(awk '{print $3}' f1.txt | sort -u) $(awk ...
5
votes
1answer
3k views

Sorting a file based on one column using Unix and Awk

I need to sort the input file according the 6th column, which is the score. Input File: Sc2/80 20 . A T 86 Pass N=2 F=5;U=4 Sc2/80 20 . A C 80 Pass N=2 F=5;U=4 Sc2/60 55 . G T 90 Pass N=2 F=5;U=4 ...
1
vote
3answers
864 views

How to limit MAX an MIN length of a string?

"nothing" means no OUTPUT. Q: how can I limit the MIN and MAX length of a piped STRING? (MIN: x>=3 chars, MAX: x<=32 chars) INPUT: echo 12 | MAGIC OUTPUT: nothing INPUT: echo 123 | ...
1
vote
1answer
55 views

How to make URL's out of this text with least resources?

INPUT: nyeeeeeeeeeeeeeee.txt:sadfsadf sadjflsajfd:jasdlfjalsfd foo/asdfsafd.txt:laksjdlfajsfd OUTPUT: <a href="nyeeeeeeeeeeeeeee.txt">nyeeeeeeeeeeeeeee.txt</a>:sadfsadf <a ...
2
votes
1answer
2k views

How to integrate a multiline awk script in a shell script

I am a beginner in shell scripting and my question is a continuation of How to parse a file to extract 3 digits numbers kept in a "group number" I am trying to integrate in a single shell ...
7
votes
1answer
806 views

grep: display filename once, then display context with line numbers

Our source code has error codes scattered throughout. Finding them is easy with grep, but I'd like a bash function find_code that I can execute (eg. find_code ####) which will provide output along ...
2
votes
3answers
121 views

Pulling out keywords using standard command line utilities?

As an example, this is the pronunciation of "when" according to Wiktionary. enPR, IPA and X-SAMPA are different schemes for showing pronunciations. when:* {{a|US}} {{enPR|wĕn|hwĕn}}, ...
0
votes
1answer
111 views

Splitting Apache Vhosts into Individual Files

I have several vhosts in one file that I am looking to break each into a file of their own corresponding to the ServerName. I'm looking to take this file <VirtualHost *:80> ServerName ...
3
votes
7answers
4k views

Problem in splitting a string using awk or cut command

I have a properties file which has a password field (key value pair) PASSWORD=NDhhHcsOBofXUdUzGw5B0Q== I am trying to get the value in my shell script using the awk command password=`awk -F "=" ...
2
votes
2answers
4k views

Comparing two files using Unix and Awk

I need to compare two files, File1 and File2 (Separated by space) using 4 fields (Field 1, 2, 4 and 5 of File 1 with field1, 2, 4 and 5 of File2). Logic: If column 1, 2 and 4 of File 1 matches with ...
2
votes
6answers
188 views

Bash - Split the previous commands parameters

How would one parse the previous command in bash? Example: root$ ssh foo@1.2.3.4 root$ echo !$ &>/tmp/foo.txt root$ cat /tmp/foo.txt foo@1.2.3.4 Goal: Send just the ipaddress 1.2.3.4 to ...
1
vote
3answers
189 views

Cat 2 files and avoid duplication of lines

I have the file1 with the following content {"name":"clio5", "value":"13"} {"name":"citroen_c4", "value":"23"} {"name":"citroen_c3", "value":"12"} {"name":"golf4", "value":"16"} {"name":"golf3", ...
9
votes
3answers
4k views

Why does awk do full buffering when reading from a pipe

I'm reading from a serial port connected to a gps device sending nmea strings. A simplified invocation to illustrate my point: $ awk '{ print $0 }' /dev/ttyPSC9 ...
1
vote
1answer
71 views

How to design the table in awk?

My input file is file1 count month 505 Apr-2010 252 Mar-2010 426 Feb-2010 201 Jan-2010 211 Dec-2009 Same file I want the output like this below +-------+----------+ | count | month ...
1
vote
1answer
109 views

Compare the similar file and not similar files display below output in awk?

I'd like to compare the following 2 files and display a comparison of the 2 files with each other. File #1: DATE DS 2012-08-02 1 2013-06-23 1 2013-06-27 2 2013-06-28 2 2013-06-29 779 File ...
1
vote
2answers
93 views

How to use AWK record separator?

Here is the input: * true, false, yes, no, always, never. * ******************************************************************************* default: su = true ... maxexpired = ...
1
vote
3answers
4k views

Count number of lines with a certain value in a column for all files in directory recursively

I have 40 files in a directory and I want to count the number of times there is a line with a "2" in the first column in each file individually. I am trying something like this, but it prints out ...
2
votes
6answers
6k views

Repeat each line multiple times

Would like to have each line in a file repeated a fixed number of times. e.g. have each line repeated four times: a b c becomes: a a a a b b b b c c c c I've done some searching, and there are ...
1
vote
2answers
279 views

Are there versions of Unix that don't have awk in default install?

Are there versions of Unix that don't have awk installed by default? I am fine with an answer containing some really old outdated Unix variant which predates awk's creation; but would ideally prefer ...
11
votes
8answers
3k views

How to remove multiple newlines at EOF?

I have files that end in one or more newlines and should end in only one newline. How can I do that with Bash/Unix/GNU tools? Example bad file: 1\n \n 2\n \n \n 3\n \n \n \n Example corrected ...
-1
votes
2answers
55 views

Line match to trigger specific argument

I have a file with 2 types of contents as below. QJKLJFLHLKJDGFIUD 03:12:23 JHKJFDGDSFGDFGHDF 03:15:45 HJIK ASDSdSADSADSSADSA 03:30:34 DFDF KWQA I have 2 binaries. One is on the local ...
4
votes
3answers
279 views

Check for, and add, missing timestamps to individual lines in a file

I have a file in the following format: ABCD 01206001022T01YA022T01YA022T07SO 09:20:38 ABCD 01206001022ACION 09:24:40 ABCD 04006001021S01UK 09:24:42 ABCD 7878696621321312 23213213213213 ABCD ...
-1
votes
2answers
2k views

How can I print unique line base on particular column value using awk? [closed]

I have a comma separated file with multiple fields. Example: candidate1,12,56,ATTG candidate2,45,90,ATTG candidate3,90,140,ATTG candidate4,2,36,AGCT So, does anyone know the answer?
1
vote
2answers
1k views

Use FOR Loop inside END block of awk command

I'm trying to find the average number of files per day. The following script works: #!/bin/sh ls -l | grep "^-" | awk '{ key=$6$7 freq[key]++ } END { for (date in freq) printf ...
1
vote
1answer
220 views

Regex to Cisco Interface

I would like to know if have some sed, grep or awk regex to parse Cisco interface section and exclude if interface is down (shutdown), like example below. Example: ! interface FastEthernet0/1 ip ...
4
votes
3answers
2k views

Change the text on a single line number in multiple text files

Suppose that I have ten bash shell scripts: script1.sh, script2.sh, ..., script10.sh. Initially, all ten files are identical. Now, I would like to make two changes in each script: In each file, ...
1
vote
1answer
556 views

awk can't print array element

Lets sat my file1 is: A S B Q file2 is: B 2 I want to match $1 from file2 with $1 file1 and print file1 $2 && file2 $2, for the output like this: Q 2 My awk solution ...
0
votes
1answer
362 views

Regex awk to Cisco Interface

I would like to know if have some Sed/Grep or Awk regex to parse Cisco interface section, with specific attribute, like bellow. Content of file.txt ! interface FastEthernet0/1 no ip unreachables ! ...
6
votes
3answers
33k views

How to use regex with AWK for string replacement?

Suppose there is some text from a file: (bookmarks ("Chapter 1 Introduction 1" "#1" ("1.1 Problem Statement and Basic Definitions 23" "#2") ("Exercises 31" "#30") ("Notes and References 42" "#34")) ) ...
2
votes
2answers
102 views

Print line n-1 (and n) if line n fulfills condition

I have a fasta file looking something like this: >19 dna:chromosome chromosome:NCBIM37:19:1:61342430:1 37330562 ID13406 U - TACATGGGTGACAAGGAGGTCAGGTGTTA >19 dna:chromosome ...
2
votes
3answers
149 views

Delete lines containing repeated text

I have a file containing two paths on each line. I want to remove the lines containing the same path twice. I work on Linux and Solaris. I would like a one-liner in sed or awk or perl. Example input ...
6
votes
2answers
1k views

Inotifywait for large number of files in a directory

What I want to do is, to monitor a directory (not recursive, just one) for new files created and append those files to one single big file as they are being written. The number of files that are ...
7
votes
7answers
820 views

Slick one-liner to convert a list like “1: 2, 3, 4, 5” to “1.2, 1.3, 1.4, 1.5”

Let's say I have a file that looks something like this: 23: a, b, c, d 24: b, d, f 25: c, g and I want to get output like this: 23.a 23.b 23.c 23.d 24.b 24.d 24.f 25.c 25.g Of course it's not ...
4
votes
2answers
199 views

AWK - a question about columns

I have a question. I was trying to deal with it by myself, but it seems like I am too new in awk to make it work. Let's assume that we have a file (eg. database.txt) (values are tab-separated): ...
2
votes
3answers
2k views

I want to find duplicates in a file and add a character to the end of the line on the 1st match

I am trying to find duplicates in a file and once a match is found mark the 1st match with a character or word on the end of the line. eg my file (test.html) contains the following entries host= ...
2
votes
2answers
3k views

Replacing text between two HTML Comments

I am new to the world of sed/awk and regex in general and have been studying their usage, but have been floundering trying to meet my need: I have an htm page that has a single line notice that needs ...
2
votes
3answers
337 views

Text to yaml conversion

I have a tab separated text file in this format: your-email your-order-id PayPal-transaction-id your-first-name your-second-name s@ess.com 12345 54321 sooky spooky t@tee.com 23456 ...
5
votes
4answers
4k views

AWK - how to reduce the greed of a regular expression?

I have an example: echo "@article{gjn2010jucs, Author = {Grzegorz J. Nalepa}, " | awk '{ sub(/@.*,/,""); print }' Is it possible to write a regular expression that selects the shorter pattern? ...
2
votes
3answers
3k views

Easy way to parse syslog date format

I'd like to write a script that analyses the last week's syslog. But my syslog logs in a stupid date format, e.g. "May 22". If it logged as 2013-05-22, this task would be trivial. Is there a way ...
5
votes
2answers
252 views

awk + paste for cleaning up PATH?

I have seen this code in .cshrc init files on a few machines. I went through a few awk tutorials in trying to understand how it works, but I am still unable to decrypt it. setenv PATH `echo $PATH | ...
1
vote
1answer
42 views

Matching hexadecimals with gawk

I have a binary file with the following content (printed in hexadecimal): $ xxd -p virus.com 5669727573b440bb0100b90500ba0000cd21 I can match the first 5 letters (which are the alphanumeric string ...
2
votes
3answers
64 views

Replace certain strings with value from former line

I have the following text: Source IP,URL 1.1.1.1,example1.com Blank,example2.com 2.2.2.2,example3.com Blank,example4.com Blank,example5.com Blank,example6.com Blank,example7.com 7.7.7.7,example8.com ...
2
votes
2answers
648 views

xargs grep suggestion

grep -v "\<Swap" instruments.log | awk '{ idx=index($0, "MasterId="); masterId=substr($0, idx+length("MasterId=")+1); masterId=substr(masterId,1,index(masterId,"L")-3); print masterId; }' | xargs ...
3
votes
3answers
198 views

sed + how to remove character/s that start or ended on each number

How do I remove the . character(s) that start in the beginning of each number or end on each number? Remark – perl one liner also good alternative for sed. Example input: .23.12.44.5. .233.3.3.3 ...
3
votes
3answers
220 views

checking data in columns when a data or some may be missing or present?

I am not sure if this possible. say i have columns like : Team Colour Game Rainfall PlayerName XYZ Blue Cricket Yes Kapil suppose i need to search ...
1
vote
2answers
61 views

Trimming one part if equal to another part?

I use an e-mail quote attribution string like this: On dd mmm yyyy hh:mm +hhmm, from info@example.org (Example Dot Org): where info@example.org is the sender's e-mail address and Example Dot Org is ...
2
votes
2answers
120 views

confuse with awk user define function

I have solved Projecteuler Problem 3, but the thing is that I am not able to create the function in awk. I have tried this working code (without function): #!/usr/bin/awk -f BEGIN{ ...
1
vote
2answers
137 views

Problem using awk

I have a problem using awk. Print, from every file given as a parameter, the number of line that has the length at least 10. Also, print the content of that line(s), except the fist 10 characters. At ...
1
vote
1answer
778 views

Using awk to parse AIX's lsuser output in stanza format

I'm creating a simple awk script to parse AIX's lsuser -f ALL output and return information in one line. A sample of the output is something like this. root: id=0 pgrp=something ...
2
votes
6answers
199 views

How can I separate these two outputs using awk command

Here's my output: MessageID=3990592283244651750-30192b51.13df831d93a.7eb4;EsbTiming=2013-04-12T01:07:46.099 I want only the time. How can I separate them?