awk - pattern-directed scanning and processing language
1
vote
2answers
42 views
Grep and sort several unsorted things in several lines (in one document)
I have a file with several unsorted things in one line which I want to put into a new file
This is an example of part of the file I have:
X1314448: SaMi|SM_g2554.t1 SaMi|SM_g5072.t1 Des|Des_g3808.t1 ...
2
votes
2answers
55 views
Print line n-1 (and n) if line n fulfills condition
I have a fasta file looking something like this:
>19 dna:chromosome chromosome:NCBIM37:19:1:61342430:1 37330562 ID13406 U -
TACATGGGTGACAAGGAGGTCAGGTGTTA
>19 dna:chromosome ...
4
votes
1answer
41 views
Removing leading zeros from DATE
I made an alias of the date command to display date in the following format:
2013.06.14.12.10.02
using this command:
alias date = date +"%Y.%m.%d.%H.%M.%S"
Everything works great, except i ...
0
votes
0answers
26 views
Grep word in one file, and use that word to match in FASTA file, adding the FASTA sequence to the first file [closed]
I want to grep several words in file1, and use each word to grep what follows after its match in file2.fasta. And then I want to add the thing that followed the match to the word I used into file03, ...
4
votes
2answers
61 views
Use sed or awk to join lines in a certain pattern?
Not sure on how to turn this input:
A
B
C
apartner
bpartner
cpartner
into this output:
A apartner
B bpartner
c cpartner
I have this pattern in a 80 line file that needs to be 40 line. Also, how ...
0
votes
1answer
91 views
Search low values in mainframe files
I am working with mainframe files and it seems that the low values (x'00') exists in the records in the file which throws off my import for fixed block.
Does anyone know how to eliminate those ...
2
votes
3answers
78 views
Delete lines containing repeated text
I have a file containing two paths on each line. I want to remove the lines containing the same path twice.
I work on Linux and Solaris. I would like a one-liner in sed or awk or perl.
Example input ...
3
votes
2answers
80 views
Inotifywait for large number of files in a directory
What I want to do is, to monitor a directory (not recursive, just one) for new files created and append those files to one single big file as they are being written.
The number of files that are ...
6
votes
7answers
633 views
Slick one-liner to convert a list like “1: 2, 3, 4, 5” to “1.2, 1.3, 1.4, 1.5”
Let's say I have a file that looks something like this:
23: a, b, c, d
24: b, d, f
25: c, g
and I want to get output like this:
23.a
23.b
23.c
23.d
24.b
24.d
24.f
25.c
25.g
Of course it's not ...
0
votes
2answers
201 views
Select/extract columns based on the values on a certain row
Here is an example text file:
A B C D E F G
1 2 3 4 5 6 7
2 3 4 5 6 7 8
3 4 5 6 7 8 9
I would like to extract specific columns based on the values on the 3rd row, that is, 2 3 4 5 6 7 8.
Let's say, ...
4
votes
2answers
124 views
AWK - a question about columns
I have a question. I was trying to deal with it by myself, but it seems like I am too new in awk to make it work.
Let's assume that we have a file (eg. database.txt) (values are tab-separated):
...
2
votes
3answers
64 views
I want to find duplicates in a file and add a character to the end of the line on the 1st match
I am trying to find duplicates in a file and once a match is found mark the 1st match with a character or word on the end of the line.
eg my file (test.html) contains the following entries
host= ...
2
votes
2answers
62 views
Replacing text between two HTML Comments
I am new to the world of sed/awk and regex in general and have been studying their usage, but have been floundering trying to meet my need:
I have an htm page that has a single line notice that needs ...
2
votes
3answers
37 views
Text to yaml conversion
I have a tab separated text file in this format:
your-email your-order-id PayPal-transaction-id your-first-name your-second-name
s@ess.com 12345 54321 sooky spooky
t@tee.com 23456 ...
3
votes
4answers
1k views
AWK - how to reduce the greed of a regular expression?
I have an example:
echo "@article{gjn2010jucs, Author = {Grzegorz J. Nalepa}, " | awk '{ sub(/@.*,/,""); print }'
Is it possible to write a regular expression that selects the shorter pattern?
...
1
vote
1answer
67 views
Regex to Cisco Interface
I would like to know if have some sed, grep or awk regex to parse Cisco interface section and exclude if interface is down (shutdown), like example below.
Example:
!
interface FastEthernet0/1
ip ...
2
votes
3answers
148 views
Easy way to parse syslog date format
I'd like to write a script that analyses the last week's syslog.
But my syslog logs in a stupid date format, e.g. "May 22". If it logged as 2013-05-22, this task would be trivial.
Is there a way ...
5
votes
2answers
89 views
awk + paste for cleaning up PATH?
I have seen this code in .cshrc init files on a few machines. I went through a few awk tutorials in trying to understand how it works, but I am still unable to decrypt it.
setenv PATH `echo $PATH | ...
1
vote
1answer
36 views
Matching hexadecimals with gawk
I have a binary file with the following content (printed in hexadecimal):
$ xxd -p virus.com
5669727573b440bb0100b90500ba0000cd21
I can match the first 5 letters (which are the alphanumeric string ...
1
vote
2answers
136 views
Unix command for pattern matching
I want to count the number of words and print the matched pattern lines which matches exactly with following pattern:
abc-ERROR:
The input File contains:
# abc-ERROR: xyxxkkfgfr
# def-Error: ...
1
vote
2answers
64 views
awk a specific column with unexpected line break
I can't seem to find a solution for this and it's driving me crazy. I know I can use awk to print a column(s). I'm having trouble printing a specific column though because of the way my file system is ...
1
vote
1answer
232 views
Split log file by time range
I have a log file and would like to cut it into copies of 15min. In the log file the lines start with:
dd mmm yyyy hh:mm:ss,xxx
e.g.
12 Feb 2013 16:05:02,xxx log text...
and the file rotates ...
2
votes
3answers
39 views
Replace certain strings with value from former line
I have the following text:
Source IP,URL
1.1.1.1,example1.com
Blank,example2.com
2.2.2.2,example3.com
Blank,example4.com
Blank,example5.com
Blank,example6.com
Blank,example7.com
7.7.7.7,example8.com
...
1
vote
1answer
116 views
reading from a file and changing its pattern into an array?
I have a CSV file containging entries like this :
ipaddress,VLAN,VLANid
10.192.168.1,vlan-xyz,3
10.192.168.1,vlan-abc,8
10.192.168.1,vlan-mnp,11
10.192.163.24,vlan-llz,3
10.192.163.24,vlan-bbz,5
...
2
votes
2answers
245 views
xargs grep suggestion
grep -v "\<Swap" instruments.log | awk '{ idx=index($0, "MasterId="); masterId=substr($0, idx+length("MasterId=")+1); masterId=substr(masterId,1,index(masterId,"L")-3); print masterId; }' | xargs ...
3
votes
3answers
82 views
sed + how to remove character/s that start or ended on each number
How do I remove the . character(s) that start in the beginning of each number or end on each number?
Remark – perl one liner also good alternative for sed.
Example input:
.23.12.44.5.
.233.3.3.3
...
3
votes
3answers
77 views
checking data in columns when a data or some may be missing or present?
I am not sure if this possible.
say i have columns like :
Team Colour Game Rainfall PlayerName
XYZ Blue Cricket Yes Kapil
suppose i need to search ...
1
vote
2answers
39 views
Trimming one part if equal to another part?
I use an e-mail quote attribution string like this:
On dd mmm yyyy hh:mm +hhmm, from info@example.org (Example Dot Org):
where info@example.org is the sender's e-mail address and Example Dot Org is ...
5
votes
5answers
239 views
How to remove multiple blank lines from a file?
I have some text-files I use to take notes in - just plain text, usually just using cat >> file. Occasionally I use a blank line or two (just return - the new-line character) to specify a new ...
2
votes
2answers
36 views
confuse with awk user define function
I have solved Projecteuler Problem 3, but the thing is that I am not able to create the function in awk.
I have tried this working code (without function):
#!/usr/bin/awk -f
BEGIN{
...
1
vote
2answers
90 views
Problem using awk
I have a problem using awk. Print, from every file given as a parameter, the number of line that has the length at least 10. Also, print the content of that line(s), except the fist 10 characters. At ...
1
vote
1answer
68 views
Using awk to parse AIX's lsuser output in stanza format
I'm creating a simple awk script to parse AIX's lsuser -f ALL output and return information in one line. A sample of the output is something like this.
root:
id=0
pgrp=something
...
2
votes
6answers
101 views
How can I separate these two outputs using awk command
Here's my output:
MessageID=3990592283244651750-30192b51.13df831d93a.7eb4;EsbTiming=2013-04-12T01:07:46.099
I want only the time.
How can I separate them?
0
votes
1answer
30 views
moving columns across rows
I am dealing with a txt file and trying to keep the first column/field in place, but shift the 2nd column down by 1 row. I tried this, but it gave the error unmatched .'s.
awk '{for(i=1; i<NR; ...
10
votes
6answers
2k views
How to print the longest line in a file?
I'm looking for the simplest method to print the longest line in a file. I did some googling and surprisingly couldn't seem to find an answer. I frequently print the length of the longest line in a ...
2
votes
3answers
178 views
How can I use SED or AWK to replace placeholders in a template file with variable content that contains special characters?
CentOS 6.3
I'm trying to get a small script to send an email containing a copy of email headers in the body (for the purpose of internal reporting).
The template file contains the following:
...
5
votes
3answers
3k views
How to sed only that lines that contains given string?
INPUT:
Select ASDF 325 sdfg sdflk lk
Select TRG 46sdg rasdftz fsgs 45
Select ASDF 6ffg sdfg 4456 sdrg
OUTPUT:
Select ASDF 325 XXXX sdflk lk
Select TRG 46sdg rasdftz fsgs 45
Select ASDF 6ffg XXXX ...
3
votes
2answers
55 views
Automated partition selection in bash script
I'm trying to write a bash script to run from Linux on a CD or flashdrive that can mount Windows and change stuff, but I don't know how to go about detecting which partition/drive is Windows.
My best ...
4
votes
3answers
131 views
Remove string from a particular field using awk/sed
I have a file (>80,000 lines) that looks likes this:
chr1 GTF2GFF chromosome 1 249213345 . . . ID=chr1;Name=chr1
chr1 GTF2GFF gene 11874 14408 . + . ...
4
votes
4answers
106 views
How to remove character and space from a string
Hello I have one file with output
Name : ABC
Name : CDE
Name : ZYS
I want to get the result
ABC
CDE
ZYS
How can i get it through SED command.
I have tried. sed s/'Name' but ...
16
votes
9answers
1k views
Remove duplicate $PATH entries with awk command
I am trying to write a bash shell function that will allow me to remove duplicate copies of directories from my path environment.
I was told that it is possible to achieve this with a one line ...
0
votes
1answer
149 views
comparing lines with awk vs while read line
I have two files one with 17k lines and another one with 4k lines. I wanted to compare position 115 to position 125 with each line in the second file and if there is a match, write the entire line ...
4
votes
3answers
3k views
Delete range of lines above pattern with sed (or awk)
I have the following code that will remove lines with the pattern banana and 2 lines after it:
sed '/banana/I,+2 d' file
So far, so good! But I need it to remove 2 lines before banana, but I can't ...
7
votes
6answers
1k views
Is there any alternative to grep's -A -B -C switches (to print few lines before and after )?
grep -A 2 -B 3
prints 2 lines after the grep string and prints 3 lines before.
grep -C 3
prints 3 Lines before and 3 lines after
Unfortunately, the grep I'm using does not support these ...
5
votes
4answers
187 views
Show sum of file sizes in directory listing
The Windows dir directory listing command has a line at the end showing the total amount of space taken up by the files listed. For example, dir *.exe shows all the .exe files in the current ...
2
votes
2answers
90 views
Using bash shell function inside AWK
Is it possible to use bash function inside AWK somehow?
Example file (string, int, int, int)
Mike 247808 247809 247810
Trying to convert values from decimal to hexadecimal.
Function defined ...
3
votes
1answer
157 views
Pass colors from ls through pipe to awk 'print' statement
This is a follow-up to my question from yesterday, Show sum of file sizes in directory listing.
Thanks to Zero Piraeus and a point in the right direction by Mauritz Hansen, I now have
function ...
1
vote
3answers
82 views
awk + print line only if the first field start with string as Linux1
how to print the line in case the first field start with Linux1
for example:
echo Linux1_ver2 12542 kernel-update | awk '{if ($1 ~ Linux1 ) print $0;}'
the target is to print the line , ...
3
votes
1answer
105 views
Output of `java -version` not matched by grep or awk
On my Linux machine, it isn't clear to me why if I do the following then I don't get only the version string ("1.5.0_32").
# java -version | grep version | awk '{print $NF}'
java version "1.5.0_32"
...
2
votes
2answers
55 views
cumulative totals with shifted column
Apologies in advance since I am sure this is an easy problem. Nevertheless, here goes.
cat blah.txt
aa+2
bb+4
cc+10
dd+31
I would like to process blah.txt to produce the folowing:
aa+0
bb+2
cc+6
...
