awk - pattern-directed scanning and processing language
3
votes
3answers
261 views
swapping fields in columns by fixed pattern (yes, no, no, yes)
I want to produce a following list of pages (for imposition work); (number of pages is variable, but I make a sample of my needs using a list of 16 pages, logic is the same for longer lists)
1,16
8,9
...
3
votes
6answers
740 views
Grep a directory and return list with line numbers
I'm currently trying to learn more about bash scripting and all of that fun stuff, and I pieced together this little command:
find $path | xargs grep -n $pattern | awk '{print $1}'
While this DOES ...
3
votes
1answer
206 views
awk high precision arithmetic
I am looking for a way to tell awk to do high-precision arithmetic in a substitution operation. This involves, reading a field from a file and substituting it with a 1% increment on that value. ...
3
votes
5answers
483 views
extract fields from “tail -f” of a syslog stream
Example line from syslog file:
Aug 1 10:25:50 10.10.10.1 id=firewall sn=XXXX time="2012-08-01 14:35:18 UTC" fw=x.x.x.x pri=6 c=1024 m=537 msg="Connection Closed" f=11 n=195273698 ...
3
votes
3answers
896 views
Piping in awk scripts
I'm trying to write a ls wrapper that uses awk to parse the output of ls -lhF. Right now I've split the program into two files - my_ls.sh and my_ls.awk. my_ls.sh's only purpose is to pipe the output ...
3
votes
6answers
835 views
Text file look-up by column
I have a file in this format:
[#] OWNER_NAME NAME SIZE
[6] Robottinosino Software 200
[42] Robottinosino Ideas worth zero 188
[12] ...
3
votes
2answers
161 views
Subtracting one text string from another leaving unique characters, perhaps using awk
Is there any way to subtract one column of data containing text from another column containing text and get third column containing unique charcters, for example using awk eg.
Input
ab a
cd d ...
3
votes
1answer
629 views
escaping a single dot with double backslash - awk
"effective awk programming" book has an example on Field-Splitting. here is the example:
If you want fields to be separated by a literal period
followed by any single character, use ‘FS = ...
3
votes
5answers
449 views
awk separate columns of 3
I have a file that looks like the following (the columns are separated by tabs and might contain spaces):
0637 apple 8528 1392 orange 1390 8528 unknown fruit 7537 1039 ...
3
votes
2answers
531 views
only display df lines that has more fs usage then 80%
root@SERVER ~$ df
Filesystem 512-blocks Free %Used Iused %Iused Mounted on
/dev/YXCV 655360 365632 45% 6322 13% /
/dev/ASDF 3801088 670648 83% 41759 32% /usr
/dev/ASR ...
3
votes
2answers
1k views
My awk program to change spaces into tabs doesn't work
How to write a shell script which uses awk to read in the data file students.txt and output the data in the tabbed format as shown:
Surname Forename MSc Stream Date of Birth
Smith John IT 15.01.1986
...
3
votes
2answers
279 views
Truncate a file on a certain pattern
How would I go about truncating a binary file when a certain pattern is found?
For instance, I want to truncate the file at the first occurrence of the pattern 0xFFFFFFFF.
I think something like awk ...
3
votes
3answers
75 views
sed + how to remove character/s that start or ended on each number
How do I remove the . character(s) that start in the beginning of each number or end on each number?
Remark – perl one liner also good alternative for sed.
Example input:
.23.12.44.5.
.233.3.3.3
...
3
votes
1answer
118 views
Pass colors from ls through pipe to awk 'print' statement
This is a follow-up to my question from yesterday, Show sum of file sizes in directory listing.
Thanks to Zero Piraeus and a point in the right direction by Mauritz Hansen, I now have
function ...
3
votes
1answer
42 views
How to edit files in-place while performing operation on the values matching the searched pattern?
Given a list files containing random text and many datetimes in ISO format in it (e.g "2012-07-02T10:47:24+02:00"), how can I find all these datetimes and add 2 hours to each one ?
Ideally a solution ...
3
votes
1answer
159 views
Split file based on a pattern with leading zeros
I have a book in text format. I would like to split the book into several files where each file contains a single chapter. Therefore I'm using the following command:
awk '/Chapter/{i++}{print > ...
3
votes
1answer
445 views
Make awk use bash with the system() command
Is there a way to make awk use bash instead of sh when running system commands using the system() call?
I want to use some bash-specific features such as [[ ]] and < > string comparison operators:
...
3
votes
3answers
2k views
How to list all unique ip address currently connected to a specific port?
Say I want to know how many unique clients are connected to port 5222 on a server.
Can you find a way better/faster/stronger than this?
netstat -nt | grep ':5222.*ESTABLISHED' | awk '{ print $5 }' \
...
3
votes
2answers
858 views
use awk to replace parentheses?
How can I use awk to replace the parentheses in these 2 scenarios?
Change empty parens to the word "None"
192.168.0.24,Up,()
Remove parens to only host.domain.com is left
...
3
votes
3answers
166 views
What does . match?
In working with regular expressions, I have been told that a dot character . will match everything.
Except for newlines \n.
Are there any other exceptions? What about the NUL character \0, or the ...
3
votes
1answer
141 views
Help improving my AWK skills
This program works as I intended but I feel like I used some clumsy methods to get the out put I desired(especially with my use of print commands and variable declarations)
Could some one improve ...
3
votes
3answers
431 views
Using sed to edit lines in a file with a variable
I need to edit lines in a file using sed. Now the problem is I am replacing a particular pattern with a combination of text and number. This number is a variable which keeps on incrementing for every ...
3
votes
3answers
961 views
In awk, how to print a range of fields with the original $0 delimiters intact?
How to extract a substring of $0, in awk, when that substring's position and length is based on the positions of fields in the original $0?
It effectively equates to removing the first n.lead fields ...
3
votes
2answers
47 views
Automated partition selection in bash script
I'm trying to write a bash script to run from Linux on a CD or flashdrive that can mount Windows and change stuff, but I don't know how to go about detecting which partition/drive is Windows.
My best ...
3
votes
2answers
152 views
AWK for data smoothing
I need some help to smooth some data with awk olny of column 4 and next.
This is a data example:
Date;time;Time_ms;A;B;C;D
23.11.2012;15:03:00;41236627696;1;2;2;3
...
3
votes
3answers
157 views
How can awk cut certain fields and add to the end of each line?
I would like to convert a list containing bunch of svn URLs of issues:
cat list.txt
//svn.server.address/repos/project/module1/branches/issue-001-name1
...
3
votes
1answer
81 views
Are there awk versions that provide syntax for computing aggregations?
From time to time I find myself writing awk scripts that compute some simple statistics. For example computing a histogram, the average of a value, the standard deviation or even the variance ...
...
3
votes
1answer
186 views
grep list of names and information from bigger file
I have two files: one with list of names (500 entries) and other having some more information for each entry in A.txt and extra entries too.
File A.txt (each line is starting with > (fasta format) ...
3
votes
2answers
284 views
merging two column
input_file (columns are tab delimited, only column 4 contains spaces)
input
A B C D;<>X;<>Y 0
C D E F;<>Z;<>X 1
output
A B C D;<>X;<>Y;<>apple=0
C D ...
3
votes
1answer
324 views
merging files and getting column values based on id field
bash-3.2$ cat sample.log sample.log.1 sample.log.2
ID COL1 COL2 COL4
1 col1 col2 col4
2 c1 c2 c4
3 co1 co2 co4
ID COL3 COL1
1 col3 col1
2 c3 c1
3 co3 co1
ID COL1 COL2 COL3
1 col1 ...
3
votes
1answer
553 views
using system command in awk script
In an AWK script I am using a command
system(date)
to print the current date in a file but after this command is executed next line is also added implicitly. Is there any way in AWK to print ...
3
votes
4answers
237 views
Remove empty configuration section
Files like ~/.config/vlc/vlcrc are 99% junk if you want to version control only the configuration options. I've got a script to remove the comments, but there's a ton of empty configuration sections ...
3
votes
1answer
498 views
File size limit exceeded in bash [closed]
I have tried this shell script on a SUSE 10 server, kernel 2.6.16.60, ext3 filesystem.
The script has a line like this:
cat file | awk '{print $1" "$2" "$3}' | sort -n > result
The file's size ...
3
votes
2answers
473 views
simple script rotating table
I'm trying to write a simple script, that just rotates csv table.
I mean I have some file:
head1;head2;head3
field11;field12;field13
and so on.
All I want, just to make my file
...
3
votes
1answer
205 views
Relative line offsets in awk (/pattern/+1 or /pattern/-1)
When I want to print lines between two patterns, excluding the lines with those patterns, I can do it in ex using +1 and -1 after the pattern. That is awesome. Is this possible in awk? Right now, I ...
3
votes
1answer
84 views
No output from inotifywait | awk
I'm attempting to use part of a one-liner found here: Script to monitor folder for new files?
When I try the following procedure I get no output whatsoever and I cannot figure out why.
In terminal ...
3
votes
0answers
130 views
How to use regrex with AWK for string replacement in this example? [duplicate]
Possible Duplicate:
How to use regrex with AWK for string replacement in this example?
Suppose there is some text from a file:
(bookmarks
("Chapter 1 Introduction 1" "#1"
("1.1 Problem ...
2
votes
3answers
168 views
reading entries from multiple files
I have few files like A.txt, B.txt and C.txt, each having a only single number in it i.e A.txt has 45 B.txt has 35 and C.txt has 100. How could I read these entries from each files and report with its ...
2
votes
4answers
166 views
Output some sections of a text file [closed]
I have input like the following:
>gi|481443199|gb|MH447771.1|
49 0 0 0 1
50 0 0 1 0
>gi|987078004|gb|RTYJ01067061.1|
412 0 0 1 ...
2
votes
3answers
385 views
How to truncate the second column from a dot (.)?
INPUT:
10.175.192.16 vgxyxgf.de.foo.net asdf 34efg
10.175.192.17 sdtds.de.foo.net xyyxv e5432
10.175.243.20 asdfycfb.us.foo.net dfvasf 34525
OUTPUT:
10.175.192.16 vgxyxgf
...
2
votes
1answer
700 views
Diff the output of two `awk` commands
I'm trying to compute the difference between the output of two awk commands but my simple attempts at it seem to be failing. Here is what I'm trying:
diff $(awk '{print $3}' f1.txt | sort -u) $(awk ...
2
votes
3answers
1k views
Extracting column from comma separated text
I have a long comma-separated delimited file with 20K lines. Here's a sample:
"","id","number1","number2","number3","number4","number5","number6","number7"
...
2
votes
4answers
399 views
shorten long lines in a log file
I keep a logfile of a program's output. The trouble is sometimes errors occur dumping very long lines of 7bit ascii data(encoded binary) I don't care to keep. These lines can be 200KB+ long before ...
2
votes
4answers
2k views
awk-Printing column value without new line and adding comma
input.txt
EN1
EN2
EN3
EN4
EN5
output
EN1,EN2,EN3,EN4,EN5
I have tried awk.But it is not printing with comma
awk 'BEGIN { OFS = ","} { printf $1}' input.txt
I have GNU Awk 4.0.0 version
2
votes
4answers
647 views
gawk join two TSV by columns (a'la sql join)
How to join, to tsv files, examples:
a.tsv
c 7 r z
d 6 s w
f 1 f f
b 8 p y
a 9 q x
b.tsv
a q a
c r ccc
b p bb
0 0 0
d s dddd
Here I'd like to ...
2
votes
3answers
2k views
Extracting tokens from a line of text
Using bash scripting and grep/awk/sed, how can I split a line matching a known pattern with a single character delimiter into an array, e.g. convert token1;token2;token3;token4 into a[0] = token1 … ...
2
votes
3answers
94 views
Easy way to parse syslog date format
I'd like to write a script that analyses the last week's syslog.
But my syslog logs in a stupid date format, e.g. "May 22". If it logged as 2013-05-22, this task would be trivial.
Is there a way ...
2
votes
3answers
411 views
Average rows with same first column
Given a file with two columns:
Id ht
510 69
510 67
510 65
5l0 62
510 59
601 29
601 26
601 21
601 20
I need a way to coalesce all rows with the same ID into one that has an average height. In this ...
2
votes
3answers
336 views
pad a number with zeroes
I need have put 0 on the results of my grep so my script format will be fine and i don't have any idea how to do it. here's my grep result :
261 : 261 = 0 | 1192 : 1184 = 8 |
283 : 283 = 0 | 666 : ...
2
votes
3answers
175 views
Bash commands/script to remove a line from CSV with duplicate in column
I have a lot CSV files that I have combined. However, there are duplicates, but the entire line is not duplicated. I do have a column that I want to use as the criteria to search for a duplicate. ...
