awk - pattern-directed scanning and processing language
4
votes
1answer
759 views
awk: forcing a return status?
This is a followup to my earlier question.
I am validating the number of fields in /etc/passwd using this handy snippit. In the following example, the users 'fieldcount1' and 'fieldcount2' have the ...
-1
votes
1answer
1k views
Sum of grep results
I need to output the sum of a grep results for a certain time, for now i hard coded the time grep but any help will be very much appreciated, my problem is i cant output the results on the script.
If ...
2
votes
1answer
1k views
Time/Date grep for 15 mins
I need to grep a specific type of line in a specific time of the log, do you have any ideas?
Here's my working script:
cat *.log |grep -E '2011-06-30 (1[0-1]:[0-1][0-5]|10:16)'| grep -ach '0110 ...
1
vote
1answer
293 views
Inconsistent field separator behaviour of space in awk
The following awk script shows how a leading space ' ' is ignored as a field separator, but other characters are treated as a separator with a preceding null $1 field.
Is there some way to make ...
1
vote
1answer
189 views
Need help processing a text file with awk to conform to CSV flat file format
I have the following problem. I collected data on reaction time from over 100 participants for an experiment I am running. Unfortunately, the separators between fields were not consistent, but after a ...
3
votes
2answers
523 views
only display df lines that has more fs usage then 80%
root@SERVER ~$ df
Filesystem 512-blocks Free %Used Iused %Iused Mounted on
/dev/YXCV 655360 365632 45% 6322 13% /
/dev/ASDF 3801088 670648 83% 41759 32% /usr
/dev/ASR ...
2
votes
3answers
330 views
pad a number with zeroes
I need have put 0 on the results of my grep so my script format will be fine and i don't have any idea how to do it. here's my grep result :
261 : 261 = 0 | 1192 : 1184 = 8 |
283 : 283 = 0 | 666 : ...
8
votes
3answers
2k views
Delete the first n bytes of files
I've got an extreme problem, and all of the solutions I can imagine are complicated. According to my UNIX/Linux experience there must be an easy way.
I want to delete the first 31 bytes of each file ...
11
votes
6answers
14k views
Is there a way to get the min, max, median, and average of a list of numbers in a single command?
I have a list of numbers in a file, one per line. How can I get the minimum, maximum, median and average values? I want to use the results in a bash script.
Although my immediate situation is for ...
6
votes
6answers
2k views
How can you combine all lines that end with a backslash character?
Using a common command line tool like sed or awk, is it possible to join all lines that end with a given character, like a backslash?
For example, given the file:
foo bar \
bash \
baz
dude \
happy
...
5
votes
4answers
459 views
How to append end of every line with the line itself
This is in file.txt:
redcar
bluecar
greencar
Im looking for ways to make it become:
redcar redcar
bluecar bluecar
greencar greencar
I've tried many ways using sed with no luck
3
votes
1answer
547 views
using system command in awk script
In an AWK script I am using a command
system(date)
to print the current date in a file but after this command is executed next line is also added implicitly. Is there any way in AWK to print ...
-4
votes
2answers
252 views
Manipulating a file with awk
I have a text file in the following format:
Surname, Forename: Day.Month.Year: Degree
Sellen, Jo: 03.07.1986: MSc CSE
Parfitt, Harry: 20.03.1984: MSc IT
How can I write a shell script that uses ...
6
votes
2answers
864 views
sed one-liner to delete any line that does not contain lowercase letters
So, basically
THIS LINE WOULD BE DELETED
and
(THIS LINE WOULD ALSO BE DELETED)
but
Indeed, THIS LINE WOULD NOT
7
votes
6answers
1k views
Is there any alternative to grep's -A -B -C switches (to print few lines before and after )?
grep -A 2 -B 3
prints 2 lines after the grep string and prints 3 lines before.
grep -C 3
prints 3 Lines before and 3 lines after
Unfortunately, the grep I'm using does not support these ...
3
votes
2answers
994 views
My awk program to change spaces into tabs doesn't work
How to write a shell script which uses awk to read in the data file students.txt and output the data in the tabbed format as shown:
Surname Forename MSc Stream Date of Birth
Smith John IT 15.01.1986
...
4
votes
5answers
9k views
Multiline pattern match using sed, awk or grep
Is it possible to do a multiline pattern match using sed, awk or grep? Take for example, I would like to get all the lines between { and }
So it should be able to match
1. {}
2. {.....}
3. ...
3
votes
4answers
237 views
Remove empty configuration section
Files like ~/.config/vlc/vlcrc are 99% junk if you want to version control only the configuration options. I've got a script to remove the comments, but there's a ton of empty configuration sections ...
2
votes
3answers
2k views
Loop a list through awk
I'm sorry if this is extremely elementary, but I just can't figure out how to do this, and my research has failed me as well.
I have two files: data.csv and list.txt. Here's an example of what they ...
3
votes
2answers
279 views
Truncate a file on a certain pattern
How would I go about truncating a binary file when a certain pattern is found?
For instance, I want to truncate the file at the first occurrence of the pattern 0xFFFFFFFF.
I think something like awk ...
2
votes
3answers
2k views
Extracting tokens from a line of text
Using bash scripting and grep/awk/sed, how can I split a line matching a known pattern with a single character delimiter into an array, e.g. convert token1;token2;token3;token4 into a[0] = token1 … ...
2
votes
1answer
447 views
Using grep/sed/awk to classify log file entries
I need to process a very large log file with many lines in different formats.
My goal is to extract unique line entries who have the same starting pattern, e.g. '^2011-02-21.*MyKeyword.*Error', ...
2
votes
2answers
814 views
How to build a long command string?
I've a sequence of commands to be used along with lot of pipings, something like this:
awk '{ if ($8 ~ "isad") {print $2, $5, "SE"} else {print $2, $5, "ANT"} }' ...
0
votes
2answers
649 views
How to run a shell script containing an awk command
How to run this script (called count.sh)?
awk 'BEGIN{ x=0 ; while (x < 5) { x+=1 ; print x ; } }'
I'm trying to run it with sh count.sh but it's giving me an error.
12
votes
9answers
1k views
There must be a better way to replace single newlines only?
I am in the habit of writing one line per sentence because I typically compile things to LaTex, or am writing in some other format where line breaks get ignored. I use a blank line to indicate the ...
2
votes
1answer
4k views
Filtering on dates with grep and awk
I have created the alias below in my .bash_aliases file
alias auth="grep \"$(date|awk '{print $2,$3}')\" /var/log/auth.log |
grep -E '(BREAK-IN|Invalid user|Failed|refused|su|Illegal)'"
...
5
votes
3answers
510 views
extracting “tag” information from git with a shell script
The vc bundle is a neat little package that extracts information about a git repo for easy insertion into a LaTeX document. It doesn't currently extract information about whether the current commit is ...
6
votes
3answers
1k views
AWK: wrap lines to 72 characters
$ awk 'length > 72' {HOW TO PRINT THE LINEs IN PCS?} msg
ie I want it to add \n after 72 chars and continue, so initially you may need to remove all single \ns and the add them. It may be easier ...
5
votes
4answers
209 views
How could I simplify this command to only use awk?
awk '/user/ {print $1 }' /etc/userdomains | sed 's/://'
the format of /etc/userdomains is
domain.tld: user
otherdomain.tld: otheruser
45
votes
6answers
7k views
Is there a basic tutorial for grep, awk and sed?
I've been a Linux user for a while, and I've a pretty decent understanding of most the common command line utilities. However, ones that come up and up again in relation to programming are grep, awk, ...
13
votes
9answers
6k views
Tool in UNIX to subtract dates
Is there any tool in Solaris UNIX (so no GNU tool available) to subtract dates? I know that in Linux we have gawk that can subtract one date from another. But in Solaris the maximum we have is nawk ...
8
votes
6answers
220 views
Grab certain contents of a file
So I know tools exist for this problem because I've heard about them, but I don't know what they are.
I want to do something like filter out all data but the usernames in /etc/passwd.
For example, I ...
