Questions tagged [awk]

A pattern-directed scanning and processing language.

Filter by
Sorted by
Tagged with
34 votes
9 answers
110k views

How to print only last column?

echo -e 'one two three\nfour five six\nseven eight nine' one two three four five six seven eight nine how can I do some "MAGIC" do get this output?: three six nine UPDATE: I don't need it in this ...
user avatar
  • 36.1k
3 votes
3 answers
4k views

In awk, how to print a range of fields with the original $0 delimiters intact?

How to extract a substring of $0, in awk, when that substring's position and length is based on the positions of fields in the original $0? It effectively equates to removing the first n.lead fields ...
user avatar
  • 30.9k
12 votes
2 answers
21k views

How would I extract the user agent strings from a log file?

Currently I am running a command like this, to get the most requested content: grep "17\/Jul\/2011" other_vhosts_access.log | awk '{print $8}' | sort | uniq -c | sort -nr I want to now see the user ...
user avatar
4 votes
2 answers
1k views

maximum of column with matching id

I have a file with the following fields NM001 rp1 210 NM003 rp1 220 NM005 rag 200 NM004 rag 100 I would like to have an output that for every change in column2, find maximum in column4 but also ...
user avatar
  • 41
1 vote
1 answer
128 views

awk expand window to 500% for two numbers from column file

I've got a file with two columns corresponding to start-end ranges like this: awk '{print $2"-"$3}' file.txt 91082802-91082990 94018177-94018321 146945732-146945949 88701397-88701594 80922510-...
user avatar
  • 651
20 votes
3 answers
22k views

awk: forcing a return status?

This is a followup to my earlier question. I am validating the number of fields in /etc/passwd using this handy snippit. In the following example, the users 'fieldcount1' and 'fieldcount2' have the ...
user avatar
-1 votes
1 answer
4k views

Sum of grep results

I need to output the sum of a grep results for a certain time, for now i hard coded the time grep but any help will be very much appreciated, my problem is i cant output the results on the script. If ...
user avatar
  • 3,699
3 votes
1 answer
4k views

Time/Date grep for 15 mins

I need to grep a specific type of line in a specific time of the log, do you have any ideas? Here's my working script: cat *.log |grep -E '2011-06-30 (1[0-1]:[0-1][0-5]|10:16)'| grep -ach '0110 ...
user avatar
  • 3,699
1 vote
1 answer
2k views

Inconsistent field separator behaviour of space in awk

The following awk script shows how a leading space ' ' is ignored as a field separator, but other characters are treated as a separator with a preceding null $1 field. Is there some way to make ...
user avatar
  • 30.9k
1 vote
1 answer
622 views

Processing a text file with awk to conform to CSV flat file format

I collected data on reaction time from over 100 participants for an experiment I am running. Unfortunately, the separators between fields were not consistent, but after a lot of heartache with sed, I ...
user avatar
8 votes
5 answers
33k views

only display df lines that have more fs usage than 80%

root@SERVER ~$ df Filesystem 512-blocks Free %Used Iused %Iused Mounted on /dev/YXCV 655360 365632 45% 6322 13% / /dev/ASDF 3801088 670648 83% 41759 32% /usr /dev/ASR ...
user avatar
  • 36.1k
2 votes
3 answers
2k views

pad a number with zeroes

I need have put 0 on the results of my grep so my script format will be fine and i don't have any idea how to do it. here's my grep result : 261 : 261 = 0 | 1192 : 1184 = 8 | 283 : 283 = 0 | 666 : ...
user avatar
  • 3,699
35 votes
4 answers
44k views

Delete the first n bytes of files

I've got an extreme problem, and all of the solutions I can imagine are complicated. According to my UNIX/Linux experience there must be an easy way. I want to delete the first 31 bytes of each file ...
user avatar
40 votes
9 answers
26k views

How can you combine all lines that end with a backslash character?

Using a common command line tool like sed or awk, is it possible to join all lines that end with a given character, like a backslash? For example, given the file: foo bar \ bash \ baz dude \ happy ...
user avatar
  • 16.8k
6 votes
4 answers
5k views

How to append end of every line with the line itself

This is in file.txt: redcar bluecar greencar Im looking for ways to make it become: redcar redcar bluecar bluecar greencar greencar I've tried many ways using sed with no luck
user avatar
4 votes
2 answers
4k views

Getting current time as output from "date" in awk script

In an awk, script I am using a command system(date) to print the current date in a file, but after this command is executed, the next line is also added implicitly. Is there any way in awk to ...
user avatar
  • 141
-9 votes
2 answers
494 views

Manipulating a file with awk [duplicate]

I have a text file in the following format: Surname, Forename: Day.Month.Year: Degree Sellen, Jo: 03.07.1986: MSc CSE Parfitt, Harry: 20.03.1984: MSc IT How can I write a shell script that uses ...
user avatar
  • 197
13 votes
2 answers
12k views

sed one-liner to delete any line that does not contain lowercase letters

So, basically THIS LINE WOULD BE DELETED and (THIS LINE WOULD ALSO BE DELETED) but Indeed, THIS LINE WOULD NOT
user avatar
  • 12.8k
10 votes
6 answers
4k views

Is there any alternative to grep's -A -B -C switches (to print few lines before and after )?

grep -A 2 -B 3 prints 2 lines after the grep string and prints 3 lines before. grep -C 3 prints 3 Lines before and 3 lines after Unfortunately, the grep I'm using does not support these options. ...
user avatar
5 votes
2 answers
21k views

My awk program to change spaces into tabs doesn't work [closed]

How to write a shell script which uses awk to read in the data file students.txt and output the data in the tabbed format as shown: Surname Forename MSc Stream Date of Birth Smith John IT 15.01.1986 ...
user avatar
  • 197
28 votes
5 answers
71k views

Multiline pattern match using sed, awk or grep [duplicate]

Is it possible to do a multiline pattern match using sed, awk or grep? Take for example, I would like to get all the lines between { and } So it should be able to match 1. {} 2. {.....} 3. {........
user avatar
3 votes
3 answers
425 views

Remove empty configuration section

Files like ~/.config/vlc/vlcrc are 99% junk if you want to version control only the configuration options. I've got a script to remove the comments, but there's a ton of empty configuration sections ...
user avatar
  • 46.8k
3 votes
3 answers
2k views

Get last set of non-empty lines

I'm running a cron job which should get only the last result of iostat -d 1 2. This requires some parsing: What is the simplest way to get the last set of non-empty lines from standard input to ...
user avatar
  • 46.8k
2 votes
3 answers
24k views

Loop a list through awk

I have two files: data.csv and list.txt. Here's an example of what they look like data.csv: "John","red","4" "Basketball","orange","2" "The Mike","blue","94" "Lizard","purple","3" "Johnny","pink","...
user avatar
  • 123
6 votes
2 answers
865 views

Truncate a file on a certain pattern

How would I go about truncating a binary file when a certain pattern is found? For instance, I want to truncate the file at the first occurrence of the pattern 0xFFFFFFFF. I think something like awk ...
user avatar
  • 568
5 votes
3 answers
17k views

Extracting tokens from a line of text [closed]

Using bash scripting and grep/awk/sed, how can I split a line matching a known pattern with a single character delimiter into an array, e.g. convert token1;token2;token3;token4 into a[0] = token1 … ...
user avatar
  • 315
4 votes
2 answers
2k views

Using grep/sed/awk to classify log file entries

I need to process a very large log file with many lines in different formats. My goal is to extract unique line entries who have the same starting pattern, e.g. '^2011-02-21.*MyKeyword.*Error', ...
user avatar
  • 315
3 votes
3 answers
3k views

How to build a long command string?

I've a sequence of commands to be used along with lot of pipings, something like this: awk '{ if ($8 ~ "isad") {print $2, $5, "SE"} else {print $2, $5, "ANT"} }' "/var/log/apache2/other_vhosts_access....
user avatar
  • 2,298
0 votes
2 answers
5k views

How to run a shell script containing an awk command [closed]

How to run this script (called count.sh)? awk 'BEGIN{ x=0 ; while (x < 5) { x+=1 ; print x ; } }' I'm trying to run it with sh count.sh but it's giving me an error.
user avatar
34 votes
13 answers
13k views

There must be a better way to replace single newlines only?

I am in the habit of writing one line per sentence because I typically compile things to LaTeX, or am writing in some other format where line breaks get ignored. I use a blank line to indicate the ...
user avatar
  • 3,273
3 votes
1 answer
15k views

Filtering on dates with grep and awk

I have created the alias below in my .bash_aliases file alias auth="grep \"$(date|awk '{print $2,$3}')\" /var/log/auth.log | grep -E '(BREAK-IN|Invalid user|Failed|refused|su|Illegal)'" ...
user avatar
  • 1,463
6 votes
3 answers
4k views

extracting "tag" information from git with a shell script

The vc bundle is a neat little package that extracts information about a git repo for easy insertion into a LaTeX document. It doesn't currently extract information about whether the current commit is ...
user avatar
  • 3,273
8 votes
7 answers
10k views

AWK: wrap lines to 72 characters

$ awk 'length > 72' {HOW TO PRINT THE LINEs IN PCS?} msg ie I want it to add \n after 72 chars and continue, so initially you may need to remove all single \ns and the add them. It may be easier ...
user avatar
6 votes
4 answers
348 views

How could I simplify this command to only use awk?

awk '/user/ {print $1 }' /etc/userdomains | sed 's/://' the format of /etc/userdomains is domain.tld: user otherdomain.tld: otheruser
user avatar
110 votes
7 answers
69k views

Is there a basic tutorial for grep, awk and sed? [closed]

I've been a Linux user for a while, and I've a pretty decent understanding of most the common command line utilities. However, ones that come up and up again in relation to programming are grep, awk, ...
user avatar
  • 3,640
25 votes
11 answers
23k views

Tool in UNIX to subtract dates

Is there any tool in Solaris UNIX (so no GNU tool available) to subtract dates? I know that in Linux we have gawk that can subtract one date from another. But in Solaris the maximum we have is nawk (...
user avatar
  • 657
9 votes
6 answers
657 views

Grab certain contents of a file

So I know tools exist for this problem because I've heard about them, but I don't know what they are. I want to do something like filter out all data but the usernames in /etc/passwd. For example, I ...
user avatar
  • 1,255
9 votes
1 answer
12k views

Replacing multiple lines in sed or awk [closed]

I'm trying to use sed or awk to replace 5 lines in a smb file but I just don't have any idea how to deal with the newlines.
user avatar

1
150 151 152 153
154