0

I have a log file that contains the pings from a certain website in a text file called pingoutput.txt separating each ping reply by line. Now I need to extract from this text file the roundtrip time in between time= and ms into either another text file or list that I could then sort smallest to largest.

64 bytes from onofri.org (67.222.36.105): icmp_req=1 ttl=47 time=202 ms
64 bytes from onofri.org (67.222.36.105): icmp_req=2 ttl=47 time=206 ms
64 bytes from onofri.org (67.222.36.105): icmp_req=3 ttl=47 time=215 ms

Also the pingout.txt file is large and has about 86,400 lines. I'm doing this through a shell script on linux.

  • With GNU grep: grep -Po 'time=\K\d+' file | sort -n – Stéphane Chazelas Sep 28 '16 at 14:09
  • 1
    86K lines hasn't been large in several decades. – Kaz Sep 28 '16 at 15:30
2

This works for me:

sed 's/.*time=\([0-9]*\) .*/\1/' times | sort -n > outfile

Where times is this file:

cat times 
64 bytes from onofri.org (67.222.36.105): icmp_req=1 ttl=47 time=202 ms
64 bytes from onofri.org (67.222.36.105): icmp_req=2 ttl=47 time=206 ms
64 bytes from onofri.org (67.222.36.105): icmp_req=3 ttl=47 time=215 ms

And outfile looks like this:

cat outfile 
202
206
215
  • 1
    If you do not put -n option in the sort, if you have times under 100 ms it will not sort properly. – Rob Sep 28 '16 at 14:41
  • 1
    @Rob Oops! Thanks for pointing this out. Edited as per your suggestion. – maulinglawns Sep 28 '16 at 15:54
1

You can also use Perl regex and grep

grep -oP '(?<=time\=).*' pingoutput 
  • Probably it is better to use \S+ instead of .*, so that you just get the digits (everything up to the space). – fedorqui Sep 28 '16 at 14:11
1

You can extract the time with this sample script:

awk -F\= '{print int($4)}' pingoutput.txt 

It use = as delimiter and use int function to get only the numbers of "202 ms" string

To redirect output to other file use command:

awk -F\= '{print int($4)}' pingoutput.txt > times.txt

For sorting the output you can pipe output of awk to sort command

awk -F\= '{print int($4)}' pingoutput.txt |sort -n > times.txt
  • Amazing, but I see its printing, how could I send the time to a list or different text file instead of just printing? – user192314 Sep 28 '16 at 12:41
  • See my updated answer – Romeo Ninov Sep 28 '16 at 12:42
1

The main tools for this sort of work are sed and awk. They are designed to read a pipe or file line by line and transform it.

Example sed solution:

  sed 's/.* time=\([0-9]*\) ms/\1/' pingoutput.txt 

Should output:

  202
  206
  215

Breakdown of the s/.* time=\([0-9]*\) ms/\1/:

s means you want to do a substitution command. Substitution commands have the pattern s/<pattern to match>/<replacement pattern>/.

The <pattern to match> is .* time=\([0-9]*\) ms

The . is any character. The * says that any character can appear zero or more times. This is to consume all characters up until the time= part of the line.

The \( \) part defines what is called a group. Whatever is matched by the pattern in between the parentheses will be sorted in a group named 1 since there is only one group in our pattern.

The part inside the \( \) is [0-9]*, which says match any character between the character 0 and the character 9 (all digits), and again the * means there could be zero or more. This will match the number you are interested in.

Finally we have ms at the end to consume it from the line.

The <replacement pattern> is just \1; this means you want to replace the entire matched string with what was captured by group #1 in the match pattern.

You can do it with awk too; I recommend learning how to use both tools.

Addition:

To sort the value numerically you need to stream the results using the '|' into the sort tool. But in order to make sure the times are sorted numerically you want to use sort -n, otherwise you can get a weird sorting order.

sed 's/.* time=\([0-9]*\) ms/\1/' pingoutput.txt | sort -n

Further Addition to support decimal numbers

sed 's/.* time=\([0-9]*\(\.[0-9]*\)\{0,1\}\) ms/\1/' pingoutput.txt | sort -n

What I did was add a optional part that describes the decimal part of the number within the group that was \([0-9]*\) I added:

Anther sub group that can appear 0 or 1 time only that is described by \(\)\{0,1\} the part in {} the first number is the minimum occurrence the second number maximum. The pattern inside that group is \.[0-9]* the reason we have to but \ in from of the '.' is that otherwise as you know now '.' means any character so for it to only allow a '.' you have to escape it with \

  • I just realized some of my values may contain decimals for example 12.6 ms. will the 0-9 limiter ignore the decimal point? Is there any way to take in that entire value now and still sort? – user192314 Sep 29 '16 at 11:38
  • Updating solution. :) – Rob Sep 29 '16 at 11:39
0

If you have the pcre option with grep

$ grep -oP 'time=\K\d+' pingout.txt 
202
206
215
$ grep -oP 'time=\K\d+' pingout.txt | sort -n
202
206
215
$ grep -oP 'time=\K\d+' pingout.txt | sort -nr
215
206
202
  • time=\K positive lookbehind the string time= - this is not part of output
  • \d+ one or more digits, can also use [^ ]+ to extract characters other than space

To save the output to a file, use the > redirection

$ grep -oP 'time=\K\d+' pingout.txt | sort -n > op.txt
$ cat op.txt 
202
206
215
0
awk -F'[ =]' '{ print $11 }' logfile

If 0 ms ping times never occur, we can "golf" it:

awk -F[\ =] '$0=$11'

If there is a column 11, $0 gets a non-blank, non-zero value. Such a value is Boolean true, triggering the action. The action is missing so the default action is { print }, which means { print $0 }.

The regular expression [ =] matches an equal sign or space, and with -F we install that as the FS variable: Awk's field separator. Each line of input is broken into fields by removing a space or equal sign. The fields are the pieces of the line which remain around these separators (even if empty: two consecutive spaces will have a field between them). Under this separation, it turns out that the ping times end up in the 11th column, and so we use the $ parameter access operator with argument 11 to refer to it.

If zeros occur, we can still golf it:

awk -F[\ =] '$0=$11||1'

If $0=$11 yields zero/false, then the right hand side of the || (logical OR) operator gets evaluated and that is the value of the expression. Thus we ensure the expression always yields true so the print action takes place.

-1
awk -F\= '{print int($4)}' test | sort -n  > t
-1

Use the sed command to extract the related fields. like

sed 's/.*time=\([0-9.]\+\).*/\1/p' test.log |sort

$ cat test.log
PING 10.74.88.113 (10.74.88.113) 56(84) bytes of data.
64 bytes from 10.74.88.113: icmp_seq=1 ttl=59 time=0.326 ms
64 bytes from 10.74.88.113: icmp_seq=2 ttl=59 time=0.333 ms
64 bytes from 10.74.88.113: icmp_seq=3 ttl=59 time=0.322 ms
64 bytes from 10.74.88.113: icmp_seq=4 ttl=59 time=0.335 ms
64 bytes from 10.74.88.113: icmp_seq=5 ttl=59 time=0.337 ms
  • \d might no be supported. You should use [0-9] – Raphael Ahrens Sep 28 '16 at 13:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.