I have a text file (space as delimiter) as shown below:

Date           Time---------OtherFields-------Source IP   Destination IP 

2014-11-24  12:58:59.290            

2014-11-24  12:59:01.402            

2014-11-24  13:00:01.542            

2014-11-25  14:00:02.5              

2014-11-25  15:00:01.542            

2014-11-25  16:00:01.542            

For Destination IP, I want to select a row with latest date and time and save it in a new text file for further processing. Same thing applies to all different Destination IP addresses. There are tons of rows like this.

The desired output should be like this.

2014-11-25  14:00:02.5              

2014-11-25  16:00:01.542            

No logic is clicking to me.Any help would be appreciated. Thank you

for dest in $(awk '{print $NF}' mylogfile | sort | uniq)
  grep ${dest} mylogfile | sort -k1 -k2 | tail -1

1    $(awk '{print $NF}' mylogfile | sort | uniq)

print the last field of each line in logfile, which is your destrination IP. sort it so that identical ip addresses are in consecutive blocks. uniq is to print only one instance of identical blobk of lines

2 & 4   do ... done   

I think these are self explanatory. The general loop construct

3    grep ${dest} mylogfile | sort -k1 -k2 | tail -1

find lines with one IP address (assumes this ip address is not repeating itself in any other field but the last one), sort on field 1 first and then field 2, latest of the date/time stamp falls into the last line. tail -1 catches this last line

  • Could you please explain the code – Swatesh Pakhare Jan 19 '16 at 16:43

You can try with following awk:

awk '{ x = $1" "$2; if (x > a[$4]) { a[$4] = x; b[$4] = $0; } } END { for (i in b) { print b[i]; } }' file

But as it holds two arrays in memory may be not very useful for very big files.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.