I have a log file containing such entries;query=/etc/security/limits.conf;date=02.03.16;query=/etc/security/limits.conf;date=01.06.15;query=/etc/security/limits.conf;date=02.03.16;query=/etc/security/limits.conf;date=12.07.15;query=/etc/security/limits.conf;date=03.01.16;query=/etc/security/limits.conf;date=02.11.15

I need to extract only uniq entries where only once every IP occured. So it should be;query=/etc/security/limits.conf;date=02.03.16;query=/etc/security/limits.conf;date=02.03.16

No matter what come after IP. Log file is long. I guess it should be combination of uniq -u command.


1 Answer 1


Assuming your dates are in mm.dd.yy format

sort -k1,1 -k3.12,3.13nr -k3.6,3.7nr -k3.9,3.10nr -t';' file  |
 sort -k1,1 -u -t';';query=/etc/security/limits.conf;date=02.03.16;query=/etc/security/limits.conf;date=03.01.16

Sort by the IP field followed by the date field in reverse order (thus the most recent dates for each IP sort first). Pipe this to another sort, this time by the IP field but specifying -u to ensure one record for each IP is returned. Since -u implies stable sort, the first record for each IP (the one with the most recent date) is returned

If your dates on the other hand are in dd.mm.yy format

sort -k1,1 -k3.12,3.13nr -k3.9,3.10nr -k3.6,3.7nr -t';' file |
sort -k1,1 -u -t';';query=/etc/security/limits.conf;date=02.03.16;query=/etc/security/limits.conf;date=02.03.16
  • Could you please explain how -k works ? From man: -k, --key=KEYDEF sort via a key; KEYDEF gives location and type Actually this short works for my task sort -t\; -uk1,1 someone posted and deleted this. But you gave very advanced sorting, good job!
    – Demontager
    Commented Mar 5, 2016 at 2:59
  • @Demontager, you're probably better off looking at the description of -k on the info page for sort. Don't forget to check the examples at the end as well. These do a better job at explaining than I ever could
    – iruvar
    Commented Mar 5, 2016 at 3:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .