0

I have a text file that contains the log info about the webserver.

DATE Format: Day-Month-Year

Sample content:

/tmp/archive/9-10-2020/error_04.log.gz
/tmp/archive/9-10-2020/error_05.log.gz
/tmp/archive/9-7-2020/access_01.log.gz
/tmp/archive/9-7-2020/access_02.log.gz
/tmp/archive/9-7-2020/access_03.log.gz
/tmp/archive/9-7-2020/error_03.log.gz
/tmp/archive/9-7-2020/error_04.log.gz
/tmp/archive/9-7-2020/error_05.log.gz
/tmp/archive/9-8-2020/error_01.log.gz
/tmp/archive/9-8-2020/error_02.log.gz
/tmp/archive/9-8-2020/error_03.log.gz
/tmp/archive/9-8-2020/error_04.log.gz
/tmp/archive/9-8-2020/error_05.log.gz
/tmp/archive/9-9-2020/access_01.log.gz
/tmp/archive/9-9-2020/access_02.log.gz
/tmp/archive/9-9-2020/access_03.log.gz

I want to list this content based on the date order(the 3rd column). I tried sort command, its not giving the sort by date.

Expected output:

/tmp/archive/9-7-2020/access_01.log.gz
/tmp/archive/9-7-2020/access_02.log.gz
/tmp/archive/9-7-2020/access_03.log.gz
/tmp/archive/9-7-2020/error_03.log.gz
/tmp/archive/9-7-2020/error_04.log.gz
/tmp/archive/9-7-2020/error_05.log.gz
/tmp/archive/9-8-2020/error_01.log.gz
/tmp/archive/9-8-2020/error_02.log.gz
/tmp/archive/9-8-2020/error_03.log.gz
/tmp/archive/9-8-2020/error_04.log.gz
/tmp/archive/9-8-2020/error_05.log.gz
/tmp/archive/9-9-2020/access_01.log.gz
/tmp/archive/9-9-2020/access_02.log.gz
/tmp/archive/9-9-2020/access_03.log.gz
/tmp/archive/9-10-2020/error_04.log.gz
/tmp/archive/9-10-2020/error_05.log.gz

Update:

Sort syntax:
sort -k4.7,4.11 -k4,5

/tmp/backup/7-12-2020/access_04.log
/tmp/backup/7-12-2020/error_02.log
/tmp/backup/7-12-2020/error_03.log
/tmp/backup/7-12-2020/error_04.log
/tmp/backup/7-12-2020/error_05.log
/tmp/backup/8-11-2020/access_01.log
/tmp/backup/8-11-2020/access_02.log
/tmp/backup/8-12-2020/error_01.log
/tmp/backup/8-12-2020/error_02.log
/tmp/backup/8-12-2020/error_03.log
/tmp/backup/8-12-2020/error_04.log
/tmp/backup/8-12-2020/error_05.log
/tmp/backup/9-11-2020/access_01.log
/tmp/backup/9-11-2020/access_02.log
/tmp/backup/9-11-2020/access_03.log
/tmp/backup/9-11-2020/access_04.log

6
  • 2
    If you wrote your log files with dates in the format yyyy-mm-dd you wouldn't have this sorting problem
    – roaima
    Dec 18 '20 at 10:13
  • I agree with the date format, but i have to process the existing files.
    – TheDataGuy
    Dec 18 '20 at 10:14
  • "3rd column" presumably means "4th /-separated field"?
    – roaima
    Dec 18 '20 at 10:16
  • yes, you are right
    – TheDataGuy
    Dec 18 '20 at 10:17
  • 2
    So is there a requirement also to maintain the implied ordering of the filenames access_01.log.gz, access_02.log.gz, etc. for a given date?
    – roaima
    Dec 18 '20 at 10:18
1

For a specific pattern such as this you can split the pathname into its / and - separated components, putting them at the beginning of the line,

awk '{
    split($0, f, "[/-]");
    printf "%04d-%02d-%02d\t%s\t%s\n", f[6], f[5], f[4], f[7], $0
}'

then sort the date (yyyy-mm-dd) and filename (e.g. access_NN.log.gz) accordingly

sort

and finally strip off the sorting components

cut -f3-

Assuming the sample data is in the file /tmp/logs you can put it together like this

awk '{ split($0, f, "[/-]"); printf "%04d-%02d-%02d\t%s\t%s\n", f[6], f[5], f[4], f[7], $0 }' /tmp/logs |
    sort |
    cut -f3-

/tmp/archive/9-7-2020/access_01.log.gz
/tmp/archive/9-7-2020/access_02.log.gz
/tmp/archive/9-7-2020/access_03.log.gz
/tmp/archive/9-7-2020/error_03.log.gz
/tmp/archive/9-7-2020/error_04.log.gz
/tmp/archive/9-7-2020/error_05.log.gz
/tmp/archive/9-8-2020/error_01.log.gz
/tmp/archive/9-8-2020/error_02.log.gz
/tmp/archive/9-8-2020/error_03.log.gz
/tmp/archive/9-8-2020/error_04.log.gz
/tmp/archive/9-8-2020/error_05.log.gz
/tmp/archive/9-9-2020/access_01.log.gz
/tmp/archive/9-9-2020/access_02.log.gz
/tmp/archive/9-9-2020/access_03.log.gz
/tmp/archive/9-10-2020/error_04.log.gz
/tmp/archive/9-10-2020/error_05.log.gz
1
  • Just for my understanding, split($0, f, "[/-]") how this convert the dd-mm-yyyy to yyyy-mm-dd ?
    – TheDataGuy
    Dec 18 '20 at 18:44
0

Here, assuming all lines always start with /tmp/archive/ or anything the same length, you could do:

sort -t- -k3,3.4n -k1.14,1n -k2,2n -k3.5

Which here you can simplify it to:

sort -t- -k3n -k1.14n -k2n -k3.5

as - is not going to be a thousand separator in any locale (since it's also the character for the negative sign), so with the n flag, a key specification of -k1.14n (which would select 7-12-2020/access_04.log on the first line for instance) or -k1.14,1n (which would select only 7) would both yield the 7 number.

1
  • Its sorting only based on day (like 1-1-2020,1-2-2020,2-1-2020)
    – TheDataGuy
    Dec 18 '20 at 17:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.