0

I have a text file that contains the log info about the webserver.

DATE Format: Day-Month-Year

Sample content:

/tmp/archive/9-10-2020/error_04.log.gz
/tmp/archive/9-10-2020/error_05.log.gz
/tmp/archive/9-7-2020/access_01.log.gz
/tmp/archive/9-7-2020/access_02.log.gz
/tmp/archive/9-7-2020/access_03.log.gz
/tmp/archive/9-7-2020/error_03.log.gz
/tmp/archive/9-7-2020/error_04.log.gz
/tmp/archive/9-7-2020/error_05.log.gz
/tmp/archive/9-8-2020/error_01.log.gz
/tmp/archive/9-8-2020/error_02.log.gz
/tmp/archive/9-8-2020/error_03.log.gz
/tmp/archive/9-8-2020/error_04.log.gz
/tmp/archive/9-8-2020/error_05.log.gz
/tmp/archive/9-9-2020/access_01.log.gz
/tmp/archive/9-9-2020/access_02.log.gz
/tmp/archive/9-9-2020/access_03.log.gz

I want to list this content based on the date order(the 3rd column). I tried sort command, its not giving the sort by date.

Expected output:

/tmp/archive/9-7-2020/access_01.log.gz
/tmp/archive/9-7-2020/access_02.log.gz
/tmp/archive/9-7-2020/access_03.log.gz
/tmp/archive/9-7-2020/error_03.log.gz
/tmp/archive/9-7-2020/error_04.log.gz
/tmp/archive/9-7-2020/error_05.log.gz
/tmp/archive/9-8-2020/error_01.log.gz
/tmp/archive/9-8-2020/error_02.log.gz
/tmp/archive/9-8-2020/error_03.log.gz
/tmp/archive/9-8-2020/error_04.log.gz
/tmp/archive/9-8-2020/error_05.log.gz
/tmp/archive/9-9-2020/access_01.log.gz
/tmp/archive/9-9-2020/access_02.log.gz
/tmp/archive/9-9-2020/access_03.log.gz
/tmp/archive/9-10-2020/error_04.log.gz
/tmp/archive/9-10-2020/error_05.log.gz

Update:

Sort syntax:
sort -k4.7,4.11 -k4,5

/tmp/backup/7-12-2020/access_04.log
/tmp/backup/7-12-2020/error_02.log
/tmp/backup/7-12-2020/error_03.log
/tmp/backup/7-12-2020/error_04.log
/tmp/backup/7-12-2020/error_05.log
/tmp/backup/8-11-2020/access_01.log
/tmp/backup/8-11-2020/access_02.log
/tmp/backup/8-12-2020/error_01.log
/tmp/backup/8-12-2020/error_02.log
/tmp/backup/8-12-2020/error_03.log
/tmp/backup/8-12-2020/error_04.log
/tmp/backup/8-12-2020/error_05.log
/tmp/backup/9-11-2020/access_01.log
/tmp/backup/9-11-2020/access_02.log
/tmp/backup/9-11-2020/access_03.log
/tmp/backup/9-11-2020/access_04.log

6
  • 2
    If you wrote your log files with dates in the format yyyy-mm-dd you wouldn't have this sorting problem Dec 18, 2020 at 10:13
  • I agree with the date format, but i have to process the existing files.
    – TheDataGuy
    Dec 18, 2020 at 10:14
  • "3rd column" presumably means "4th /-separated field"? Dec 18, 2020 at 10:16
  • yes, you are right
    – TheDataGuy
    Dec 18, 2020 at 10:17
  • 2
    So is there a requirement also to maintain the implied ordering of the filenames access_01.log.gz, access_02.log.gz, etc. for a given date? Dec 18, 2020 at 10:18

2 Answers 2

1

For a specific pattern such as this you can split the pathname into its / and - separated components, putting them at the beginning of the line,

awk '{
    split($0, f, "[/-]");
    printf "%04d-%02d-%02d\t%s\t%s\n", f[6], f[5], f[4], f[7], $0
}'

then sort the date (yyyy-mm-dd) and filename (e.g. access_NN.log.gz) accordingly

sort

and finally strip off the sorting components

cut -f3-

Assuming the sample data is in the file /tmp/logs you can put it together like this

awk '{ split($0, f, "[/-]"); printf "%04d-%02d-%02d\t%s\t%s\n", f[6], f[5], f[4], f[7], $0 }' /tmp/logs |
    sort |
    cut -f3-

/tmp/archive/9-7-2020/access_01.log.gz
/tmp/archive/9-7-2020/access_02.log.gz
/tmp/archive/9-7-2020/access_03.log.gz
/tmp/archive/9-7-2020/error_03.log.gz
/tmp/archive/9-7-2020/error_04.log.gz
/tmp/archive/9-7-2020/error_05.log.gz
/tmp/archive/9-8-2020/error_01.log.gz
/tmp/archive/9-8-2020/error_02.log.gz
/tmp/archive/9-8-2020/error_03.log.gz
/tmp/archive/9-8-2020/error_04.log.gz
/tmp/archive/9-8-2020/error_05.log.gz
/tmp/archive/9-9-2020/access_01.log.gz
/tmp/archive/9-9-2020/access_02.log.gz
/tmp/archive/9-9-2020/access_03.log.gz
/tmp/archive/9-10-2020/error_04.log.gz
/tmp/archive/9-10-2020/error_05.log.gz
1
  • Just for my understanding, split($0, f, "[/-]") how this convert the dd-mm-yyyy to yyyy-mm-dd ?
    – TheDataGuy
    Dec 18, 2020 at 18:44
0

Here, assuming all lines always start with /tmp/archive/ or anything the same length, you could do:

sort -t- -k3,3.4n -k1.14,1n -k2,2n -k3.5

Which here you can simplify it to:

sort -t- -k3n -k1.14n -k2n -k3.5

as - is not going to be a thousand separator in any locale (since it's also the character for the negative sign), so with the n flag, a key specification of -k1.14n (which would select 7-12-2020/access_04.log on the first line for instance) or -k1.14,1n (which would select only 7) would both yield the 7 number.

1
  • Its sorting only based on day (like 1-1-2020,1-2-2020,2-1-2020)
    – TheDataGuy
    Dec 18, 2020 at 17:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .