3

I want to sort the file based on the

  • text before the first comma, but also
  • want a second level sort based on the date YYYYMMMDD in the sample input:

Sample Input:

AX,NO,NO,\PathAX/subj/defn/some-file-name-20151229.txt,
CXX,NO,NO,\PathCXX/subj/defn/some-file-name-20151229.txt,
CXX,YES,YES,\Path/subj/defn/some-file-20140503.txt,6
DY,YES,MAYBE,\PathDY/subj/defn/some-file-name-20140720.txt,6

I can solve the first part with sort -t, -k1,1, but don't know how to daily do the second part as the number of dashes in the file name is not fixed. Thus I can't just specify the column number with a - as a separator.

Desired output:

AX,NO,NO,\PathAX/subj/categ/some-file-name-20151229.txt,
CXX,YES,YES,\Path/subj/categ/some-file-20140503.txt,6
CXX,NO,NO,\PathCXX/subj/categ/some-file-name-20151229.txt,
DY,YES,MAYBE,\PathDY/subj/categ/some-file-name-20140720.txt,6

Notes:

I case it helps:

  • The YYYYMMDD follows the last dash in the file name and before the file extension.
  • The dash prior to YYYYMMDD is always present, thus at least one dash.
  • There are no other dots in the file, except for the file extension.

I can think of a two pass process where the last - is replaced by some other character that will not appear in the file, and then sort and replace the character back with a dash, but wondering if there is an easier way.

Platform:

  • MacOS 10.9.5
  • I would use another character e.g. \x02 (that's unlikely to appear in a text file) to delimit the file into 3+ fields, sort by 1st and 3rd field using \x02 as delimiter then remove that character e.g. sed 's/\(,.*-\)\([0-9]*\)\./\x02\1\x02\2\x02./' infile | sort -t$'\002' -k1,1 -k3,3 | tr -d $'\002' . I don't know if this works with osx sed/sort (works fine on linux). – don_crissti Mar 20 '16 at 11:51
  • This is better answer than mine, so it should be accepted as corrected. – Boban P. Mar 20 '16 at 14:43
2

A general approach for this kind of problem (sorting on a field that just can't be described as the Nth on the line) is to rewrite the lines to prepend the sorting key(s), then sort, then remove the sorting keys. You can use more flexible tools such as sed or awk to determine the sorting keys.

I don't know exactly how you determine where the date is. I'll go for the last sequence of 8 digits on the line, tweak as necessary.

sed 's/.*\([0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]\)/\1,&/; t; s/^/,/' |
sort -t, -k2,2 -k1,1 |
cut -d, -f 2-

The t; s/^/,/ bit in the sed script prepends a comma if the line doesn't contain any sequence of 8 digits, otherwise the last step would remove the first comma-separated field.

  • [0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] should be replaced by [0-9]\{8\} – Boban P. Mar 20 '16 at 17:15
2

You can use something like this:

rev text | sed 's/-/,/1' | rev | sort -t, -k1,1 -k5,5 | rev | sed 's/,/-/2' | rev

where text is file with you contents. This works if there is one comma anywhere after timestamp, and if there is no commas in filename.

And what it actually does?

  1. Reverse the line
  2. change first '-' to ',' in reversed line (this is actaully last '-' in real line)
  3. reverse it again so it matches real line
  4. now sort it on first and fifth field (datetime), with ',' as a field separator
  5. reverse it again
  6. change 2nd occurrence of ',' to '-' (back as it was before)
  7. reverse it and print output
  • Awesome. I was not aware of rev. – Peter Grill Mar 20 '16 at 13:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.