1

I have a file consisting of several lines and columns. The columns are delimited by \t. Now I would like to sort the file based on the second column, which can be done with, e.g. sort -f -t$'\t' -k2 file.txt. However, the second column consists of the following values: +, o, and -. How can I sort such that + is on top, o in the middle, and - on the bottom?

Furthermore, I have another column, say column 5, and would like to sort the file w.r.t. that column. However, I want to specify my personal order, so that n is on top, s is in the middle, and l is on the bottom.

If there is a program, which can achieve my goals efficiently, I would prefer that solution. Otherwise I'm looking for a self-written function.

  • How big is your file? AFAIK the sort utility doesn't support plugging in your own comparator, and there are no other readily available tools capable of sorting huge files. If loading the whole file in the memory is OK, then try perl -e 'sub k { index "+o-", ($_[0]=~/\t(.)/, $1) } print sort {k($a)<=>k($b) } <>' file – mosvy Jan 19 at 18:37
  • Of course, you can probably define your own locales & collation rules, and get sort to use them: I too am interested in the whole recipe ;-) – mosvy Jan 19 at 18:43
  • When you say second row consists ..., you mean second column ? Do column 2 and 5 only contain single letter ? – Philippe Jan 19 at 18:46
  • It's a small file consisting of no more than 100 lines whereby each line represents a task. Tasks have a priority (+, o, -) and deadline ((n)ow, (s)oon, (l)ater). No other values occur in the mentioned rows. – hyhno01 Jan 19 at 18:49
  • 1
    That would take a whole perl tutorial, but: just perl -ne 'print sort <>' file will fetch all the input lines, sort and print them. But perl's sort allows you to use your own comparator, as in sort {k($a)<=>k($b)} <>; in my example, the k function returns 0, 1, 2 based on whether the 2nd column starts with -, o or +. For your extra n, s, l, you can change it to: perl -e 'sub k { $_[0] =~ s/.*?\t//r =~ y/-o+nsl/abcabc/r } print sort {k($a) cmp k($b)} <>' file – mosvy Jan 19 at 19:20
0

We can still use sort, in case it's a big file :

#!/usr/bin/env bash

perl -F'\t' -ne '{$F[1] =~ s/o/,/; $F[4] =~ s/l/t/; print join "\t",@F}' file|\
sort -t $'\t' -k 2,2 -k 5,5 |\
perl -F'\t' -ne '{$F[1] =~ s/,/o/; $F[4] =~ s/t/l/; print join "\t",@F}'

The first perl script changes o to , on the second column, l to t on the fifth, to prepare the sort.

The second perl script changes back.

| improve this answer | |
0

There is a technique called DSU, decorate-sort-undecorate, or the Schwartzian transform.

The basic method is to invent your own substitute keys, which can be any combinations of your data (extracted from any parts of your records, perhaps dependant on other fields). These have only to be rankings in the required order of significance. For example, zero-padding numeric fields is often helpful. You can make the final order stable by adding the input line number as the final subkey. The total key length needs to be the same for all records.

Then you sort them according to the -k1.1 order (whole record). This can often be faster that a multikey sort, because typically a sort will reconstruct two multikeys for every comparison.

Finally, you undecorate the sorted output by cutting off the fixed-length decoration.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.