1

I have a file with following contents. I am looking to sort this file based on the last column (and third last but for another file) while retaining rest of the contents of the row.

ABC,DEF,GHI,-5,-8,-0.6,0.488 
XYZ,JKL,MNO,3,-5,0.2,-0.342 
STU,WXY,DEF,-1,4,0.01,0.345 

If I use this command, it works as expected and shows the correct results:

awk '{print $NF,$0}' FILE | sort -nr | cut -f2- -d' '
XYZ,JKL,MNO,3,-5,0.2,-0.342 
STU,WXY,DEF,-1,4,0.01,0.345
ABC,DEF,GHI,-5,-8,-0.6,0.488 

But the same command on a bigger file gives an incorrect result. (The file I am looking to sort has 4M rows) Input:

ABC,DEF,GHI,-5,-8,-0.6,0.0488 
XYZ,JKL,MNO,3,-5,0.2,-0.0342 
STU,WXY,DEF,-1,4,0.01,0.0345 
JKL,JKL,GHI,-2,-3,0.31,-0.0524 
QRS,GHI,YUT,-3,-1,0.20,-0.0503 
HUR,JTL,ZST,1,1,0.52,-0.0556 
FTT,JL,MKI,0,2,0.21,-0.0529 
FTC,JKL,ERW,-1,6,0.23,-0.0441 
HJI,MHP,VGT,1,-6,0.80,-0.0433 
BUT,IOP,HGT,2,2,0.2,-0.0439 
XYZ,BGY,MNO,-2,1,0.01,-0.0416 
1
  • 2
    Is the problem that you don't know how many fields will be in your input or something else? Regarding But the same command on a bigger file gives an incorrect result. - please tell us what that incorrect result is and create a minimal example that demonstrates that problem (if you don't know how else to do it, just do divide-and-conquer on your failing input to get to the minimal lines you need to reproduce it).
    – Ed Morton
    Sep 22, 2021 at 14:00

2 Answers 2

1

If you know how many fields you have:

$ sort -t, -k7,7n file
XYZ,JKL,MNO,3,-5,0.2,-0.342
STU,WXY,DEF,-1,4,0.01,0.345
ABC,DEF,GHI,-5,-8,-0.6,0.488

or if you don't:

$ awk 'BEGIN{FS=OFS=","} {print $NF,$0}' file | sort -t, -k1,1n | cut -d, -f2-
XYZ,JKL,MNO,3,-5,0.2,-0.342
STU,WXY,DEF,-1,4,0.01,0.345
ABC,DEF,GHI,-5,-8,-0.6,0.488

and to sort by the 3rd-last instead of last field would obviously just be:

$ awk 'BEGIN{FS=OFS=","} {print $(NF-2),$0}' file | sort -t, -k1,1n | cut -d, -f2-
ABC,DEF,GHI,-5,-8,-0.6,0.488
XYZ,JKL,MNO,3,-5,0.2,-0.342
STU,WXY,DEF,-1,4,0.01,0.345

If you want to retain input order when multiple rows have the same value for the field you're sorting on then if you have GNU sort you can use -s, otherwise include the line number as a secondary sort key:

$ awk 'BEGIN{FS=OFS=","} {print $NF,NR,$0}' file | sort -t, -k1,1n -k2,2n | cut -d, -f3-
XYZ,JKL,MNO,3,-5,0.2,-0.342
STU,WXY,DEF,-1,4,0.01,0.345
ABC,DEF,GHI,-5,-8,-0.6,0.488
2
  • I don't see why you would want to pass the input through awk first, and then pass it through cut one more time, and then comment about "efficiency" on my answer. Objectively speaking, your answer does more work than my original answer. Sep 22, 2021 at 16:37
  • 1
    I'm doing what I do because it's very robust, efficient, portable, and extendable to do so, will work even if the input is coming from a pipe rather than a file, and decorate/sort/undecorate is an extremely common idiom. I commented on your answer because you were reading 4 million lines unnecessarily and just discarding them when it's a trivial tweak to simply not do that and thought you'd appreciate the constructive feedback.
    – Ed Morton
    Sep 22, 2021 at 16:46
0

Assuming the input is relatively simple (delimiter does not appear in the records) you can sort by a given column like this:

$ c=$(< input awk -F, 'NR==1 { print NF; exit }')
$ < input sort -t, -k $c,${c}n --debug

output:

XYZ,JKL,MNO,3,-5,0.2,-0.342
                     ______
___________________________
STU,WXY,DEF,-1,4,0.01,0.345
                      _____
___________________________
ABC,DEF,GHI,-5,-8,-0.6,0.488
                       _____
____________________________

Here, we say that sort is using a comma as the delimiter between fields and sort (only) by a specific field, which we figured out to be last in the previous step.

By the way, your question is likely to get closed because no doubt it has been asked and answered before.

2
  • 2
    Please note that many users here, Ed included, comment on posts to improve them (if they don't edit the post directly themselves). That's just the nature of Stack Exchange, here. Please take their comments in a positive light and see whether you can incorporate their feedback.
    – Jeff Schaller
    Sep 22, 2021 at 17:10
  • 2
    Separately, if you believe a question is going to get closed, consider looking at that duplicate question to see if you can provide another answer there, then vote to close this question as a duplicate. There's really no need to put "your question will get closed" as part of an answer. Thank you!
    – Jeff Schaller
    Sep 22, 2021 at 17:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .