sort multiple files and merge

Question

I want to sort several text files in reverse order and then merge/cat to one single text file.

a.txt

0   33.1
2   33.0
10  21.1
20  21.8

b.txt

0   30.1
2   33.0
10  28.1
20  27.8

and so on *.txt files

I want output like this

20  21.8
10  21.1
2   33.0
0   33.1
20  27.8
10  28.1
2   33.0
0   30.1

I don't want like this

20  21.8
20  27.8
10  21.1
10  28.1
2   33.0
2   33.0
0   33.1
0   30.1

I tried these code

for file in *.txt ; do 
   sort -nrk 1,1 *.txt > "$file" ; 
done

and also I tried

sort -m *.txt

But the output from these codes is not I wanted.

I am looking for solutions using sort merge paste cat or some other relevant options.

Many thanks for any help.

Answer 1

Sort files individually, and redirect the whole output to the resulting file:

for file in *.txt ; do 
   sort -k1,1rn < "$file"
done > file.concatenated

(here it's important the output file doesn't have a .txt extension as it's created first by the redirection).

Or if you want to sort the files in place (rewriting them sorted over themselves):

set -- *.txt
ok=true
for file do
  sort -o "$file" -k1,1rn -- "$file" || ok=false
done
"$ok" && cat -- "$@" > file.concatenated

That two stage approach allows us to detect problems in the sorting of files before creating the concatenated file.

Your first loop didn't work as you were passing the full list of .txt files in each pass of the loop.

sort -m is to merge already sorted files into a sorted output. It's the opposite of what you want. You want to sort files that are not already sorted and just concatenate the results without merging them into a sorted output.

Here, the files seem to be sorted in forward order. If you can rely on that being always the case, you should be able to just reverse them which would be much more efficient than sorting them in reverse.

To do that, GNU systems have a tac command, and several others tail -r (though beware that some implementations only take one file argument, so you may need to resort to loop with those).

tac -- *.txt > file.concatenated

Also note that -k1,1rn is not the same as -rnk1,1 when it comes to resolving ties. When two lines compare equally, sort resorts to a lexical comparison of the whole line (for instance here, 1 a and 1 b compare equally with -k1,1n, but 1 a comes before 1 b lexically).

With the -r option, that last resort comparison is done in reverse. That doesn't apply when a r flag is added to one of the key specifications.

GNU sort has -s to disable that last resort comparison which will cause it to preserve the original order of the lines that compare equally.