Sort files by highest number in filename

Question

I've got a bunch of files all named like this:

name_file-1.txt
name_file-2.txt
name_file-3.txt
some_other_file-1.txt
some_other_file-2.txt

There are thousands of different filenames, some with just one -1.txt at the end, some with -1.txt, -2.txt ... -60.txt

I need to copy the highest numbers of each file, so name_file-3.txt, some_other_file-2.txt. How do I do that on a Linux command line?

Answer 1

With zsh:

typeset -A greatest
for f (*-*(n)) greatest[${f%-*}]=$f
cp -- $greatest /destination

• *-*(n): non-hidden files whose name contains a - (*-*), sorted numerically ((n) glob qualifier).
• ${f%-*}: part of the filename up to the right-most - (or to the end if there's no -).
• $greatest: expands to the non-empty values of the associative arrays. So here, for files that share the same root, only the file with the greatest number will be expanded.

Answer 2

files=(*)
mapfile -t prefixes < <(printf "%s\n" "${files[@]%-*}" | sort -u)
for p in "${prefixes[@]}"; do ls -v "$p"* | tail -1; done

name_file-3.txt
some_other_file-2.txt

And then to copy those to some other directory:

for ...; done | xargs cp -t /destination/directory

Answer 3

If the files are in the current working directory and their names conform to the samples (a single dash which precedes a number), the following POSIX-compliant pipeline should work:

ls | sort -t- -k1,1 -k2,2rn | awk -F- 'k!=$1 {print; k=$1}' | pax -rw /path/to/dir

The awk component can be replaced by a sort -u, if the sort's -u option is stable (so that the first line of a set is always chosen to represent that set). POSIX does not require this stability, but, according to their manuals, the {Free,Net,Open}BSD and GNU implementations provide it. If you enjoy tempting fate:

ls | sort -t- -k1,1 -k2,2rn | sort -mut- -k1,1 | pax -rw /path/to/dir

In either case, the target directory must not be in the current working directory.

Answer 4

If pwd is the source dir

cp -t /path_to_destination $(
      ls -v *[0-9].txt |
      sed '$b;N;/^\(.*\)[0-9]\+\.txt\n\1[0-9]\+\.txt/!P;D')

NOTE: If there are any spaces in file names it should be prepared before by escaping to proper operation. + see other restrictions in Stéphane Chazelas' comments

Answer 5

I'd split the file into tab-delimited parts for more reliable, customizable filename parsing, then use awk to find the highest rank of each and report. Try each part of the pipeline first before proceeding to next!

find DIR -type f <other find criteron> -print | 
perl -lne 'print join("\t",(/^(.*?-)(\d+)(\.\w+)$/))' |
awk -F\\t '$2 > f[$1] { f[$1]=$2;e[$1]=$3; } END { for (k in f) { print k f[k] e[k] }}' |
xargs cp -t <desination_directory>

The awk script puts each filename in an associate array entry, always keeping the highest rank found. The extension is stored in its own array. After all input is processed, all array entries are output, one per line. The xargs cp -t line copies all the files to the directory you specify.

There's another method which won't work very well if the numbers are greater than 9 and are not 0-padded. That method sorts the files lexicographically, then when parsing the list, the first part changes, the most recently seen filename is used. When the filenames are like this, it won't work:

file-9.txt
file-10.txt

because file-10.txt will appear before file-9. The awk script above does a numerical comparison.

CAVEAT: Filenames with tabs and newlines will cause this to choke.

CAVEAT 2: If multiple extensions per filename prefix are possible, we will have to make some tweaks to get it right.

Answer 6

This is not a command-line answer, per-se, but on the assumption that you have bash version >= 4 available, here's a bash script that gathers all of the *.txt files, determines their numbered suffix, then saves the highest-numbered suffix seen into an associative array (indexed by the base part of the filename before the numbered suffix). It the prints back out a sample cp commands for each filename with the highest suffix seen.

Adjust the destination "somedir" as needed.

#!/bin/bash

declare -A highest
for f in *.txt
do
  base=${f%%.txt}
  prefix=${base%-*}
  postfix=${base/$prefix-/}
  if [[ ${highest[$prefix]} -lt $postfix ]]
  then
    highest[$prefix]=$postfix
  fi
done

for prefix in "${!highest[@]}"
do
  escaped=${prefix//\"/\\\"}
  printf "cp -- \"%s\" somedir/\n" "$escaped-${highest[$prefix]}.txt"
done

In a directory of these files:

-dash-1.txt
-dash-2.txt
double"quote-3.txt
file'here-1.txt
filetwo'here'-2.txt
name_file-1.txt
name_file-2.txt
name_file-3.txt
somefile-5.txt
somefile-60.txt
some_other_file-1.txt
some_other_file-2.txt
space file-1.txt
space file-2.txt

The output is (manually sorted, for easier reading):

cp -- "-dash-2.txt" somedir/
cp -- "double\"quote-3.txt" somedir/
cp -- "file'here-1.txt" somedir/
cp -- "filetwo'here'-2.txt" somedir/
cp -- "name_file-3.txt" somedir/
cp -- "somefile-60.txt" somedir/
cp -- "some_other_file-2.txt" somedir/
cp -- "space file-2.txt" somedir/