4

I've got a bunch of files all named like this:

name_file-1.txt
name_file-2.txt
name_file-3.txt
some_other_file-1.txt
some_other_file-2.txt

There are thousands of different filenames, some with just one -1.txt at the end, some with -1.txt, -2.txt ... -60.txt

I need to copy the highest numbers of each file, so name_file-3.txt, some_other_file-2.txt. How do I do that on a Linux command line?

  • In the source dir cp -t /path_to_destination $(ls -v *[0-9].txt | sed 'N;/\([^\n]*-\)[0-9]\+\.txt\n\1/!P;D') – Costas Feb 12 '16 at 11:22
  • @Costas: Why don't you make this an answer? – user unknown Feb 12 '16 at 11:24
  • If any of the existing answers solves your problem, please consider accepting it via the checkmark. Thank you! – Jeff Schaller Apr 23 '17 at 13:05
4

With zsh:

typeset -A greatest
for f (*-*(n)) greatest[${f%-*}]=$f
cp -- $greatest /destination
  • *-*(n): non-hidden files whose name contains a - (*-*), sorted numerically ((n) glob qualifier).
  • ${f%-*}: part of the filename up to the right-most - (or to the end if there's no -).
  • $greatest: expands to the non-empty values of the associative arrays. So here, for files that share the same root, only the file with the greatest number will be expanded.
1
files=(*)
mapfile -t prefixes < <(printf "%s\n" "${files[@]%-*}" | sort -u)
for p in "${prefixes[@]}"; do ls -v "$p"* | tail -1; done
name_file-3.txt
some_other_file-2.txt

And then to copy those to some other directory:

for ...; done | xargs cp -t /destination/directory
  • Cool! Never seen mapfile before! – Otheus Feb 12 '16 at 11:47
  • Great! But the copy part does not seem to work...? – MIB Feb 12 '16 at 12:35
  • When copy I get this: "xargs: unmatched single quote; by default quotes are special to xargs unless you use the -0 option" and also sometimes "cp: warning: source file 'fgfggfffg-2.txt' specified more than once – MIB Feb 12 '16 at 12:38
  • Where are those quotes coming from? Do you have a filename with a quote in it? – glenn jackman Feb 12 '16 at 16:42
1

If the files are in the current working directory and their names conform to the samples (a single dash which precedes a number), the following POSIX-compliant pipeline should work:

ls | sort -t- -k1,1 -k2,2rn | awk -F- 'k!=$1 {print; k=$1}' | pax -rw /path/to/dir

The awk component can be replaced by a sort -u, if the sort's -u option is stable (so that the first line of a set is always chosen to represent that set). POSIX does not require this stability, but, according to their manuals, the {Free,Net,Open}BSD and GNU implementations provide it. If you enjoy tempting fate:

ls | sort -t- -k1,1 -k2,2rn | sort -mut- -k1,1 | pax -rw /path/to/dir

In either case, the target directory must not be in the current working directory.

1

If pwd is the source dir

cp -t /path_to_destination $(
      ls -v *[0-9].txt |
      sed '$b;N;/^\(.*\)[0-9]\+\.txt\n\1[0-9]\+\.txt/!P;D')

NOTE: If there are any spaces in file names it should be prepared before by escaping to proper operation. + see other restrictions in Stéphane Chazelas' comments

  • Might break if OP has files such as file-2.txt and file-other-1.txt – Otheus Feb 12 '16 at 11:44
  • @Otheus You are right. Have edited. – Costas Feb 12 '16 at 11:52
  • 1
    @Otheus The ! negates match == if not – Costas Feb 12 '16 at 11:59
  • Ahh, so that's what sed's D is good for. – Otheus Feb 12 '16 at 12:03
  • Remember the problem with unquoted expansions is not only above spaces, it's about any $IFS character and glob wildcards. – Stéphane Chazelas Feb 12 '16 at 12:06
0

I'd split the file into tab-delimited parts for more reliable, customizable filename parsing, then use awk to find the highest rank of each and report. Try each part of the pipeline first before proceeding to next!

find DIR -type f <other find criteron> -print | 
perl -lne 'print join("\t",(/^(.*?-)(\d+)(\.\w+)$/))' |
awk -F\\t '$2 > f[$1] { f[$1]=$2;e[$1]=$3; } END { for (k in f) { print k f[k] e[k] }}' |
xargs cp -t <desination_directory>

The awk script puts each filename in an associate array entry, always keeping the highest rank found. The extension is stored in its own array. After all input is processed, all array entries are output, one per line. The xargs cp -t line copies all the files to the directory you specify.

There's another method which won't work very well if the numbers are greater than 9 and are not 0-padded. That method sorts the files lexicographically, then when parsing the list, the first part changes, the most recently seen filename is used. When the filenames are like this, it won't work:

file-9.txt
file-10.txt

because file-10.txt will appear before file-9. The awk script above does a numerical comparison.

CAVEAT: Filenames with tabs and newlines will cause this to choke.

CAVEAT 2: If multiple extensions per filename prefix are possible, we will have to make some tweaks to get it right.

0

This is not a command-line answer, per-se, but on the assumption that you have bash version >= 4 available, here's a bash script that gathers all of the *.txt files, determines their numbered suffix, then saves the highest-numbered suffix seen into an associative array (indexed by the base part of the filename before the numbered suffix). It the prints back out a sample cp commands for each filename with the highest suffix seen.

Adjust the destination "somedir" as needed.

#!/bin/bash

declare -A highest
for f in *.txt
do
  base=${f%%.txt}
  prefix=${base%-*}
  postfix=${base/$prefix-/}
  if [[ ${highest[$prefix]} -lt $postfix ]]
  then
    highest[$prefix]=$postfix
  fi
done

for prefix in "${!highest[@]}"
do
  escaped=${prefix//\"/\\\"}
  printf "cp -- \"%s\" somedir/\n" "$escaped-${highest[$prefix]}.txt"
done

In a directory of these files:

-dash-1.txt
-dash-2.txt
double"quote-3.txt
file'here-1.txt
filetwo'here'-2.txt
name_file-1.txt
name_file-2.txt
name_file-3.txt
somefile-5.txt
somefile-60.txt
some_other_file-1.txt
some_other_file-2.txt
space file-1.txt
space file-2.txt

The output is (manually sorted, for easier reading):

cp -- "-dash-2.txt" somedir/
cp -- "double\"quote-3.txt" somedir/
cp -- "file'here-1.txt" somedir/
cp -- "filetwo'here'-2.txt" somedir/
cp -- "name_file-3.txt" somedir/
cp -- "somefile-60.txt" somedir/
cp -- "some_other_file-2.txt" somedir/
cp -- "space file-2.txt" somedir/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.