I have a large directory containing subdirectories and files that I wish to copy recursively.

Is there any way to tell cp that it should perform the copy operation in order of file size, so that the smallest files get copied first?

  • 2
    Just to be sure there's not an XY problem involved, can you explain why you want to do this?
    – goldilocks
    Apr 8, 2014 at 17:32
  • 6
    @TAFKA'goldilocks' - I have a lot of video files, and I'd like to quality test each directory. The smallest video will give me a quick indication of if the rest of the files are bad as well. Apr 8, 2014 at 17:34

6 Answers 6


Here is a quick and dirty method using rsync. For this example I am considering anything under 10 MB to be "small".

First transfer just the small files:

rsync -a --max-size=10m srcdir dstdir

Then transfer the remaining files. The previously transferred small files will not be re-copied unless they were modified.

rsync -a srcdir dstdir

From man 1 rsync

          This  tells  rsync to avoid transferring any file that is larger
          than the specified SIZE. The SIZE value can be suffixed  with  a
          string  to  indicate  a size multiplier, and may be a fractional
          value (e.g. "--max-size=1.5m").

          This option is a transfer rule, not an exclude,  so  it  doesn’t
          affect  the  data  that  goes  into  the file-lists, and thus it
          doesn’t affect deletions.  It just limits  the  files  that  the
          receiver requests to be transferred.

          The  suffixes  are  as  follows:  "K"  (or  "KiB") is a kibibyte
          (1024), "M" (or "MiB") is a mebibyte (1024*1024),  and  "G"  (or
          "GiB")  is  a gibibyte (1024*1024*1024).  If you want the multi‐
          plier to be 1000 instead of  1024,  use  "KB",  "MB",  or  "GB".
          (Note: lower-case is also accepted for all values.)  Finally, if
          the suffix ends in either "+1" or "-1", the value will be offset
          by one byte in the indicated direction.

          Examples:    --max-size=1.5mb-1    is    1499999    bytes,   and
          --max-size=2g+1 is 2147483649 bytes.

Of course, the order of transfer file-by-file is not strictly smallest to largest, but I think it may be the simplest solution that meets the spirit of your requirements.

  • Here you get 2 copies of hard-links and soft-links are transformed into actual files for two copies of each. You'd do a lot better with --copy-dest=DIR and/or --compare-dest=DIR I think. I only know cause I had to add --hard-dereference myself to tar after posting my own answer because I was missing the links. I think rsync actually behaves more specific to local filesystems with those others anyway - I used to use it with USB keys and it would flood the bus unless I set a bandwidth limit. I think I should have used either of those others instead.
    – mikeserv
    Apr 8, 2014 at 20:48
  • 3
    +1 for the "quick and dirty method". Simpler is usually better at least for automation purposes and future maintainability. I think this is actually pretty clean. "Elegant" vs "kludgy" and "robust" vs "unstable" may sometimes conflict as design goals but there is a good balance that can be struck, and I think this is elegant and fairly robust.
    – Wildcard
    Dec 31, 2015 at 10:06

This does the whole job in one go - in all child directories, all in a single stream without any filename problems. It'll copy from smallest to largest every file you have. You will need to mkdir ${DESTINATION} if it doesn't already exist.

find . ! -type d -print0 |
du -b0 --files0-from=/dev/stdin |
sort -zk1,1n | 
sed -zn 's/^[^0-9]*[0-9]*[^.]*//p' |
tar --hard-dereference --null -T /dev/stdin -cf - |
    tar -C"${DESTINATION}" --same-order -xvf -

You know what, though? What this doesn't do is empty child directories. I could do some redirection over that pipeline, but it's just a race condition waiting to happen. Simplest is probably best. So just do this afterwards:

find . -type d -printf 'mkdir -p "'"${DESTINATION}"'/%p"\n' |
    . /dev/stdin

Or, since Gilles makes a very good point in his answer to preserve directory permissions, I should try also. I think this will do it:

find . -type d -printf '[ -d "'"${DESTINATION}"'/%p" ] || 
    cp "%p" -t "'"${DESTINATION}"'"\n' |
. /dev/stdin

I'd be willing to bet that's faster than mkdir anyway.

  • 1
    Damn you mikeserv! +1
    – goldilocks
    Apr 8, 2014 at 20:53
  • 3
    @TAFKA'goldilocks' I'll take that as a compliment. Thanks very much.
    – mikeserv
    Apr 8, 2014 at 20:58

Not cp directly, that's well beyond its abilities. But you can arrange to call cp on the files in the right order.

Zsh conveniently allows sorting files by size with a glob qualifier. Here's a zsh snippet which copies files in increasing order of size from under /path/to/source-directory to under /path/to/destination-directory.

cd /path/to/source-directory
for x in **/*(.oL); do
  mkdir -p /path/to/destination-directory/$x:h &&
    cp -- $x /path/to/destination-directory/$x:h

Instead of a loop, you can use the zcp function. However you need to create the destination directories first, which can be done in a cryptic oneliner.

autoload -U zmv; alias zcp='zmv -C'
cd /path/to/source-directory
mkdir -p **/*(/e\''REPLY=/path/to/destination-directory/$REPLY'\')
zcp -Q '**/*(.oL)' '/path/to/destination-directory/$f'

This doesn't preserve the ownership of the source directories. If you want that, you'll need to enlist a suitable copying program such as cpio or pax. If you do that, you don't need to call cp or zcp in addition.

cd /path/to/source-directory
print -rN -- **/*(^.) **/*(.oL) | cpio -0 -p /path/to/destination-directory

Here I'm using a --max-size option for rsync, but doing it in steps of 1 MB and then the other (larger) files, with this rsync proxy script:

for i in {1..1024}
        echo Max Size $i\M
        rsync --max-size=$i\M "$@"
rsync "$@"

I don't think there is any way to get cp -r to do this directly. Since it may be an indeterminate period of time before you get a wizardly find/awk solution, here's a quick perl script:

use strict;
use warnings FATAL => qw(all);

use File::Find;
use File::Basename;

die "No (valid) source directory path given.\n"
    if (!$ARGV[0] || !-d -r "/$ARGV[0]");

die "No (valid) destination directory path given.\n"
    if (!$ARGV[1] || !-d -w "/$ARGV[1]");

my $len = length($ARGV[0]);
my @files;
find (
    sub {
        my $fpath = $File::Find::name;
        return if !-r -f $fpath;
        push @files, [
            substr($fpath, $len),
    }, $ARGV[0]

foreach (sort { $a->[1] <=> $b->[1] } @files) {
    if ($ARGV[2]) {
        print "$_->[1] $ARGV[0]/$_->[0] -> $ARGV[1]/$_->[0]\n";
    } else {
        my $dest = "$ARGV[1]/$_->[0]";
        my $dir = dirname($dest);
        mkdir $dir if !-e $dir;
        `cp -a "$ARGV[0]/$_->[0]" $dest`;
  • Use this: ./whatever.pl /src/path /dest/path

  • The arguments should be both be absolute paths; ~, or anything else which the shell expands to an absolute path is fine.

  • If you add a third argument (anything, except for a literal 0), instead of copying it will print to standard out a report of what it would do, with files sizes in bytes prepended, e.g.

    4523 /src/path/file.x -> /dest/path/file.x
    12124 /src/path/file.z -> /dest/path/file.z

    Notice these are in ascending order by size.

  • The cp command on line 34 is a literal shell command, so you can do whatever you want with the switches (I just used -a to preserve all traits).

  • File::Find and File::Basename are both core modules, i.e. they are available in all installations of perl.

  • arguably, this is the only correct answer here. Or it was... the title - just changed...? My browser window is called cp - copy smallest files first? but the title of the post is just copy smallest files first? Anyway, options never hurt is my philosophy, but still, you and David are the only ones that used cp and you're the only one that pulled it off.
    – mikeserv
    Apr 8, 2014 at 21:19
  • @mikeserv The only reason I used cp was because it's the simplest way to preserve *nix file characteristics in the (cross-platform oriented) perl. The reason your browser bar says cp - is because of a (IMO goofy) S.E. feature whereby the most popular of the selected tags appears prefixed to the actual title.
    – goldilocks
    Apr 8, 2014 at 21:57
  • Ok, then I withdraw my compliment. Not really, you don't often see pearl coming out of the woodwork around here.
    – mikeserv
    Apr 8, 2014 at 21:59

another option would be to use cp with the output from du:

for i in $(du -sk *mpg | sort -n | cut -f 2)
    cp $i destination

This could still be done on one line, but I split it so you can read it

  • Don't you at least need to do something about $IFS?
    – mikeserv
    Apr 8, 2014 at 19:32
  • Yes...I keep assuming no one has newlines in their filenames Apr 8, 2014 at 19:33
  • 1
    This also doesn't seem to handle recursion through the directory hierarchy the OP described. Apr 8, 2014 at 19:39
  • 1
    @cpugeniusmv Correct...I somehow missed the recursive part....I could modify this to handle recursion, but I think at this point other answers do a better job. I'll leave this here in case it helps someone who sees the question. Apr 8, 2014 at 19:41
  • 1
    @DavidWilkins - this helps a lot. Apr 8, 2014 at 22:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .