0

after searching and trying various way I couldn't get it work.

I need to copy lots of files (more than 100 thousand files) to another folder with dot appended to the start of the file name.

Example:

  • /foo/bar/filename1.txt to /foo2/bar/.filename1.txt
  • and then rename it back to original name /foo2/bar/filename.txt

Why I need to do this is because I have an application that will keep scanning /foo2/bar folder and ignore those file with a dot in front of the file name so that it will not process those file that are copying half way. This mainly because the 2 folder can be in 2 different network drive or some mounted devices.

And i cannot simply use mv or cp because I have some folder that has too many files and it will simply throw argument list too long error thus I have been trying to use find command but to no avail.

Trying out with different command:

find /foo/bar/ -type f -exec cp -t /foo2/bar {} +

and

find /foo/bar/ -type f -exec mv {} /foo2/bar/.{} \;

I know the above command won't do what I wanted that is along the line of what i've tired.

Appreciate anyone that can help...

1
  • why not use mkdir /foo2; cp -a /foo/bar /foo2? The command cp -a copies all files recursively.
    – Pak Uula
    Commented Jan 16, 2020 at 12:36

1 Answer 1

1
rsync -a -T tmpdir /foo/bar/ /foo2/bar

(Note that the / at the end of /foo/bar/ is significant.)

The rsync utility will write files in tmpdir before moving these to the correct name under /foo2/bar. You could, for example, use a hidden directory under /foo2/bar to store the in-transit files:

mkdir -p /foo2/bar/.tmp
rsync -a -T /foo2/bar/.tmp /foo/bar/ /foo2/bar

See also the rsync manual, especially the section about the -T option.

The above command will copy /foo/bar recursively to /foo2/bar. To disable the recursiveness, use rsync -a --no-recursive ... (the -a option enables recursion as well as syncing of file metadata, and --no-recursion turns off only the recursion aspect of -a).

1
  • I forgot to mention that my application will scan /foo2/bar and its sub folder thus I think if those transferring file are place temporary in .tmp folder it will get process... thus it will not work using this rsync
    – JL007
    Commented Jan 17, 2020 at 1:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .