10

I have millions of files with the following nomenclature on a Linux machine:

1559704165_a1ac6f55fef555ee.jpg

The first 10 digits are timestamp and the ones followed by a _ are specific ids. I want to move all the files matching specific filename ids to a different folder.

I tried this on the directory with files

find . -maxdepth 1 -type f | ??????????_a1ac*.jpg |xargs mv -t "/home/ubuntu/ntest"

However I am getting an error indicating:

bash 1559704165_a1ac6f55fef555ee.jpg: command not found

When I tried, mv ??????????_a1ac*.jpg I am getting argument list too long error. I have atleast 15 different filename patterns. How do I move them.

  • 1
    The bash says it all: it tries to execute that filename as it is the first on the line in the 2nd stage of the pipe (your 2nd stage pipe is: | ??????????_a1ac*.jpg: bash expands it to several filename, the first being 1559704165_a1ac6f55fef555ee.jpg, si you end up, in that 2nd pipe stage, trying to execute: 1559704165_a1ac6f55fef555ee.jpg next_matching_filename 3rd_matching_filename ... nth_matching_filename. I guess you tried instead to filter to that filename (see answers below for that) – Olivier Dulac Jul 8 '19 at 13:08
15

You should use:

find . -maxdepth 1 -type f -name '??????????_a1ac*.jpg' \
-exec mv -t destination "{}" +

So maxdepth 1 means that you want to search in current directory no subdirectories.

type f means find only files.

name '??????????_a1ac*.jpg' is a pattern that matches with file you are searching.

mv -t destination "{}" + means move matched files to destination. Here + adds new matched files to previous one like:

mv -t dest a b c d

Here a b c d are different files.

| improve this answer | |
  • Thanks for concisely answering this persons question. Rather than simply dumping a solution, maybe you could explain how/what/why. Instead of being useful to one person, one time, it can be useful to everyone, all the time. Same question has been asked & answered countless times over the last 40-50 years. Problem is, it's never explained well. Teach a man to fish.. In the meantime: gnu.org/software/findutils/manual/html_node/find_html/… and as is often the case Wikipedia's more useful than the official docs: en.wikipedia.org/wiki/Find_(Unix) – voices Jul 8 '19 at 17:22
  • See updated answer. – Prvt_Yadav Jul 8 '19 at 17:48
  • Note that -t is a GNU extension and so may not be available on other types of UNIX derivatives. – Kevin Jul 8 '19 at 18:22
  • When you say "Double quotes prevents word splitting." I presume you are referring to "{}", in which case I want to point out that {} is not expanded by the shell and does not need to be quoted. The shell passes {} to find, and find sees {} and replaces it with pathnames. Find exec does not use the shell parser and does not do any word-splitting of its own. Quoting it does not do any harm, it is just that the justification given is a bit inaccurate. – jw013 Jul 9 '19 at 18:59
  • @jw013 thanks . – Prvt_Yadav Jul 10 '19 at 6:49
11

Your command,

find . -maxdepth 1 -type f | ??????????_a1ac*.jpg |xargs mv -t "/home/ubuntu/ntest"

Pipes the list of all the files TO all the files!

find . -maxdepth 1 -type f -name `*_a1ac*.jpg` -print0 |\
xargs  -0 -r mv -t "/home/ubuntu/ntest"

will do the trick.

| improve this answer | |
  • 1
    many thanks...your solution worked too...thank you for letting me know where I went wrong – Apricot Jul 6 '19 at 14:13
8

You're very close. You should use the -name option to find. And remember to quote the pattern.

So

find . -maxdepth 1 -type f -name '??????????_a1ac*.jpg' |xargs mv -t "/home/ubuntu/ntest"
| improve this answer | |
  • Many thanks...your solution worked too....additional thanks for letting me know i was close to the solution....its a motivator for a novice like me – Apricot Jul 6 '19 at 14:12
  • 1
    you should add a -print0 as the last argument to the find (instead of the default: -print), and add a -0 as the first option to xargs (ie: xargs -0 mv -t "/home/ubuntu/ntest" ). that way, all kind of weird filenames (with spaces in it, with "newline" in it, etc) can be handled. find . -maxdepth 1 -type f -name '??????????_a1ac*.jpg' -print0 |xargs -0 mv -t "/home/ubuntu/ntest" (works only with GNU-like find, though) – Olivier Dulac Jul 8 '19 at 13:10
2

Not as "good" as the find solutions, but another valid solution is to make the mv commands more granular.

This does 4096 moves, with a fewer number of files moved per mv operation.

FILEPAT=a1ac
for i in $(seq $((0x000)) $((0xfff))); 
do 
   H=$(printf '%x\n' $i)
   mv 1559704165_${FILEPAT}${H}*.jpg /home/ubuntu/ntest
done
| improve this answer | |
  • This is a clever hack for those without find (for whatever reason). – forest Jul 8 '19 at 7:44
-1

If you want to move files on the same host system, which I guess you are doing with your mv, rsync could be a faster option:

rsync -av --inplace -W /source/??????????_a1ac*.jpg /home/ubuntu/ntest/

--inplaceand -W are set to speed up the process.

Should this yield another argument list too long error then you could feed lists to rsync

Make the list with find, for example

find . -maxdepth 1 -type f -name '??????????_a1ac*.jpg' > /tmp/my_image_list.txt

and give it to rsync

rsync -av --inplace -W --files-from=/tmp/my_image_list.txt /path/to/files /home/ubuntu/ntest/

The source here is /path/to/files, because rsync will treat the list you give it as relative to your source.


The point being: rsync is faster than mv, if the files are not on the same filesystem.

| improve this answer | |
  • This is likely to hit the same "argument list too long" error the OP mentioned – Grump Jul 8 '19 at 13:24
  • @Grump, to avoid this, OP could write the list of files to copy to a file, i.e. find . -maxdepth 1 -type f -name '??????????_a1ac*.jpg' > /tmp/my_image_list.txt and then pass it to rsync with --files-from=/tmp/my_image_list.txt. The point being that rsync is faster. Unless the files reside on the same filesystem, which OP has not indicated. – Robert Riedl Jul 8 '19 at 13:46
  • @RobertRiedl: you should edit your answer and add this information. Comments can be impermanent. – NickD Jul 10 '19 at 4:13
  • @NickD, I've updated my answer. – Robert Riedl Jul 10 '19 at 6:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.