1

I'm trying to recursively preform a series of operations on many directories containing files. In each directory I want to create 2 directories and move some of the files into each.

For instance, my directory structure looks like this ...

   root --
     subdirectory1 --
         file_a1_a.tif
         file_a2_a.tif
         file_a1_b.tif
         file_a2_b.tif
     subdirectory2 --
         file_b1_a.tif
         file_b2_a.tif
         file_b1_b.tif
         file_b2_b.tif

I'd like to reorganize the files to be like this ...

   root --
     subdirectory1 --
         subdirectory_a --
             file_a1_a.tif
             file_a2_a.tif
         subdirectory_b --
             file_a1_b.tif
             file_a2_b.tif
     subdirectory2 --
         subdirectory_a --
             file_b1_a.tif
             file_b2_a.tif
         subdirectory_b --
             file_b1_b.tif
             file_b2_b.tif

I'm doing the following but it only seems to work in the top level directory.

find . -type d -execdir bash -c "mkdir -p subdirectory_a && mkdir -p subdirectory_b && mv *_a.tif subdirectory_a 2>/dev/null && mv *_b.tif subdirectory_b 2>/dev/null" \;

Any help would be greatly apprecited

4 Answers 4

2

Since root has only children and not grandchildren, you can run this from root directory:

subA='subdirectory_a'
subB='subdirectory_b'
for d in */; do
    mkdir -p "$d$subA" "$d$subB"
    mv "$d"*_a.tif "$d$subA"
    mv "$d"*_b.tif "$d$subB"
done

$d expands to subdirectory1/ and subdirectory2/ because of the glob on the for loop, */, which matches only immediate subdirectories. The corresponding root grandchildren are created with mkdir (the -p is there just so that it does not complain if they already exist), and then globbing is used again to move the files to grandchildren.

Notice that globs should be unquoted if the shell is to expand them. That is why mv "$d"*_a.tif "$d$subA" is correct but mv "$d*_a.tif" "$d$subA" would be incorrect.

5
  • Very nice ! For a small amount of files in each subdirectory $d this will work beautifully. +1'ed Jun 6, 2020 at 2:35
  • I see how this works and it answers my question as asked. Will this no work with grandchildren of root? My real application does have grandchildren and great-grandchildren in some cases.
    – agf1997
    Jun 6, 2020 at 4:57
  • Many thanks @SergiyKolodyazhnyy for your words. agf1997, it does not work with grandchilden. I'm sorry, but your sample did not provide one :/
    – Quasímodo
    Jun 6, 2020 at 11:39
  • It does work with "grandchildren". As in, the grandchildren would be ignored (unless they happened to be pre-existing directories subdirectory_a and subdirectory_b). It is unclear what @agf1997 means by "work".
    – Kusalananda
    Jun 8, 2020 at 6:21
  • @Kusalananda Indeed! Except if there is a directory grandchild named with something like file_a3_b.tif, in which case it matches the glob pattern and will also be moved.
    – Quasímodo
    Jun 8, 2020 at 10:59
2

The command you've posted in the question

find . -type d -execdir bash -c "mkdir -p subdirectory_a && mkdir -p subdirectory_b && mv *_a.tif subdirectory_a 2>/dev/null && mv *_b.tif subdirectory_b 2>/dev/null" \;

behaves correctly ( that is , it does what it is supposed to do ). But what it spells out is not what you want.

The -execdir argument operates such that "...the specified command is run from the
subdirectory containing the matched file," (source, find(1) )

That is to say, when your current working directly is root and the command locates subdirectory1 it will not descend there as you expect. The subdirectory1 which matches -type d was found when find was inside root, hence it will stay there and execute the mkdir command specified from there. Naturally, there is no files inside root, hence nothing will be moved around, but also note that mkdir -p won't produce an error, as specified by the mkdir(1) : "no error if existing".


So how do we make this match what you want? Let's assume the same structure you've posted

$ tree /tmp/root
/tmp/root
├── subdir1
│   ├── file_a1_a.tif
│   ├── file_a1_b.tif
│   ├── file_a2_a.tif
│   └── file_a2_b.tif
└── subdir2
    ├── file_b1_a.tif
    ├── file_b1_b.tif
    ├── file_b2_a.tif
    └── file_b2_b.tif

I'd suggest making it a three-step process. There is no need to make a one single command do all the work. Focus on the desired result rather than "code golf". So make the directories first

$ find /tmp/root -maxdepth 1 -mindepth 1 -exec bash -c 'mkdir ${1}/subdirectory_{a,b}' sh  {} \;
$ tree
.
├── subdir1
│   ├── file_a1_a.tif
│   ├── file_a1_b.tif
│   ├── file_a2_a.tif
│   ├── file_a2_b.tif
│   ├── subdirectory_a
│   └── subdirectory_b
└── subdir2
    ├── file_b1_a.tif
    ├── file_b1_b.tif
    ├── file_b2_a.tif
    ├── file_b2_b.tif
    ├── subdirectory_a
    └── subdirectory_b

6 directories, 8 files

And then find regular files at specific level in directory tree and move them. Now we can use -execdir to achieve exactly what we want

$ find /tmp/root -maxdepth 2 -mindepth 2 -type f -execdir bash -c 'case ${1} in *a.tif) echo cp ${1} ./subdirectory_a ;; *b.tif) echo cp ${1} subdirectory_b ;; esac' sh {} \;
cp ./file_b2_b.tif subdirectory_b
cp ./file_b2_a.tif ./subdirectory_a
cp ./file_b1_a.tif ./subdirectory_a
cp ./file_b1_b.tif subdirectory_b
cp ./file_a1_b.tif subdirectory_b
cp ./file_a1_a.tif ./subdirectory_a
cp ./file_a2_b.tif subdirectory_b
cp ./file_a2_a.tif ./subdirectory_a

Note that in this case, I use echo before moving to ensure everything behaves properly , and no actual moving is done. So if we remove echo, this will behave correctly:

$ find /tmp/root -maxdepth 2 -mindepth 2 -type f -execdir bash -c 'case ${1} in *a.tif) cp ${1} ./subdirectory_a ;; *b.tif) cp ${1} subdirectory_b ;; esac' sh {} \;
$ tree /tmp/root
/tmp/root
├── subdir1
│   ├── file_a1_a.tif
│   ├── file_a1_b.tif
│   ├── file_a2_a.tif
│   ├── file_a2_b.tif
│   ├── subdirectory_a
│   │   ├── file_a1_a.tif
│   │   └── file_a2_a.tif
│   └── subdirectory_b
│       ├── file_a1_b.tif
│       └── file_a2_b.tif
└── subdir2
    ├── file_b1_a.tif
    ├── file_b1_b.tif
    ├── file_b2_a.tif
    ├── file_b2_b.tif
    ├── subdirectory_a
    │   ├── file_b1_a.tif
    │   └── file_b2_a.tif
    └── subdirectory_b
        ├── file_b1_b.tif
        └── file_b2_b.tif

6 directories, 16 files

Note of course that this uses copying to avoid data loss. We can further clean up and in the end - we get the desired directory tree structure.

$ find /tmp/root -maxdepth 2 -mindepth 2 -type f -delete
$ tree 
.
├── subdir1
│   ├── subdirectory_a
│   │   ├── file_a1_a.tif
│   │   └── file_a2_a.tif
│   └── subdirectory_b
│       ├── file_a1_b.tif
│       └── file_a2_b.tif
└── subdir2
    ├── subdirectory_a
    │   ├── file_b1_a.tif
    │   └── file_b2_a.tif
    └── subdirectory_b
        ├── file_b1_b.tif
        └── file_b2_b.tif

6 directories, 8 files

In conclusion what we've done can be expressed as one single script

find /tmp/root -maxdepth 1 -mindepth 1 -exec bash -c 'mkdir ${1}/subdirectory_{a,b}' sh  {} \; &&
find /tmp/root -maxdepth 2 -mindepth 2 -type f -execdir bash -c 'case ${1} in *a.tif) cp ${1} ./subdirectory_a ;; *b.tif) cp ${1} subdirectory_b ;; esac' sh {} \; &&
find /tmp/root -maxdepth 2 -mindepth 2 -type f -delete

Of course, steps 2 and 3 can be condensed into one if we use mv instead of cp, but I would rather err on the side of caution than to risk data loss.

1
  • This is a wonderful answer, Sergiy. You go step by step so that anyone here can follow the chain of thought, especially useful to newcomers. Thanks for that, +1 too.
    – Quasímodo
    Jun 6, 2020 at 11:36
1

Your command, when run inside of the root directory, is going to create subdirectoy_a and subdirectory_b at the top level alongside subdirectory1 and subdirectory2 and then move all of the files ending in a.tiff into subdirectory1 and all of the files ending in b.tiff into subdirectory_b which isn't what you want. This would be easier for you to troubleshoot if you weren't redirecting the errors to /dev/null and could actually see them. You also don't need the -p option with mkdir because the parent directories already exist.

Instead of what you have, run this from inside of root (or whatever the directory is actually called).

mkdir subdirectory1/subdirectory_a subdirectory1/subdirectory_b subdirectory2/subdirectory_a subdirectory2/subdirectory_b && mv subdirectory1/file_a*a*  subdirectory1/subdirectory_a && mv subdirectory1/file_a*b*  subdirectory1/subdirectory_b && mv subdirectory2/file_b*a*  subdirectory2/subdirectory_a && mv subdirectory2/file_b*b*  subdirectory2/subdirectory_b
0

With zsh:

cd root || exit
subdirs=(*(-/))
mkdir -p -- $^subdirs/subdirectory_{a,b} || exit
autoload -Uz zmv
zmv '(*/)*_(a|b).tif' '${1}subdirectory_$2/$f:t'

If it needs to be recursive as your question says (but your sample doesn't suggest), that is if you also need that to be done in subdir1/subsubdir1 and so on, replace the *s in *(-/) and (*/) above with **.

If there are zillions of files and speed is a concern, run zmodload zsh/files for mkdir and mv (called by zmv) to become zsh builtins.

For a dry-run where zmv only shows you what it would do, add a -n option.

If you only want the subdirectory_a and subdirectory_b to be created in directories that do contain some *_a.tif and *_b.tif respectively, you could take this other approach:

mkmv() { mkdir -p -- $2:h && mv -- "$@"; }
zmv -P mkmv '(**/)*_(a|b).tif' '${1}subdirectory_$2/$f:t'

(that one would also move tif files located in the root itself).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.