0

first of all I have very limited knowledge in Linux but I am always trying my best. I also have already automated a lot of things in my Unraid Server and RPi by using Cronjobs and scripts, but now I am a little bit lost.

Following situation: I am using icloud photo downloader to automatically download all photos and videos from my icloud storage to my local NAS. It uses the subfolder structure /yyyy/mm/ and downloads .jpg and .mov, e.g. a video taken today will be downloaded this night to /2020/12/IMG_0123.mov

In addition, I am using 'handbrake' to then convert the .mov to .mkv as mkv can be played with all my devices and .mov always wants to be converted e.g. in Plex. For Handbrake I managed to have a watch folder (/videowatch/) and an output folder (/videooutput/) which handbrake uses to automatically converts video files which I copy in the watch folder and exports it to output folder by recreating the same folder structure. Example: I copy the video file mentioned above to subfolder /videowatch/2020/12/IMG_0123.mov, handbrake will transcode it to mkv and export the file to /videooutput/2020/12/IMG_0123.mkv. The .mov then will be deleted.

This step is what I am currently doing manually and want to have this automated.

How can I manage this script? The following steps should be performed:

  1. search for files of type .mov in all subfolders
  2. move all found .mov files to folder /videowatch/ but recreates the subfolder structure

Example: icloudpd downloads two new .mov (file1.mov and file2.mov) in folder /2020/11/ as well as /2020/12/. These two folders already store a lot of other .jpg as well as .mkv files

  1. The script recognizes two .mov files /2020/11/file1.mov and /2020/12/file2.mov
  2. The script moves these two files to folder /videowatch/ and recreates the subfolder structure. We then will have /videowatch/2020/11/file1.mov and /videowatch/2020/12/file2.mov

Is that possible? As a final step, the content of videooutput folder should be moved back to / to move the new mkv files to the destination folder of the former .mov files. As handbrake keeps the subfolder structure, this is easy to handle by a mv entry in crontab.

Hopefully I did not describe it to complicated.

Thank you so much for your help!

I am only struggling with the script to move the .mov files to the watch folder including folder structure.

e.g. I could use find to search for the files:

find . -name "*.MOV" -type f

and this brings:

./2008/08/CIMG0035.MOV 
./2010/11/IMG_0212.MOV

These two files should then be moved to the watch folder and at the end be like:

./videowatch/2008/08/CIMG0035.MOV
./videowatch/2010/11/IMG_0212.MOV

I am able to handle all the other stuff (handbrake, icloudpd, cronjob etc.) by my own.

I could use

find . -name "*.MOV" -type f -exec mv '{}' /watchfolder/ \;

This then moves all .MOV identified by the find command in the watchfolder main but without any hierarchy and subfolders.So I don't have all the files in the respective folder by year / month.

./videowatch/CIMG0035.MOV 
./videowatch/IMG_0212.MOV

But this is not what I want, I want / need it including the subfolder for year / month as the files are located in the source folder.

Thanks!

Improve this question
2
  • 1
    It is possible, but that's a lot of stuff to help you with. What part of the process are you specifically having trouble with?
    – Eduardo Trápani
    Dec 11, 2020 at 16:47
  • @EduardoTrápani thanks for your question. Please refer to my answer below
    – Daniel
    Dec 11, 2020 at 17:39

1 Answer 1

Reset to default
0

You can invoke a shell script and use one of the solutions to this question: Is there a way to make mv create the directory to be moved to if it doesn't exist?

Also you could use rsync with --include/--exclude to sync just what you want and then remove manually.

Or use find to just create the list of files, that you will then read from the script that does all the other tasks (and now will also do the moving). In that case doing a mkdir -p before the move would solve the problem.

Please note that files you are moving might or not be completely downloaded. You need to have a way to know if they are fully transferred from the cloud at the time you run your script. The way to do it depends on the tool doing the download, it might be the presence of a hidden file, or you might be able to find that out by doing an fuser on the file about to be moved. Otherwise, you might risk losing data.

Improve this answer
8
  • Thank you! By using mkdir -p I get the following: find . -name "*.MOV" -type f -exec mkdir -p '{}' /mnt/user/iCloud/watchfoler/ \;mkdir: cannot create directory ‘./2012/12/IMG_1370.MOV’: File exists mkdir: cannot create directory ‘./2012/12/IMG_1371.MOV’: File exists as I mentioned above, with the find command I was able to determine the files to be moved and managed to move the files, but without the directories. Guess this is way too complicated for my very very limited knowledge in commands / scripts. I even don't know how to use the mkdir -p in my command.
    – Daniel
    Dec 11, 2020 at 19:15
  • Just do -exec yourscript {} and put all the logic in yourscript.sh. While you could eventually pack everything into find, or call xargs, calling your own script will be more readable and you will know exactly what you are doing at every step of the way (that means you can also add error control and suff like that).
    – Eduardo Trápani
    Dec 11, 2020 at 23:01
  • Thanks but I think I will give up here and keep going the manual way. I can handle very basic commands but really have no clue about writing scripts and where should I do the -exec yourscript command? And how could I script that the folders according to the files source folders will be created? I am already struggling with the code formatting in this platform ;-) guess this platform is only for people with very profound knowledge who want to have the very last confirmation, but not for people like me beginning from the bottom. But anyway, thank you for your help!
    – Daniel
    Dec 12, 2020 at 6:50
  • Just a hint, if your put the commands one after the other inside a text file, you pretty much have the basic script. Of course it takes more than that to have a well written one, but it's not that hard to add a command line parameter for example. Don't go for the whole thing, you can automatize bits of your process in different scripts, mixing them with manual interventions and then, maybe, someday put them all together. Check something like Bash Scripting Tutorial for Beginners to get you started. Good luck!
    – Eduardo Trápani
    Dec 12, 2020 at 18:01
  • Thank you Eduardo. But I still have no clue about how to create the folder hierarchy according to the files‘ source destination.
    – Daniel
    Dec 13, 2020 at 5:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .