bash parallel encoding and uploading

Question

I am working on a bash script for encoding videos with ffmpeg in a special format. To save the resulting videos I want to upload the video as soon as the encoding is finished. Therefore I want to use parallel processing in my script.

A short example:

 #!/bin/bash

 uploading() {
 upload $1
 }

 encoding() {
 ffmpeg video video_encoded

 uploading $video_encoded
 }

If my encoding is faster then the uploading function it would create a new background process. This is not what I want. The wanted behavior is that if the uploading process is still alive my encoding function should save the required parameters (e.g. video path) in some kind of queue so that the upload function check on its end whether the queue is empty or not and push and pop as required.

Is something like this possible in bash?

Thanks in advanced!

Answer 1

Have two scripts. One which manages encoding of your videos and one for uploading. Use a spool directory where you place a file (symlink if just video itself necessary or use a text file if you need to store more information) when encoding is completed.

Uploading script should check every n seconds if there is new files to upload in spool directory. After successful upload the upload script removes the file from spool directory.

Spawn the uploading script from your main/encoding script. You can safely kill the unloader script after all files been uploaded successfully (spool directory is empty; non-empty directory means uploading hasn't finished/succeeded yet).