3

I have a system with a watch folder where people/companies can upload new files via FTP or SMB. In this watch folder They should ALWAYS upload 2 files: 1 media file with a prefix name of ABC*.mxf and always a number for '*'. The other is the same file name, but with the .xml extension.

example: uploaded files are: ABC0001.mxf, ABC0001.xml

if a second file ABC0002.xml is uploaded but the ABC0002.mxf is not yet uploaded, or not complete the ABC0002.xml file should NOT yet be moved. only when both ABC*.mxf and ABC*.xml have matchng names excist and are 5 min or older they should be moved.

I need to create a script that finds these 2 identical files (by name not extension) and only moves them if the modification time (mmin) is older than 5 minutes. So only completed files are moved.

Also I must say that multiple files can be uploaded at the same time by different suppliers. ABC0001.mxf + .xml by company 1, and ABC0100.mxf + .xml by company 2, and ABC1003.mxf + .xml by company 3. Not all finishing at the same times.

I already started with a partial script, but I'm struggling with the matching names part.


SOURCEDIR="/dir/to/source"
DESTDIR="/destination/dir"

for FOUND in `find $SOURCEDIR/AutoIngest -maxdepth 1 \
    -type f -name ^ABC* -mmin +5 `;     
do     
    mv "$FOUND" "$DESTDIR/"    
done

EDIT: changed the file names from ABC* to ABC*.mxf because the media file always have the .mxf extension. And added a file upload example.

  • Are you asking a question or requesting everybody to debug your scripting? – mdpc May 14 '14 at 8:19
  • Is this some type of homework assignment? – mdpc May 14 '14 at 8:20
2

The simplest approach will depend on how much you can trust your users. If you don't need to test that both files exist or that the names are correct or anything, you don't even need a script. You can do this with a simple find:

find /dir/to/source -name "ABC*" -mmin +5 -exec mv {} /destination/dir \;

If you need to make sure that i) both files are there and ii) both have a modification time of at least 5 minutes ago, on a GNU system, you could do this:

#!/usr/bin/env bash

SOURCEDIR="/dir/to/source"
DESTDIR="/destination/dir"

for f in "${SOURCEDIR}"/*.xml; do
    ## Make sure the file exists and is a regular file (or symlink to regular file),
    ## and that its modification date is at least 5 minutes ago
    [ -f "$f" ] && [ "$(( $(date +%s) - $(stat -c %Y "$f") ))" -ge 300 ] || continue

    ## Do the same for a file of the same name but with the .mxf extension.
    mxf="${SOURCEDIR}/$(basename "$f" .xml).mxf";
    [ -f "$mxf" ] && [ "$(( $(date +%s) - $(stat -c %Y "$no_ext") ))" -ge 300 ] || continue

    ## We will only get to this point if all of the above tests were successful
    echo mv -v "$f" "$mxf" "$DESTDIR"
done
  • As I edited in the initial post the second file does have an extention .mxf Where is the compair of basename.mxf and basename.xml performed? because there could be multiple uploads of different files at the same time, but only matching files that do not change anymore for 5 min should be moved. (Sorry for the initial BAD explenation in OriginalPost. didn't make it clear). – SHLelieveld May 15 '14 at 7:35
  • @user67588 OK, see updated post. There is no comparison of the mxf and xml files, I only check that both exist and that both are more than 5 minutes old. The tests are [-f file ] to check that the file exists, and the screwing around with date and stat to get their age. Each file (xml and msf) needs to pass both tests, else the loop will move on to the next file name (that's what the || continue does). – terdon May 15 '14 at 8:28
  • @user67588 you're welcome. If one of these answers solves your issue, please take a minute and check the check mark under the vote count to the left, this will signify to everyone that your issue's been resolved. – terdon May 15 '14 at 9:17
  • I will, this is a new thing for me so I need to get used to it. But Yes, this one did the job, thanks. – SHLelieveld May 15 '14 at 13:36
1

On a GNU system:

SOURCEDIR="/dir/to/source"
DESTDIR="/destination/dir"
TIMESTAMP_MINDIFF=300

timestamp="$(date +%s)"
find "$SOURCEDIR/AutoIngest" -maxdepth 1 -type f -name 'ABC*' ! -name '*.xml' |
  while IFS= read -r file; do
    xmlfile="${file}.xml"
    test -f "$xmlfile" || continue
    ts_file="$(date --reference="$file" +%s)"
    ts_xmlfile="$(date --reference="$xmlfile" +%s)"
    if [ "$((timestamp-ts_file))" -gt "$TIMESTAMP_MINDIFF" ] &&
       [ "$((timestamp-ts_xmlfile))" -gt "$TIMESTAMP_MINDIFF" ]; then
      echo mv "$file" "$xmlfile" "$DESTDIR/"
    fi
  done

Remove the echo if the output is what you want.

1

With zsh:

cd /dir/to/source || exit
files=(ABC*(N.mm+5))
for f ($files[(k)*.xml]) {
   (($files[(I)$f:r])) && print -r mv -v -- $f $f:r /destination/dir/
}

(remove the print -r if happy).

Or to avoid calling mv several times:

cd /dir/to/source || exit
files=(ABC*(N.mm+5))
tomove=()
for f ($files[(k)*.xml]) {
   (($files[(I)$f:r])) && tomove+=($f $f:r)
}
print -r mv -- $tomove /destination/dir/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.