find 2 files with similar names and move them to a new location

Question

I have a system with a watch folder where people/companies can upload new files via FTP or SMB. In this watch folder They should ALWAYS upload 2 files: 1 media file with a prefix name of ABC*.mxf and always a number for '*'. The other is the same file name, but with the .xml extension.

example: uploaded files are: ABC0001.mxf, ABC0001.xml

if a second file ABC0002.xml is uploaded but the ABC0002.mxf is not yet uploaded, or not complete the ABC0002.xml file should NOT yet be moved. only when both ABC*.mxf and ABC*.xml have matchng names excist and are 5 min or older they should be moved.

I need to create a script that finds these 2 identical files (by name not extension) and only moves them if the modification time (mmin) is older than 5 minutes. So only completed files are moved.

Also I must say that multiple files can be uploaded at the same time by different suppliers. ABC0001.mxf + .xml by company 1, and ABC0100.mxf + .xml by company 2, and ABC1003.mxf + .xml by company 3. Not all finishing at the same times.

I already started with a partial script, but I'm struggling with the matching names part.

---

SOURCEDIR="/dir/to/source"
DESTDIR="/destination/dir"

for FOUND in `find $SOURCEDIR/AutoIngest -maxdepth 1 \
    -type f -name ^ABC* -mmin +5 `;     
do     
    mv "$FOUND" "$DESTDIR/"    
done

EDIT: changed the file names from ABC* to ABC*.mxf because the media file always have the .mxf extension. And added a file upload example.

Answer 1

The simplest approach will depend on how much you can trust your users. If you don't need to test that both files exist or that the names are correct or anything, you don't even need a script. You can do this with a simple find:

find /dir/to/source -name "ABC*" -mmin +5 -exec mv {} /destination/dir \;

If you need to make sure that i) both files are there and ii) both have a modification time of at least 5 minutes ago, on a GNU system, you could do this:

#!/usr/bin/env bash

SOURCEDIR="/dir/to/source"
DESTDIR="/destination/dir"

for f in "${SOURCEDIR}"/*.xml; do
    ## Make sure the file exists and is a regular file (or symlink to regular file),
    ## and that its modification date is at least 5 minutes ago
    [ -f "$f" ] && [ "$(( $(date +%s) - $(stat -c %Y "$f") ))" -ge 300 ] || continue

    ## Do the same for a file of the same name but with the .mxf extension.
    mxf="${SOURCEDIR}/$(basename "$f" .xml).mxf";
    [ -f "$mxf" ] && [ "$(( $(date +%s) - $(stat -c %Y "$no_ext") ))" -ge 300 ] || continue

    ## We will only get to this point if all of the above tests were successful
    echo mv -v "$f" "$mxf" "$DESTDIR"
done

Answer 2

On a GNU system:

SOURCEDIR="/dir/to/source"
DESTDIR="/destination/dir"
TIMESTAMP_MINDIFF=300

timestamp="$(date +%s)"
find "$SOURCEDIR/AutoIngest" -maxdepth 1 -type f -name 'ABC*' ! -name '*.xml' |
  while IFS= read -r file; do
    xmlfile="${file}.xml"
    test -f "$xmlfile" || continue
    ts_file="$(date --reference="$file" +%s)"
    ts_xmlfile="$(date --reference="$xmlfile" +%s)"
    if [ "$((timestamp-ts_file))" -gt "$TIMESTAMP_MINDIFF" ] &&
       [ "$((timestamp-ts_xmlfile))" -gt "$TIMESTAMP_MINDIFF" ]; then
      echo mv "$file" "$xmlfile" "$DESTDIR/"
    fi
  done

Remove the echo if the output is what you want.

Answer 3

With zsh:

cd /dir/to/source || exit
files=(ABC*(N.mm+5))
for f ($files[(k)*.xml]) {
   (($files[(I)$f:r])) && print -r mv -v -- $f $f:r /destination/dir/
}

(remove the print -r if happy).

Or to avoid calling mv several times:

cd /dir/to/source || exit
files=(ABC*(N.mm+5))
tomove=()
for f ($files[(k)*.xml]) {
   (($files[(I)$f:r])) && tomove+=($f $f:r)
}
print -r mv -- $tomove /destination/dir/