Can you get the remote machine to create a file just before it starts uploading the files, and then delete it afterwards?
e.g. using ssh (similar could be done with ftp or a HTTP PUT):
ssh yourhost touch ../Test_Data/upload-in-progress
scp local_enc* yourhost:../Test_Data/
ssh yourhost rm ../Test_Data/upload-in-progress
Then all your script has to do is wait for the upload-in-progress file to disappear. This could be done with a sleep loop, or perhaps using
inotifywait from the
NOTE: If the remote host dies or its script is killed before it completes the upload, it will leave a stale upload-in-progress file around. IMO this is a much smaller problem than the risk of race-conditions from trying to guess when the upload has finished (as all solutions running on the destination machine alone will be prone to)
I initially thought of using
lsof | grep local_enc | wc -l, but that's just as prone to races as your
find .. | wc -l.
As is using
inotify or similar to get notified of changes to the Test_Data directory - you can tell when files are created/altered in that directory but that doesn't tell you when an upload session has completed...however inotify in combination with a semaphore file would work. inotify to wait for the Test_Data dir to be changed, then inotify to sleep until upload-in-progress has been deleted.
Also, if the upload script on the remote host runs from cron the stale upload-in-progress file will fix itself on the next run. Alternatively, your script could be written to assume that any upload-in-progress file older than X minutes is stale and should be deleted/ignored (
inotifywait has a
--timeout option which would be useful here), but network delays or temporary outages could cause you problems here.