2

I have a NFS mount that files (sometimes of some size) are being written to by a different system, and right now I'm polling for new files. I need to wait until the file is fully complete before processing it with my script. I don't control the process that's writing these files or their names.

It seems that fuser and lsof check my local system, but don't properly handle if a different system is writing to the NFS mount. I don't believe I can listen for file events if it's v3 (but I can for v4?), and short of waiting some amount of time for the file size to stop growing (and praying the network doesn't have hiccups), I don't know of a way to guarantee it's complete like you could locally by looking for file handles. Is there a solution for this? If not, is there a solution that would work with NFSv4?

6
  • 1
    Is there anything at the end of the file that indicates its completeness? An "END" marker of some sort? If not how would you differentiate from a completed file and a file that was written only partially by the sender and then treated by that sender as sent? The best approach would be for the sender to transfer as a temporary file and rename on completion (rename is an atomic operation even on NFS)
    – Chris Davies
    Mar 25, 2022 at 16:47
  • You ask a good question, I just came up with earlier trying to open the file (netcdf), but still need to do some research into what that looks like incomplete if it has an end token. 100% agree on the rename operation for other people's use case. I think both of those are potential answers
    – user519353
    Mar 25, 2022 at 17:13
  • I'll put you something together if you can identify and END token. Unless these two suggestions are sufficient and you can code it yourself
    – Chris Davies
    Mar 25, 2022 at 18:24
  • @roaima Ran some experiments and in my use case it looks like trying to open the file is sufficient to verify completeness, thanks for the tip. Definitely an answer there
    – user519353
    Mar 26, 2022 at 8:00
  • I don't know of a way to guarantee it's complete like you could locally by looking for file handles. A file no longer being open by a process writing it only tells you the file is no longer open - it does not tell you the file is complete. That's just hoping the file is complete.
    – Andrew Henle
    Mar 29, 2022 at 10:18

2 Answers 2

Reset to default
1

nfsv3 is stateless, there is no way to guarantee its state. But there is NLM, Network Lock Manager which has to run on both sides.

nfsv4 has locking operations open/read/write/lock/close and maintains OPEN CLOSE state. If you can you should change to nfsv4.

http://nfs.sourceforge.net/#faq_a6

1
+50

An ideal solution would be for the sender to place the file on the NFS share with a temporary suffix (.tmp), and rename it only when the copy is complete:

# Sender
# There are better ways of writing this code; it's just an illustration
#
if cp /from/source/data.xml /to/nfs/share/data.xml.tmp
then
    # copy succeeded; rename
    mv -f /to/nfs/share/data.xml.tmp /to/nfs/share/data.xml
fi

The rename is atomic even over NFS so provided the receiver ignores files with a .tmp suffix, any other file (data.xml in this example) will appear to the receiver that it has instantly and completely arrived on the NFS share.

However, you explained that unfortunately you had no control over the sender.

In this instance there isn't really very much you can do to guarantee the file has been completely transferred. Various options include looking for an END marker in the data (a marker that is specific to the data being transferred, not necessarily a literal "END" string), or trying to parse the file before processing it. Here's an example of checking an XML file:

# XML validation
if xmlstarlet validate /to/nfs/share/data.xml
then
    # An XML file validated so it must be complete
    ...
fi

Join this approach to one that only considers files unchanged for a few minutes and you might have a workable solution:

find /to/nfs/share -type f -mmin +5 -name '*.xml' -print0 |
    while IFS= read -d '' file
    do
        if validate-the-file "$file"
        then
            process-the-file "$file"
            rm -f "$file"
        fi
    done

Or

find /to/nfs/share -type f -mmin +5 -name '*.xml' -exec sh -c '
    for file in "$@"
    do
        process-the-file "$file"
        rm -f "$file"
    done
' _ {} +

You must log in to answer this question.