3

I have cluster using NFS with a program myprogram which writes to a fixed location, e.g. /home/sharedfs/somedir/somefile.txt. This means that if you run myprogram from each node, they all write to the shared /home/sharedfs/somedir/somefile.txt, but this is not desirable.

So, I was thinking: is there a way to create a file for each node, and make a soft link in NFS such that the link on each node points to a different node-specific file? That is, having

/home/sharedfs/somedir/somefile.node0.txt
/home/sharedfs/somedir/somefile.node1.txt
/home/sharedfs/somedir/somefile.nodeN.txt

and

/home/sharedfs/somedir/somefile.txt -> link to somefile.node0.txt only on node0
/home/sharedfs/somedir/somefile.txt -> link to somefile.node1.txt only on node1
/home/sharedfs/somedir/somefile.txt -> link to somefile.nodeN.txt only on nodeN

Is this possible?

Another option is to have different directories somedir which are mounted differently for each node, so that somedir/somefile.txt points to a different (non shared) directory on each node.

Any idea is welcome!

1
  • 1
    I'm pretty sure this isn't possible w/ NFS.
    – slm
    Jun 6 '14 at 12:26
3

I suppose that when you say that myprogram writes in a fixed file, your mean that you cannot modify this program to make it write to another file (maybe you don't have the source code).

I'm also pretty sure that what you want to do is not possible with NFS. Anyway the solution you mentioned using different mount points will work, but isn't very practical if you have a lot of nodes.

Does your program writes the file somefile.txt using an absolute path, or just the file in the current directory? If it is in the current directory you can just make some sub-directories like:

/home/sharedfs/somedir/nodeN/somefile.txt

and run myprogram in the corresponding directory:

cd /home/sharedfs/somedir/nodeN
./myprogram

Then, on each node (nodeN) use a symbolic link in a local directory like something like this:

ln -s /home/sharedfs/somedir/nodeN/somefile.txt /usr/local/somefile.txt

The file somefile.txt can be accessed on each node with the same name, but points to a different file according to the node. If you want to access to file of another node, this can be done through the mounted file system.

5
  • Your assumption is correct: myprogram writes to fixed file, in a relative position, but unluckily the program depends on a weird directory structure which determines where the file will be written. Your solution is fine, it will just need a bit of replication of the data in the nodes' /usr/local/ that you provided as example.
    – AkiRoss
    Jun 6 '14 at 14:46
  • 1
    Maybe you can use the chroot command to change the root directory of the program and make it write to the write place on each node. Another "ugly" solution consist in editing the program binary. You have to search for the string that correspond to the relative path and change it (eventually padding the string with \0). This is really not the best idea because you should have as many versions of the edited binary as nodes.
    – Xavier
    Jun 7 '14 at 18:11
  • The problem with the binary is just the string length: padding is ok if the new path is shorter, won't work if longer and will require symlinks - but could work (but very ugly :D). I didn't think about chroot... That may actually work! I'll give a try, thanks!
    – AkiRoss
    Jun 8 '14 at 12:47
  • Did chroot worked?
    – Xavier
    Jun 26 '14 at 2:49
  • Sorry but did not had the chance to try with chroot :(
    – AkiRoss
    Jun 26 '14 at 12:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.