I am trying to use a Python script to read/write from an external disk in linux, which works fine. The problem comes when I use a job scheduler to execute the Python program.
I am running all this on a Rocks Cluster (the linux version is CentOS 6.5) and the job scheduler is Sun Grid Engine (SGE). I've mounted the external disk such that all users have read and write permissions. The line from fstab for mounting the disk is:
/dev/sdb1 /mnt/drive ntfs-3g auto,users,permissions 0 0
Let me give you a simple example, which fails. The following python program (simple_write.py):
#!/usr/bin/env python2.7 f = open('/mnt/drive/sean/test.txt', 'w') f.write('Writing in the file') f.close()
runs fine if executed in a terminal like this:
If I then create a little bash script (job_submit.sh) for submitting to the scheduler (SGE) like this:
#!/bin/bash # #$ -cwd #$ -j y #$ -S /bin/bash python2.7 /home/sean/simple_write.py
and submit from a terminal:
Then I get the following (Python) error:
IOError: [Errno 2] No such file or directory: '/mnt/drive/sean/test.txt'
The problem can be resolved by (a) running simple_write.py directly from the terminal or (b) changing the write location in simple_write.py to somewhere on the local disk, rather than the external disk (a similar error occurs for reading a file/directory rather than writing). It therefore seems like it must be some issue with the scheduler not playing nicely with the mounted drive, but I'm at a loss at what to try next. Any help would be greatly appreciated.