Path Variable and script's shebang will not run correct version of python

Question

Problem Statement:

A bash script is being used to create a new screen session for each name from a list of names. At every name, a python script is run using that name as an input. The bash script sets the path containing the correct version of python (python 3 in the anaconda package):

#!/bin/sh

export PATH=~/anaconda3/bin/python:$PATH

while read p; do
  screen -dm -S $p bash -c "cd /inside/home/thjmatth/essential; python3 essentialpairs_ttest_tissue_1.py; exec sh"
done <cells.txt

Setting the path as above didn't allow the correct version of python to run, so I added the following shebang to the python script to run:

#!~/anaconda3/bin/python python3

Still no dice: /usr/bin/python is still being used when it shouldn't. How can I have this program run the version of python according to the path I specify?

Attempt 1:

new shebang:

#!/inside/home/thjmatth/anaconda3/bin/python/

new bash script:

#!/bin/sh

export PATH=~/anaconda3/bin/python:$PATH

while read p; do
  screen -dm -S $p bash -c "cd /inside/home/thjmatth/essential; python3 essentialpairs_ttest_tissue_1.py; exec sh"
done <cells.txt

new permissions on the python script:

chmod +x essentialpairs_ttest_tissue_1.py

error:

bash: python3: command not found

running which python in that screen still shows /usr/bin/python

Solution:

Same as Attempt 1 above, but changed line 3 of the bash script to make it a directory rather than an executable:

export PATH=~/anaconda3/bin/python:$PATH

Answer 1

export PATH=~/anaconda3/bin/python:$PATH

This seems like the path to an executable. PATH should contain directories:

export PATH=~/anaconda3/bin:$PATH

---

#!~/anaconda3/bin/python python3

1. Shebang lines need actual paths and do not perform tilde expansion (that's in your shell). Write the actual path to the executable, starting with /.

2. This shebang line would be running ~/anaconda3/bin/python with the argument python3, followed by the script name. You probably don't want that, but instead to run either python or python3.

   #!/home/thomas/anaconda3/bin/python
   

---

screen -dm -S $p bash -c "cd /inside/home/thjmatth/essential; python3 essentialpairs_ttest_tissue_1.py; exec sh"

If you were still running this line after updating the shebang line, it would be ignored; the python3 from your PATH (which is not the same as python!) would be executed and would run the script in turn, and would just skip the shebang line as a comment. If you make the script executable (chmod +x essentialpairs_ttest_tissue_1.py), you could run it directly:

./essentialpairs_ttest_tissue_1.py

and the shebang line would be processed.

---

In this case, you can probably get away with just fixing the PATH variable from the first section, unless you have Bash startup configuration that resets the variable. If the executable name is python, make sure you update your screen command line to use that instead of python3 as well.

Answer 2

Possible solution

regarding your second attempt, with the shebang in the python script:

• You can't use ~ in a shebang; it has to be the actual path (#!/inside/home/thjmatth/...).
• There shouldn’t be a space between the directory name and the program name; it has to be #!/inside/home/thjmatth/anaconda3/bin/python/python3.

Sanity check: Do you have a directory called /inside/home/thjmatth/anaconda3/bin/python that contains an executable program called python3 (i.e., the full pathname to the interpreter program is /inside/home/thjmatth/anaconda3/bin/python/python3?

Something to try

Change your script to do this:

#!/bin/sh

export PATH=~/anaconda3/bin/python:$PATH

while read p; do
  type python3
  screen -dm -S "$p" bash -c "type python3; exec sh"
done <cells.txt

and see if you can figure out what’s happening.  If you’re still stuck, edit your question to include the output of the above.

Another thing: You should always quote all references to shell variables (e.g., "$p") unless you have a good reason not to, and you’re sure you know what you’re doing.