Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I wanted to write a little bash function such that I can tell bash, import os or from sys import stdout and it will spawn a new Python interpreter with the module imported.

The latter from function looks like this:

from () {
    echo "from $@" | xxd
    python3 -i -c "from $@"
}

If I call this:

$ from sys import stdout
00000000: 6672 6f6d 2073 7973 2069 6d70 6f72 7420  from sys import 
00000010: 7374 646f 7574 0a                        stdout.
  File "<string>", line 1
    from sys
           ^
SyntaxError: invalid syntax
>>> 

The bytes in from sys are

66 72 6f 6d 20 73 79 73 20
f  r  o  m     s  y  s    

There's no EOF in there, yet the Python interpreter is behaving as if it read EOF. There is a newline at the end of the stream, which is to be expected.

from's sister, that imports a whole Python module, looks like this, and which solves the problem by sanitising and processing the string, and by failing on non-existent modules.

import () {
  ARGS=$@
  ARGS=$(python3 -c "import re;print(', '.join(re.findall(r'([\w]+)[\s|,]*', '$ARGS')))")
  echo -ne '\0x04' | python3 -i
  python3 -c "import $ARGS" &> /dev/null
  if [ $? != 0 ]; then
    echo "sorry, junk module in list"
  else
    echo "imported $ARGS"
    python3 -i -c "import $ARGS"
  fi
}

That solves the problem of an unexplained EOF in the stream, but I would like to understand why Python thinks there is an EOF.

share|improve this question
up vote 42 down vote accepted

The table in this Stack Overflow answer (which got it from the Bash Hackers Wiki) explains how the different Bash variables are expanded:

You're doing python -i -c "from $@", which turns into python -i -c "from sys" "import" "stdout", and -c only takes a single argument, so it's running the command from sys. You want to use $*, which will expand into python -i -c "from sys import stdout" (assuming $IFS is unset or starts with a space).

share|improve this answer
2  
Thank you for undeleting, as this is valuable information :) – cat Feb 18 at 17:17
1  
I think this should be the accepted answer as this actually solves the problem, the other upvoted one just explains the problem, but it doesn't give the solutions or alternative workarounds – Ferrybig Feb 19 at 11:05
    
@Ferrybig you're right, thank you – cat Feb 19 at 17:16
    
Good answer. That table actually comes from the Bash Hackers Wiki. Could you add the proper attribution and verify that you have the right to distribute? – Lightness Races in Orbit Feb 21 at 13:44

strace, as always, will show what is going on:

bash-4.1$ echo $$
3458

And, elsewhere (or you could figure out how to strace bash ... the function call):

bash-4.1$ strace -ff -o blah -p 3458

And back in that first shell:

bash-4.1$ from sys import stdout
  File "<string>", line 1
    from sys
           ^
SyntaxError: invalid syntax
>>> 
bash-4.1$ 

And then back in the strace shell:

Process 3458 attached
Process 25224 attached
^CProcess 3458 detached
bash-4.1$ grep exec blah.*
blah.25224:execve("/usr/bin/python", ["python", "-i", "-c", "from sys", "import", "stdout"], [/* 54 vars */]) = 0

Thus, the actual -c argument is -c "from sys" because of how "$@" is expanded, or a truncated command that python barfs on.

share|improve this answer

$@ in double quotes expands to a list of elements "$1" "$2" "$3" etc.

#!/bin/bash
expand () {
    for string in "from $@" ; do
        echo "$string"
    done
}

expand sys import stdout

Python expects the code to be in one argument, not a series of arguments.

share|improve this answer

Python is being invoked as

execve("/usr/bin/python", ["python", "-i", "-c", "from sys", "import", "stdout"], [/* 54 vars */])

(see thrig's answer).

To get $@ expanded as a single string (assuming a sane $IFS), you can use $* inside double quotes:

python3 -i -c "from $*"

Confirmed with strace -e execve:

execve("/usr/bin/python", ["python", "-i", "-c", "from sys import stdout"], [/* 54 vars */]) = 0
share|improve this answer

Strace do show what are the arguments used. But the simplest method to see what is being processed is to add a printf '<%s> ' before each relevant line, and a closing echo (to generate as new line):

So, the function could be changed to this:

from () {
    printf '<%s> ' "from $@"; echo
    printf '<%s> ' python3 -i -c "from $@"; echo
}

And when called:

$ from sys import stdout
<from sys> <import> <stdout> 
<python3> <-i> <-c> <from sys> <import> <stdout>

It is clear that "from sys" is being sent to python as one argument.
That is what python receives, and python acts on "from sys".

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.