I am trying to work with the expect command, everything works well until I try to pipe the results of a command into a Gzip file. Here is the snippet of the bash file:

XYZ=$(expect -c "
spawn python log-connector.py -s $strt -e $end -i 600 -a https://server:9000 -u someaccount -f ./conf/firewall-fields2.txt -q ./conf/query.txt -z | gzip >> /data/sources/results-$strt2-$end2.json.gz

expect \"Password:\"

send \"$pass\r\"



echo "$XYZ"

$start - variable (start date)
$end - variable (end date)
$strt2 - variable (start date with 00:00)
$end2 - variable (end date with 23:59)
$pass - Password entered by the user previously.

If I remove

| gzip >> /data/sources/results-$strt2-$end2.json.gz

from the above it works as expected. When I add it I get the following error:

send: spawn id exp7 not open
    while executing

usage: API [-h] [-s START] [-e END] [-d DAY] [-i INTERVAL]
                    [-a LOGGER] [-l LOGFILE] [-f FIELDS] [-q QUERY]
                    [-o OUTPUT] [-t FORMAT] [-u USER] [-p PASS] [-z]
API: error: unrecognized arguments: | gzip >> /data/sources/results-20180604-20180604.json.gz

1 Answer 1


You might want to use a shell to interpret the pipe and redirection: I'll use a heredoc to make quoting easier

XYZ=$(expect <<END_EXPECT
    set timeout -1
    spawn sh -c {python log-connector.py -s $strt -e $end -i 600 -a https://server:9000 -u someaccount -f ./conf/firewall-fields2.txt -q ./conf/query.txt -z | gzip >> /data/sources/results-$strt2-$end2.json.gz}
    expect "Password:"
    send "$pass\r"
    expect eof

echo "$XYZ"
  • Well, I thought this did the trick and it sort of does, It runs the the python script once, gets the first chunk of a 10 min log, but does not continue getting the others. So its not really executing to completeness
    – ximian
    Jun 6, 2018 at 12:18
  • It should go until I get to the $end2 - variable (end date with 23:59), is there a way to check this in the expect ?
    – ximian
    Jun 6, 2018 at 12:48
  • If it takes longer than 10 seconds then set timeout -1 at the top of the expect body Jun 6, 2018 at 12:56
  • It can take 10 minutes like it could take a few hours for the logs to be parsed, so will the set timeout -1 handle this ? Also at the top of the expect body, you mean before XYZ=$.
    – ximian
    Jun 6, 2018 at 13:08
  • -1 means "do not time out". Put it before spawn Jun 6, 2018 at 13:43

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .