I am trying to work with the expect command, everything works well until I try to pipe the results of a command into a Gzip file. Here is the snippet of the bash file:

XYZ=$(expect -c "
spawn python log-connector.py -s $strt -e $end -i 600 -a https://server:9000 -u someaccount -f ./conf/firewall-fields2.txt -q ./conf/query.txt -z | gzip >> /data/sources/results-$strt2-$end2.json.gz

expect \"Password:\"

send \"$pass\r\"



echo "$XYZ"

$start - variable (start date)
$end - variable (end date)
$strt2 - variable (start date with 00:00)
$end2 - variable (end date with 23:59)
$pass - Password entered by the user previously.

If I remove

| gzip >> /data/sources/results-$strt2-$end2.json.gz

from the above it works as expected. When I add it I get the following error:

send: spawn id exp7 not open
    while executing

usage: API [-h] [-s START] [-e END] [-d DAY] [-i INTERVAL]
                    [-a LOGGER] [-l LOGFILE] [-f FIELDS] [-q QUERY]
                    [-o OUTPUT] [-t FORMAT] [-u USER] [-p PASS] [-z]
API: error: unrecognized arguments: | gzip >> /data/sources/results-20180604-20180604.json.gz

You might want to use a shell to interpret the pipe and redirection: I'll use a heredoc to make quoting easier

XYZ=$(expect <<END_EXPECT
    set timeout -1
    spawn sh -c {python log-connector.py -s $strt -e $end -i 600 -a https://server:9000 -u someaccount -f ./conf/firewall-fields2.txt -q ./conf/query.txt -z | gzip >> /data/sources/results-$strt2-$end2.json.gz}
    expect "Password:"
    send "$pass\r"
    expect eof

echo "$XYZ"
  • Well, I thought this did the trick and it sort of does, It runs the the python script once, gets the first chunk of a 10 min log, but does not continue getting the others. So its not really executing to completeness – ximian Jun 6 '18 at 12:18
  • It should go until I get to the $end2 - variable (end date with 23:59), is there a way to check this in the expect ? – ximian Jun 6 '18 at 12:48
  • If it takes longer than 10 seconds then set timeout -1 at the top of the expect body – glenn jackman Jun 6 '18 at 12:56
  • It can take 10 minutes like it could take a few hours for the logs to be parsed, so will the set timeout -1 handle this ? Also at the top of the expect body, you mean before XYZ=$. – ximian Jun 6 '18 at 13:08
  • -1 means "do not time out". Put it before spawn – glenn jackman Jun 6 '18 at 13:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.