Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

For some reason I cannot pass the bash variable $FOLDER as a python argument on the following code. The python script downloads some files from amazon s3.

Bash script:

#!/bin/bash
FOLDER=$(./aws get $BUCKET"/location.txt")
python /data/pythonscript.py $FOLDER
#

The output of the $FOLDER is a regenerated date eg. 2014/07/31/14-16-34 which is used as a path.

Here is the python script:

#!/usr/bin/python

import boto, sys
from boto.s3.connection import S3Connection

access_key = 'accesskey'
secret_key = 'secretkey'
bucket_name = 'a name'
folder_path =  str(sys.argv[1]) if len(sys.argv) > 1 else ''

print("Forwarded folder path " + folder_path)

conn = S3Connection(access_key, secret_key)
bucket = conn.get_bucket(bucket_name)

print("Bucket Location:" + bucket.get_location())

for key in bucket.list(prefix=folder_path, delimiter=''):
        if '.' in key.name:
                file_name = key.name[len(folder_path)+1:]
                print("Downloading file " + file_name)
                key.get_contents_to_filename('/data/temp/' + file_name)

When I execute the bash script without changing the python /data/pythonscript.py $FOLDER line, I get the following output:

Forwarded folder path 2014/07/31/14-16-34 
Buckect Location: 

But when I change it to python /data/pythonscript.py 2014/07/31/14-16-34 , everything works:

Forwarded folder path 2014/07/31/14-16-34
Bucket Location: 
Downloading 2014/07/31/14-16-34/FacetedSearch.zip 
Downloading file FacetedSearch.zip
Downloading 2014/07/31/14-16-34/Location.zip
Downloading file Location.zip
Downloading 2014/07/31/14-16-34/LocationPage.zip
Downloading file LocationPage.zip
share|improve this question
    
What about this doesn't work? Does the python script not get called with $FOLDER expanded? –  drs Jul 31 at 14:20
    
Python is getting the $FOLDER argument but rest of the code doesn't work as expected. But if I change the /data/pythonscript.py $FOLDER to /data/pythonscript.py 2014/07/31/14-16-34 in bash script, it works. Python file also works from command line independently. –  Bris Jul 31 at 14:29
    
You've framed your question nicer than a lot of new users, but "rest of the code doesn't work as expected" is still pretty vague. What happens? Is there an error? What is the value of folder_path when using the bash variable $FOLDER as the argument? –  drs Jul 31 at 14:42
    
Man, is s3download.py the same script, you called pythonscript.py earlier? –  Bob Jul 31 at 15:42
    
Just a minor complaint, but you mistyped bucket: Buckect Location in line 16. –  polym Jul 31 at 15:49

2 Answers 2

up vote 6 down vote accepted

Perhaps the aws bash command is returning non-printable characters that you don't see with print(). Try removing them with tr:

FOLDER=$(./aws get $BUCKET"/location.txt" | tr -cd "[:print:]")
share|improve this answer
    
This works perfectly! Many thanks drs –  Bris Jul 31 at 16:57

Try to put $FOLDER inside double quotes:

python script.py "$FOLDER"

Change the first line of your script to

#!/usr/bin/env python

This should work.

And be more clear on your questions.

As @drs said, your question is still incomplete:

  • Your examples are still vague and with flaws, like what is the correct name of your script, is it pythonscript or s3download?
  • Is it "work as expected" supposed to have a blank value for bucket.get_location()?
share|improve this answer
    
I tried your suggestions but unfortunately, I am still getting the same result. The script is named pythonscript.py and the bucket.get_location() indeed returns a blank value. But apart from that blank value the script manages to download the files from amazon s3 when I use the mentioned method. –  Bris Jul 31 at 16:04

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.