4

I want to store in a text file some select statement that contain variables like a date:

$ cat res.txt
select field1, field2 from log and log_time~'"${TODAY}"*'

I want to use a script that will parse this file and for each statement, execute it. Of course, ${TODAY} will be replaced with a variable.

Unfortunately, the ${TODAY} is never replaced by a date when I read the file...

Is there a way to achieve this?

UPDATE:

Let say that I want to iterate through a file that containts some sql statements, and I want to execute them using psql (postgres)

Here is the file:

$cat sql.res
select path from log where log_host='toto' and log_time~'"${TODAY}"*' and log_msg='POLICY MISSING' and entry_status!='ACK' and path~'^/logs-${cli}/"${THREE_MONTHS_AGO}"_[0-9][0-9][0-9][0-9][0-9][0-9](_[0-9]{1,2}\.|\.)log(ptr|initial_ptr|account_ptr)?\.bz2$';

Now in a bash script, I want to:

$cat script.sh
TODAY=$(date "+%Y-%m-%d")
THREE_MONTHS_AGO=$(date -d "$TODAY -91 days" "+%Y-%m-%d")
THREE_MONTHS_AGO_2=$(date -d "$TODAY -92 days" "+%Y-%m-%d")
ONE_MONTH_AGO=$(date -d "$TODAY -36 days" "+%Y-%m-%d")
ONE_MONTH_AGO_2=$(date -d "$TODAY -37 days" "+%Y-%m-%d")
YESTERDAY=$(date -d "$TODAY -1 day" "+%Y-%m-%d")
while read item; do
    psql -P pager=off -t -c "$item" log | sed -e 's|^ *||' -e '/^$/d' > result.txt
done < sql.res

Unfortunately, the variables are not used in the script. So I have tried eval. I have like this:

$while read item; do
    c=$(echo $item | sed "s:':@:g" | sed "s:|:EE:g" | sed "s:(:AA:g" | sed "s:):BB:g" | sed 's:]{:TT:g' | sed 's|\.|OO|g')
    b=$(eval echo ${c} 2>&1)
    d=$(echo $b | sed "s:@:':g" | sed "s:EE:|:g" | sed "s:AA:(:g" | sed "s:BB:):g" | sed 's:TT:]{:g' | sed 's:OO:\\\.:g' )
    echo  "-> $b"
done < sql.res

But I have some issues with

\.
  • Execute it using what? – Mikel Mar 28 '12 at 15:37
3

You can simply replace the variable string with the value:

sed -e "s/\${TODAY}/$TODAY/g" res.txt

But:

  • Be absolutely sure you have run the variable through a reliable (as in, created by the DB provided or part of a popular programming language API, not something homebrewed) escape mechanism.
  • You'll also have to escape any characters specific to sed (and you can't just add backslashes before each character).
  • You're using two levels of quotes, which you probably didn't intend.
  • Please use a better language than Bash for this. It's horrendous for this kind of thing, because all the quoting and escaping business which complicates and makes it really easy to miss special cases. Python, Ruby and even PHP handle this sort of thing much better than Bash.
  • Thanks! I have also found this way, even if it is quite complex: while read item; do TODAY="2012-03-28";c=$(echo $item | sed "s/'/@/g");b=$(eval echo "$c");d=$(echo $b | sed "s/\@/\'/g"); echo "-> $d";done < res -> select field1, field2 from log and log_time~'2012-03-28*' Maybe there could be some more efficient code but it seems to work. – Romain Pelissier Mar 28 '12 at 15:51
  • Can you please put that code in the question (or a separate one), if it's relevant to the original question? It's quite hard to read in a comment. Also, use more quotes :) – l0b0 Mar 29 '12 at 6:47
  • I agree with point 4, pick a better tool. A related discussion in How to replace a list of placeholders in a text file? mentions cpp and m4. – manatwork Mar 29 '12 at 7:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.