0

I'm confused on how bash interprets blanks while executing a script. In the end, I want a script that is fed by a csv file containing software used by an organisation and it should return the CVEs for these programs, by calling a python script to fetch this information.

My file would look this, but with over 500 lines:

"CRYSTAL REPORTS 2008";"SAP";"reporting software";;"Mr. Smith"

And is fed to my script like

$ getcve.sh < test.csv

When I run this small script I get strange results concerning the wordcount (which I wanted to use to loop through the python script's output to store in another file):

Read from file
Source: CVE-2010-2590 CVE-2010-3032 
Words in variable: 2
CVE-2010-2590 CVE-2010-3032 
Words processed: 1

However, when I hardcode "SAP CRYSTAL REPORTS 2008" in the script, the count changes to what I would expect:

 Hardcoded query
 Query: "SAP CRYSTAL REPORTS 2008"
 Source: CVE-2010-2590 CVE-2010-3032
 Words in variable: 2
 CVE-2010-2590
 CVE-2010-3032
 Words processed: 2

The script itself looks like this:

#!/bin/bash
clear
echo "Hardcoded query"
query='"SAP CRYSTAL REPORTS 2008"'
echo "Query: "$query
var2=$(python3 $HOME/cve-search/bin/search_fulltext.py -q "$query"  | tr '\n' ' ')
echo "Source: "$var2
i=0
echo "Words in variable: "$(echo "$var2"|wc -w)
for cve in $var2
do
    echo $cve
    i=$[ $i+1 ]
done
echo "Words processed: "$i

echo
echo "Read from file"
IFS_OLD=$IFS
IFS=";"
while read title firm desc version manager
do
    query='"'$(echo $firm $title $version | tr -d '"')'"'
    var3=$(python3 $HOME/cve-search/bin/search_fulltext.py -q "$query" | tr '\n' ' ')
    echo "Source: "$var3
    i=0
    echo "Words in variable: "$(echo "$var3"|wc -w)
    for cve in $var3
    do
        echo $cve
        i=$[ $i+1 ]
    done
    echo "Words processed: "$i
done
IFS=$IFS_OLD

Is there a trick or method to get the same results as the hard coded query when reading from a file?

I stumbled onto this by toying a little bit around (shell scripting is new for me) and this weird result is bothering me ^^"

Thank you in advance for your help :)

  • I think you have a problem of quotes, could you add the line echo "Query: "$query before line var3=... and post the result? – Mathieu Jan 7 '16 at 14:29
  • Hi @purplepsycho. Thank you for your reply. The result is the same: Hardcoded query Query: "SAP CRYSTAL REPORTS 2008" Source: CVE-2010-2590 CVE-2010-3032 Words in variable: 2 CVE-2010-2590 CVE-2010-3032 Words processed: 2 Read from file Query: "SAP CRYSTAL REPORTS 2008" Source: CVE-2010-2590 CVE-2010-3032 Words in variable: 2 CVE-2010-2590 CVE-2010-3032 Words processed: 1 – AaA Jan 7 '16 at 14:34
2

Your problem comes from IFS=";" I think: this modification will have an impact on the for loop.

Try:

IFS_OLD=$IFS
IFS=";"
while read title firm desc version manager
do
    query='"'$(echo $firm $title $version | tr -d '"')'"'
    var3=$(python3 $HOME/cve-search/bin/search_fulltext.py -q "$query" | tr '\n' ' ')
    echo "Source: "$var3
    i=0
    echo "Words in variable: "$(echo "$var3"|wc -w)
    IFS=" "
    for cve in $var3
    do
        echo $cve
        i=$[ $i+1 ]
    done
    IFS=";"
    echo "Words processed: "$i
done
IFS=$IFS_OLD
  • Excellent :D That solved my problem. I read to be careful when using IFS, but I thought it only applied to file and not so somethings like looping through a variable. I'll make sure to remember this. Thank you very much @purplepsycho – AaA Jan 7 '16 at 14:48
  • ... or don't change IFS in the body of the script at all, instead set it explicitly for the read command while IFS=";" read title firm desc version manager – steeldriver Jan 7 '16 at 14:48
  • That sounds like a much safer way @steeldriver. What would be a good book to learn shell scripting in a proper way? I'm mostly experimenting and reading small articles on blogs etc. – AaA Jan 7 '16 at 14:51
  • @AkiraTakizawa unfortunately there's a lot of so-so scripting advice out there: my go-to for this kind of stuff is mywiki.wooledge.org/BashGuide – steeldriver Jan 7 '16 at 15:15
  • That's added to my bookmarks (^_^)b – AaA Jan 7 '16 at 17:57
0

If you place a string with spaces in a variable, then that will expand to one value when used like for i in $var3. However, when you use echo $var3 the output will contain the spaces and wc -w will count those.

You need to modify your loop to

for cve in $(echo $var3)

so that the value is re-evaluated and the spaces cause the value to be split up into discrete words.

  • Thank you for your suggestion. Unfortunately, this didn't change the output. @purplepsycho's solution below did the trick :) – AaA Jan 7 '16 at 14:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.