Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

My bash script needs to read the variables from a text file that contains hundreds of variables, most of which conform to standard bash variable syntax (e.g., VAR1=VALUE1). However, a few lines in this file can be problematic, and I hope to find a simple solution to reading them into bash.

Here's what the file looks like:

VAR6=VALUE 6 #a comment
VAR9=a name with spaces

The rules about the file structure include:

  • one variable (with value) per line
  • the assignment (=) has no spaces around it
  • the variable name is in the first column (unless it is a comment line)
  • comments can follow the value (after #)
  • the values can include spaces, parens, slashes, commas, and other chars
  • the values can be floating point numbers (2.1), integers, true/false, or strings.
  • string values are not quoted, and they can be a thousand chars long or longer
  • the variable name contains only letters and underscores UPDATE: and numbers.

Most of the variables are of a type that would just allow me to source the file into my bash script. But those few problematic ones dictate a different solution. I'm not sure how to read them.

share|improve this question
Can you just put the RHS into single quotes? –  Ketan Feb 22 '14 at 2:05
@Ketan - how do you mean? I can't change the text file with the variables. –  MountainX Feb 22 '14 at 2:15
You don't have to change the original text file, run it through sed and put the output in a new file and source it. –  Ketan Feb 22 '14 at 2:17
@Ketan - that sounds like it should work... can you provide an answer with example code? The devil is often in the details, especially with bash. Thank you. –  MountainX Feb 22 '14 at 2:20

2 Answers 2

up vote 1 down vote accepted

While you can transform this file to be a shell snippet, it's tricky. You need to make sure that all shell special characters are properly quoted.

The easiest way to do that is to put single quotes around the value and replace single quotes inside the value by '\''. You can then put the result into a temporary file and source that file.

sed <"config-file" >"$script" \
  -e '/^[A-Z_a-z][A-Z_a-z]*=/ !d' \
  -e s/\'/\'\\\\\'\'/g \
  -e s/=/=\'/ -e s/\$/\'/

I recommend doing the parsing directly in the shell instead. The complexity of the code is about the same, but there are two major benefits: you avoid the need for a temporary file, and the risk that you accidentally got the quoting wrong and end up executing a part of a line as a shell snippet (something like dangerous='$(run me)'). You also get a better chance at validating potential errors.

while IFS= read -r line; do
  line=${line%%#*}  # strip comment (if any)
  case $line in
      case $var in
          echo "Warning: invalid variable name $var ignored" >&2
      if eval '[ -n "${'$var'+1}" ]'; then
        echo "Warning: variable $var already set, redefinition ignored" >&2
      eval $var='"$line"'
done <"config-file"
share|improve this answer
I'm using your recommended (2nd) method. Works great. Is there an easy way to check for a variable naming collision during this while-loop? With this approach it seems like that check might be appropriately incorporated. –  MountainX Feb 23 '14 at 0:43
Regarding name collisions, I guess I could use eval IMPORTED_$var='"$line"' but I would rather first check for collisions and not change/prefix the variable names. –  MountainX Feb 23 '14 at 0:53
@MountainX See my edit. –  Gilles Feb 23 '14 at 0:55
BTW, I made a mistake regarding the variable name convention. The standard letters, numbers and underscores are allowed. So the char class expression should be [!A-Z_a-z0-9]. –  MountainX Feb 23 '14 at 0:57
Thanks for the edit. This is an awesome solution! –  MountainX Feb 23 '14 at 0:59

Assuming your contents are in a file x.txt, you can use a sed expression:

sed -e 's/#.*//' -e 's/=.*$/="&"/g' -e 's/=//2' x.txt

First expression puts everything after the = sign in quotes. Second expression removes extra = sign and the third expression removes everything between # and " as they are comments.

Following will quote RHS into single quotes:

sed -e "s/#.*//" -e "s/=.*$/='&'/g" -e "s/=//2" x.txt
share|improve this answer
The result is VAR1"=value1" –  MountainX Feb 22 '14 at 2:33
Yep, realized that and made an update. See if it works now. –  Ketan Feb 22 '14 at 2:34
Yes, that works! Thanks –  MountainX Feb 22 '14 at 2:42
This doesn't handle values containing " or '. –  Gilles Feb 22 '14 at 23:13
It also handles comments incorrectly (the leading quote is kept, the trailing quote is deleted together with the #). BTW, it's probably useful to delete the comment together with any preceding whitespace. –  Uwe Feb 22 '14 at 23:27

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.