23

I need to create a config file for my own script: here an example:

script:

#!/bin/bash
source /home/myuser/test/config
echo "Name=$nam" >&2
echo "Surname=$sur" >&2

Content of /home/myuser/test/config:

nam="Mark"
sur="Brown"

that's works!

My question: is this the correct way to do this or there're other ways?

  • The variables should be at the top. I'm surprised it works. Anyway, why do you need a config file? Are you planning to use these variables somewhere else? – Faheem Mitha Dec 23 '14 at 17:01
  • Faheem, I need the variables because my script has many options: using a config file will semplify the script. Thanks – Pol Hallen Dec 23 '14 at 17:05
  • 5
    IMHO its fine. I would do this way. – Tinti Dec 23 '14 at 17:11
  • abcde also does it this way and that is a quite big program (for a shell script). You can have a look at it here. – Lucas Dec 23 '14 at 21:04
18

source is not secure as it will execute arbitrary code. This may not be a concern for you, but if file permissions are incorrect, it may be possible for an attacker with filesystem access to execute code as a privileged user by injecting code into a config file loaded by an otherwise-secured script such as an init script.

So far, the best solution I've been able to identify is the clumsy reinventing-the-wheel solution:

myscript.conf

password=bar
echo rm -rf /
PROMPT_COMMAND='echo "Sending your last command $(history 1) to my email"'
hostname=localhost; echo rm -rf /

Using source, this would run echo rm -rf / twice, as well as change the running user's $PROMPT_COMMAND. Instead, do this:

myscript.sh (Bash 4)

#!/bin/bash
typeset -A config # init array
config=( # set default values in config array
    [username]="root"
    [password]=""
    [hostname]="localhost"
)

while read line
do
    if echo $line | grep -F = &>/dev/null
    then
        varname=$(echo "$line" | cut -d '=' -f 1)
        config[$varname]=$(echo "$line" | cut -d '=' -f 2-)
    fi
done < myscript.conf

echo ${config[username]} # should be loaded from defaults
echo ${config[password]} # should be loaded from config file
echo ${config[hostname]} # includes the "injected" code, but it's fine here
echo ${config[PROMPT_COMMAND]} # also respects variables that you may not have
               # been looking for, but they're sandboxed inside the $config array

myscript.sh (Mac/Bash 3-compatible)

#!/bin/bash
config() {
    val=$(grep -E "^$1=" myscript.conf 2>/dev/null || echo "$1=__DEFAULT__" | head -n 1 | cut -d '=' -f 2-)

    if [[ $val == __DEFAULT__ ]]
    then
        case $1 in
            username)
                echo -n "root"
                ;;
            password)
                echo -n ""
                ;;
            hostname)
                echo -n "localhost"
                ;;
        esac
    else
        echo -n $val
    fi
}

echo $(config username) # should be loaded from defaults
echo $(config password) # should be loaded from config file
echo $(config hostname) # includes the "injected" code, but it's fine here
echo $(config PROMPT_COMMAND) # also respects variables that you may not have
               # been looking for, but they're sandboxed inside the $config array

Please reply if you find a security exploit in my code.

  • 1
    FYI this is a Bash version 4.0 solution which sadly is subject to insane licensing issues imposed by Apple and is not available by default on Macs – Sukima Dec 6 '16 at 16:16
  • @Sukima Good point. I've added a version that is compatible with Bash 3. Its weakness is that it will not handle * in inputs properly, but then what in Bash handles that character well? – Mikkel Dec 6 '16 at 20:16
  • The first script fails if the password contains a backslash. – Kusalananda Feb 27 '17 at 20:06
  • @Kusalananda What if the backslash is escaped? my\\password – Mikkel Feb 27 '17 at 23:38
9

Here is a clean and portable version which is compatible with Bash 3 and up, on both Mac and Linux.

It specifies all defaults in a separate file, to avoid the need for a huge, cluttered, duplicated "defaults" config function in all of your shell scripts. And it lets you choose between reading with or without default fallbacks:

config.cfg:

myvar=Hello World

config.cfg.defaults:

myvar=Default Value
othervar=Another Variable

config.shlib (this is a library, so there is no shebang-line):

config_read_file() {
    (grep -E "^${2}=" -m 1 "${1}" 2>/dev/null || echo "VAR=__UNDEFINED__") | head -n 1 | cut -d '=' -f 2-;
}

config_get() {
    val="$(config_read_file config.cfg "${1}")";
    if [ "${val}" = "__UNDEFINED__" ]; then
        val="$(config_read_file config.cfg.defaults "${1}")";
    fi
    printf -- "%s" "${val}";
}

test.sh (or any scripts where you want to read config values):

#!/usr/bin/env bash
source config.shlib; # load the config library functions
echo "$(config_get myvar)"; # will be found in user-cfg
printf -- "%s\n" "$(config_get myvar)"; # safer way of echoing!
myvar="$(config_get myvar)"; # how to just read a value without echoing
echo "$(config_get othervar)"; # will fall back to defaults
echo "$(config_get bleh)"; # "__UNDEFINED__" since it isn't set anywhere

Explanation of the test script:

  • Note that all usages of config_get in test.sh are wrapped in double quotes. By wrapping every config_get in double quotes, we ensure that text in the variable value will never be misinterpreted as flags. And it ensures that we preserve whitespace properly, such as multiple spaces in a row in the config value.
  • And what's that printf line? Well, it's something you should be aware of: echo is a bad command for printing text that you have no control over. Even if you use double quotes, it will interpret flags. Try setting myvar (in config.cfg) to -e and you will see an empty line, because echo will think that it's a flag. But printf doesn't have that problem. The printf -- says "print this, and don't interpret anything as flags", and the "%s\n" says "format the output as a string with a trailing newline, and lastly the final parameter is the value for printf to format.
  • If you aren't going to be echoing values to the screen, then you'd simply assign them normally, like myvar="$(config_get myvar)";. If you're going to print them to the screen, I suggest using printf to be totally safe against any echo-incompatible strings that may be in the user config. But echo is fine if the user-provided variable isn't the first character of the string you are echoing, since that's the only situation where "flags" could be interpreted, so something like echo "foo: $(config_get myvar)"; is safe, since the "foo" doesn't begin with a dash and therefore tells echo that the rest of the string isn't flags for it either. :-)
  • @user2993656 Thanks for spotting that my original code still had my private config filename (environment.cfg) in it instead of the correct one. As for the "echo -n" edit you did, that depends on the shell used. On Mac/Linux Bash, "echo -n" means "echo without trailing newline", which I did to avoid trailing newlines. But it seems to work exactly the same without it, so thanks for the edits! – gw0 Feb 27 '17 at 19:12
  • Actually, I just went through and rewrote it to use printf instead of echo, which ensures that we'll get rid of the risk of echo misinterpreting "flags" in the config values. – gw0 Feb 27 '17 at 19:49
  • I really like this version. I dropped the config.cfg.defaults in lieu of defining them at the time of calling $(config_get var_name "default_value"). tritarget.org/static/… – Sukima Oct 22 '17 at 4:12
7

Parse the configuration file, don't execute it.

I'm currently writing an application at work that uses an extremely simple XML configuration:

<config>
    <username>username-or-email</username>
    <password>the-password</password>
</config>

In the shell script (the "application"), this is what I do to get at the username (more or less, I've put it in a shell function):

username="$( xml sel -t -v '/config/username' "$config_file" )"

The xml command is XMLStarlet, which is available for most Unices.

I'm using XML since other parts of the application also deals with data encoded in XML files, so it was easiest.

If you prefer JSON, there's jq which is an easy to use shell JSON parser.

My configuration file would look something like this in JSON:

{                                 
  "username": "username-or-email",
  "password": "the-password"      
}                

And then I'd be getting the username in the script:

username="$( jq -r '.username' "$config_file" )"
  • Executing the script has a number of advantages and disadvantages. The main disadvantages are security, if someone can alter the config file then they can execute code, and it is harder to make it idiot proof. The advantages are speed, on a simple test it is more than 10,000 times faster to source a config file than to run pq, and flexibility, anyone who likes monkey patching python will appreciate this. – icarus Jan 4 '17 at 21:33
  • @icarus How big configuration files do you usually encounter, and how often do you need to parse them in one session? Notice too that several values may be had out of XML or JSON in one go. – Kusalananda Jan 4 '17 at 21:36
  • Usually only a few (1 to 3) values. If you are using eval to set multiple values then you are executing selected parts of the config file :-). – icarus Jan 4 '17 at 22:39
  • 1
    @icarus I was thinking arrays... No need to eval anything. The performance hit of using a standard format with an existing parser (even though it's an external utility) is negligible in comparison to the robustness, the amount of code, the ease of use, and maintainability. – Kusalananda Jan 4 '17 at 23:32
  • 1
    +1 for "parse the config file, don't execute it" – Iiridayn Jul 3 '17 at 22:56
4

The most common, efficient and correct way is to use source, or . as a shorthand form. For example:

source /home/myuser/test/config

or

. /home/myuser/test/config

Something to consider, however, is the security issues that using an additional externally-sourced configuration file can raise, given that additional code can be inserted. For more information, including on how to detect and resolve this issue, I would recommend taking a look at the 'Secure it' section of http://wiki.bash-hackers.org/howto/conffile#secure_it

  • 4
    I had high hopes for that article (came up in my search results too), but the author's suggestion of attempting to use regex to filter out malicious code is an exercise in futility. – Mikkel May 28 '15 at 23:02
  • The procedure with dot, requires an absolute path? With the relative one it doesn't work – Davide Oct 23 '18 at 11:50
1

I use this in my scripts:

sed_escape() {
  sed -e 's/[]\/$*.^[]/\\&/g'
}

cfg_write() { # path, key, value
  cfg_delete "$1" "$2"
  echo "$2=$3" >> "$1"
}

cfg_read() { # path, key -> value
  test -f "$1" && grep "^$(echo "$2" | sed_escape)=" "$1" | sed "s/^$(echo "$2" | sed_escape)=//" | tail -1
}

cfg_delete() { # path, key
  test -f "$1" && sed -i "/^$(echo $2 | sed_escape).*$/d" "$1"
}

cfg_haskey() { # path, key
  test -f "$1" && grep "^$(echo "$2" | sed_escape)=" "$1" > /dev/null
}

Should support every character combination, except keys can't have = in them, since that's the seperator. Anything else works.

% cfg_write test.conf mykey myvalue
% cfg_read test.conf mykey
myvalue
% cfg_delete test.conf mykey
% cfg_haskey test.conf mykey || echo "It's not here anymore"
It's not here anymore

Also, this is completely safe since it doesn't use any kind of source/eval

0

For my scenario, source or . was fine, but I wanted to support local environment variables (ie, FOO=bar myscript.sh) taking precedence over configured variables. I also wanted the config file to be user editable and comfortable to someone used to sourced config files, and to keep it as small/simple as possible, to not distract from the main thrust of my very small script.

This is what I came up with:

CONF=${XDG_CONFIG_HOME:-~/config}/myscript.sh
if [ ! -f $CONF ]; then
    cat > $CONF << CONF
VAR1="default value"
CONF
fi
. <(sed 's/^\([^=]\+\) *= *\(.*\)$/\1=${\1:-\2}/' < $CONF)

Essentially - it checks for variable definitions (without being very flexible about whitespace) and rewrites those lines so that the value is converted to a default for that variable, and the variable is unmodified if found, like the XDG_CONFIG_HOME variable above. It sources this altered version of the config file and continues on.

Future work could make the sed script more robust, filter out lines that look weird or aren't definitions, etc, not break on end of line comments - but this is good enough for me for now.

-2

You can do it:

#!/bin/bash
name="mohsen"
age=35
cat > /home/myuser/test/config << EOF
Name=$name
Age=$age
EOF

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.