Reading configuration from an openssl config file

Question

I'm struggling to find hint or solution for reading openssl config values in a shell script. Let me provide you a bit more details.

I have an openssl.conf file with the following simplified content:

[ca_one]
dir = /home/auhorities/ca_one
certs = $dir/certs
database = $dir/index.txt
serial = $dir/serial
[ca_two]
dir = /home/auhorities/ca_two
certs = $dir/certs
database = $dir/index.txt
serial = $dir/serial

I'm writing bash script, which will look into dir/certs directory for every certification authority, i.e. I need to get values /home/auhorities/ca_one/certs and /home/auhorities/ca_two/certs form the file. For now, I ended up with the following solution:

#!/usr/bin/env bash

for certs_dir in $(grep -E ^dir.+ openssl.conf | tr -d ' ' | cut -d '=' -f 2); do
  echo "$certs_dir/certs"
done

However, I'm afraid that this is not ideal if the script functionality will be updated in the future. I'm looking for solution which enables the the future me or colleagues to navigate the certification authority entries in a more conveniet way. For example, I've found openssl config manual(https://www.openssl.org/docs/manmaster/man5/config.html), which says that the config library "can be used to read configuration files". As far as my understanding goes, this library is used internally by openssl tools such as ca, req or other. Is it then possible to read the entries from the config file using some of the openssl utilities? I'm probably missing something, but I just can't find out if it is possible or not. If this is not an option, how would you deal with such a situation?

Thank you in advance.

Joseph Riopelle · Accepted Answer · 2020-06-07 03:08:37Z

There is no official command line tool to parse or validate openssl.cnf. I built a compatible bash function however. man config is certainly something you need to read and reread.

IMPORTANT: If you config is busted, this won't work. Fix it!

The example openssl.cnf needs to be a bit dirtier for the sake of testing:

#set this! it forces ${var}/$(var); treats $var as a string literal
.pragma = dollarid:true
#with a huge config you are eventually going to want to section it out
.include my_other_conf.cnf

#safe variable mapping, always use to avoid config errors
DS = / #set a default variable value (DS = \ in Windows)
DS = ${ENV::DS} #value above used if DS isn't mapped in the shell

[ca_one]
  #indent every section for readability
  dir      = /home/auhorities/ca_one
  certs    = $dir/certs
  database = $dir/index.txt
  serial   = $dir/serial

[ca_two]
  dir      = /home/auhorities/ca_two
  certs    = $dir/certs
  database = $dir/index.txt
  serial   = $dir/serial

  [  section_test0    ] #this is just how nasty (but valid) things get
  space_test=   " space  " ' test  '
var_test_ = boop

var_test.0 = ${var_test_}                #local syntax
var_test.1 = $(var_test_)                #alt syntax
var_test.2 = ${section_test0::var_test_} #$section::name syntax
var_test.3 = ${ENV::DS}                  #$ENV::name syntax
dollarid_test = $var_test_               #$dollarid:off only
escape_test = H\C3\A0 N\E1\BB\99i \ \# \\
test_multiline= 123 \\ \ \
    456\
    #789

Now sanitize the config file- remove comments, empty lines, trailing/leading white space, and white space inside [ section ] labels and name = value pairs.

Assign any resulting name=value pairs like SSL_CONF[section,name]=value with the following:

#!/bin/bash
declare -A SSL_CONF #initialize our config array
declare SSL_CONF_SECTION #set last section
function ssl_include(){
  local m a id d="$3" c="${1:-$OPENSSL_CONF}" e='a-zA-Z0-9_' #start in last section. dollarid:false (default). set conf file. regex to match sections
  [[ ! -r "$c" ]] && return     #the file isn't readable
  SSL_CONF_SECTION="${2/ /}" #set section
  [ -d "$c" ] && local d='nodir' c="\"${c%/}/*.cnf\" \"${c%/}/*.conf\""    #conf is a dir
  while IFS= read -r l || [ -n "$l" ]; do         #build SSL_CONF array
    l="${l%%[^\\]#*}"              #remove comment
    if [ "$m" != '' ]; then      #last line ended with /
      [[ "$l" =~ [^\\]\\$ ]] && l="${l:0:-1}" || m=''               #check for continued multiline
      SSL_CONF[${SSL_CONF_SECTION// /},${m}]="${SSL_CONF[${SSL_CONF_SECTION// /},${m}]}${l//[^\\]\\$/}" && continue #add current line to last conf and move to next line
    fi
    l="${l#"${l%%[![:space:]]*}"}"; l="${l%"${l##*[![:space:]]}"}"; [[ "$l" == '' ]] && continue #remove leading/trailing whitespace, then skip empty lines
    if [[ "$l" =~ ^\.include[[:space:]]*=[[:space:]]*(.*)$ ]]; then #include additional files
      [ "$d" == 'nodir' ] && continue            #dir loaded conf files cant include further
      local d='no' i="${BASH_REMATCH[1]}" o="${OPENSSL_CONF_INCLUDE:-${c%/*}}" #no variable parsing, store last match, handle default include path
      [[ ! "$i" =~ ^\.{0,2}/ ]] && i="${o%/}/${i}"  #append default path to relative paths
      for f in "$i"; do [ -r "$f" ] && ssl_include "$f" "${SSL_CONF_SECTION// /} " "$d"; done #parse additional configuration files, keeping section
      continue
    fi
    [[ "${SSL_CONF_SECTION// /}" == '' && "$l" =~ ^\.pragma[[:space:]]*=[[:space:]]*(dollarid)[[:space:]]*:[[:space:]]*(true|on)$ ]] && id=${BASH_REMATCH[2]} && continue #see how local variables are parsed
    [[ "$l" =~ ^\[[[:space:]]*([${e}]+)[[:space:]]*\]$ ]] && SSL_CONF_SECTION=${BASH_REMATCH[1]} && continue #set section name
    if [[ "$l" =~ ^([${e},\;.]+)[[:space:]]*=[[:space:]]*(.+)$ ]]; then #name value pair
      local n="${BASH_REMATCH[1]}" v="${BASH_REMATCH[2]}"
      [[ "$v" =~ [^\\]\\$ ]] && o="$n"          #found a multiline value
      SSL_CONF[${SSL_CONF_SECTION// /},${n}]="${v//\\[^nbrt\$\\\#]/}" && continue #add name value to SSL_CONF array
    fi
  done< <(cat $c 2>/dev/null) #loop through the config(s)
}

The logic followed was:

comments, blank lines, leading/trailing white space is ignored
[ section_name ] and name = value internal white space is ignored
.include and .pragma don't require an =. It was allowed for backwards compatability
.include can be anywhere in the file
.include /dir/ includes *.cnf and *.conf in the defined directory. It disables .include processing in the included files
section and $var names can consist of a-z,A-Z,0-9, and _
name can consist of a-z,A-Z,0-9,_,;,., and ,
a value ending in a single \ (backslash) continue to the next line
some sequences require escaping with \ like so: \\,\\$,\#,\n,\b,\r,\t

Now you have an associative array like SSL_CONF[section,name]=value. The .include=(.*) files are parsed recursively as soon as they are encountered. .pragma=dollarid:true is also processed so you can accurately parse variables.

Now you have one last problem: CONFIGURATION VARIABLES. They are currently assigned as ${var}, ${section::var}, ${ENV::var}, $var, $section::var, $ENV::var AND $(var), $(section::var), $(ENV::var) (who knew?). Luckily we can loop through our SSL_CONF array and assign the real values:

#!/bin/bash
[ "$id" != '' ] && a='\{'                     #ignore not bracketed variables
local f="[^\\\\]*(\\\$(\(|\{|${a})([${e}]+)(::([${e}]+))?(\)|\}|))" # match $var ${var} $(var) unless preceded by \
for k in "${!SSL_CONF[@]}"; do                #loop through our array looking for variables
  local o #last value placeholder
  while [ "${SSL_CONF[$k]}" != "$o" ]; do     #only loop if the variable changed
    o="${SSL_CONF[$k]}"                       #set loop to exit on no change
    if [[ "${SSL_CONF[$k]}" =~ $f ]] && \
     [[ "${BASH_REMATCH[2]}${BASH_REMATCH[6]}" == '' || "${BASH_REMATCH[2]}${BASH_REMATCH[6]}" == '()' ||  "${BASH_REMATCH[2]}${BASH_REMATCH[6]}" == '{}' ]] #brackets match
    then                                      #the value contains a variable
      local r=' #'                            #replacement indicator (illegal value)
      [[ "$r" == ' #' && ${SSL_CONF[${k%%,*},${BASH_REMATCH[3]}]+isset} ]]             && r="${SSL_CONF[${k%%,*},${BASH_REMATCH[3]}]}"              #local variable
      [[ "$r" == ' #' && ${SSL_CONF[default,${BASH_REMATCH[3]}]+isset} ]]              && r="${SSL_CONF[default,${BASH_REMATCH[3]}]}"               #'default' variable
      [[ "$r" == ' #' && ${SSL_CONF[,${BASH_REMATCH[3]}]+isset} ]]                     && r="${SSL_CONF[,${BASH_REMATCH[3]}]}"                      #default variable
      if [ "${BASH_REMATCH[5]}" != '' ]; then #variable is from another section, default, or ENV
        [[ "$r" == ' #' && "${BASH_REMATCH[3]}" == "ENV" ]]                            && r="${!BASH_REMATCH[5]:-${SSL_CONF[,${BASH_REMATCH[5]}]}}" #environment variable
        [[ "$r" == ' #' && ${SSL_CONF[${BASH_REMATCH[3]},${BASH_REMATCH[5]}]+isset} ]] && r="${SSL_CONF[${BASH_REMATCH[3]},${BASH_REMATCH[5]}]}"    #section variable
      fi
      [ "$r" != ' #' ] && SSL_CONF[$k]="${SSL_CONF[$k]//${BASH_REMATCH[1]}/$r}" #replace our variable with the value
    fi
  done
done

And now all of your variables are their computed values!

Here is the full function and example code:

#!/bin/bash
declare -A SSL_CONF #initialize our config array
declare SSL_CONF_SECTION #set last section
function ssl_include(){
  local m a id d="$3" c="${1:-$OPENSSL_CONF}" e='a-zA-Z0-9_' #start in last section. dollarid:false (default). set conf file. regex to match sections
  [[ ! -r "$c" ]] && return     #the file isn't readable
  SSL_CONF_SECTION="${2/ /}" #set section
  [ -d "$c" ] && local d='nodir' c="\"${c%/}/*.cnf\" \"${c%/}/*.conf\""    #conf is a dir
  while IFS= read -r l || [ -n "$l" ]; do         #build SSL_CONF array
    l="${l%%[^\\]#*}"              #remove comment
    if [ "$m" != '' ]; then      #last line ended with /
      [[ "$l" =~ [^\\]\\$ ]] && l="${l:0:-1}" || m=''               #check for continued multiline
      SSL_CONF[${SSL_CONF_SECTION// /},${m}]="${SSL_CONF[${SSL_CONF_SECTION// /},${m}]}${l//[^\\]\\$/}" && continue #add current line to last conf and move to next line
    fi
    l="${l#"${l%%[![:space:]]*}"}"; l="${l%"${l##*[![:space:]]}"}"; [[ "$l" == '' ]] && continue #remove leading/trailing whitespace, then skip empty lines
    if [[ "$l" =~ ^\.include[[:space:]]*=[[:space:]]*(.*)$ ]]; then #include additional files
      [ "$d" == 'nodir' ] && continue            #dir loaded conf files cant include further
      local d='no' i="${BASH_REMATCH[1]}" o="${OPENSSL_CONF_INCLUDE:-${c%/*}}" #no variable parsing, store last match, handle default include path
      [[ ! "$i" =~ ^\.{0,2}/ ]] && i="${o%/}/${i}"  #append default path to relative paths
      for f in "$i"; do [ -r "$f" ] && ssl_include "$f" "${SSL_CONF_SECTION// /} " "$d"; done #parse additional configuration files, keeping section
      continue
    fi
    [[ "${SSL_CONF_SECTION// /}" == '' && "$l" =~ ^\.pragma[[:space:]]*=[[:space:]]*(dollarid)[[:space:]]*:[[:space:]]*(true|on)$ ]] && id=${BASH_REMATCH[2]} && continue #see how local variables are parsed
    [[ "$l" =~ ^\[[[:space:]]*([${e}]+)[[:space:]]*\]$ ]] && SSL_CONF_SECTION=${BASH_REMATCH[1]} && continue #set section name
    if [[ "$l" =~ ^([${e},\;.]+)[[:space:]]*=[[:space:]]*(.+)$ ]]; then #name value pair
      local n="${BASH_REMATCH[1]}" v="${BASH_REMATCH[2]}"
      [[ "$v" =~ [^\\]\\$ ]] && o="$n"          #found a multiline value
      SSL_CONF[${SSL_CONF_SECTION// /},${n}]="${v//\\[^nbrt\$\\\#]/}" && continue #add name value to SSL_CONF array
    fi
  done< <(cat $c 2>/dev/null) #loop through the config(s)
  [ "$d" != '' ] && return         #don't parse variables in included files, just return the section name
[ "$id" != '' ] && a='\{'                     #ignore not bracketed variables
local f="[^\\\\]*(\\\$(\(|\{|${a})([${e}]+)(::([${e}]+))?(\)|\}|))" # match $var ${var} $(var) unless preceded by \
for k in "${!SSL_CONF[@]}"; do                #loop through our array looking for variables
  local o #last value placeholder
  while [ "${SSL_CONF[$k]}" != "$o" ]; do     #only loop if the variable changed
    o="${SSL_CONF[$k]}"                       #set loop to exit on no change
    if [[ "${SSL_CONF[$k]}" =~ $f ]] && \
     [[ "${BASH_REMATCH[2]}${BASH_REMATCH[6]}" == '' || "${BASH_REMATCH[2]}${BASH_REMATCH[6]}" == '()' ||  "${BASH_REMATCH[2]}${BASH_REMATCH[6]}" == '{}' ]] #brackets match
    then                                      #the value contains a variable
      local r=' #'                            #replacement indicator (illegal value)
      [[ "$r" == ' #' && ${SSL_CONF[${k%%,*},${BASH_REMATCH[3]}]+isset} ]]             && r="${SSL_CONF[${k%%,*},${BASH_REMATCH[3]}]}"              #local variable
      [[ "$r" == ' #' && ${SSL_CONF[default,${BASH_REMATCH[3]}]+isset} ]]              && r="${SSL_CONF[default,${BASH_REMATCH[3]}]}"               #'default' variable
      [[ "$r" == ' #' && ${SSL_CONF[,${BASH_REMATCH[3]}]+isset} ]]                     && r="${SSL_CONF[,${BASH_REMATCH[3]}]}"                      #default variable
      if [ "${BASH_REMATCH[5]}" != '' ]; then #variable is from another section, default, or ENV
        [[ "$r" == ' #' && "${BASH_REMATCH[3]}" == "ENV" ]]                            && r="${!BASH_REMATCH[5]:-${SSL_CONF[,${BASH_REMATCH[5]}]}}" #environment variable
        [[ "$r" == ' #' && ${SSL_CONF[${BASH_REMATCH[3]},${BASH_REMATCH[5]}]+isset} ]] && r="${SSL_CONF[${BASH_REMATCH[3]},${BASH_REMATCH[5]}]}"    #section variable
      fi
      [ "$r" != ' #' ] && SSL_CONF[$k]="${SSL_CONF[$k]//${BASH_REMATCH[1]}/$r}" #replace our variable with the value
    fi
  done
done
}

Usage: ssl_include <file/dir>

Your two desired values are in ${SSL_CONF[ca_one,certs]} and ${SSL_CONF[ca_two,certs]} (The example script echos these values).

Minor Notes:

bash -version 5.03.3(1), openssl version OpenSSL 1.1.1g
wc -l openssl.cnf 824, grep -c '[^\\]=.*[^\\]\$' openssl.cnf 167, time ./ssl_include.sh openssl.cnf 0m0.854s
If $var isn't assigned by the time it is referenced, openssl will fail. The script doesn't
all config options set before any section are set as SSL_CONF[,name]. This avoids collisions with [default]. [default] is checked first.
any invalid escape sequence should be deleted properly, but there's an openssl bug (see "Bugs" in man config) that prevents me from verifying
can't get .pragma working with openssl, so I cannot verify my implementation. can someone make me a working .pragma config so I can remove this note?
not yet tested on libressl,etc
openssl splices an .import into the file where it was declared. If your import has a [section] defined, you are in that [section] moving forward. It is responsible for many confusing errors

Stack Exchange Network

Reading configuration from an openssl config file

1 Answer 1

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged
bash
configuration
openssl
.

Hot Network Questions

Reading configuration from an openssl config file

1 Answer 1

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged bashconfigurationopenssl.

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
bash
configuration
openssl
.