Tool to create text files from a template

Question

I have to regularly create 100+ text files from templates.
I currently use an overly complicated shell script. I think there is a smarter way to handle this, but I don't know how.

I have a "database" :

# outputfile      template            data1   data2    data3
first.txt         $template_main      $text1  abcd     1234
second.txt        $template_main      $text2  efgh     5678
third.txt         $template_other     $text1  ij       90

And a configuration file :

template_main=main.txt
template_other=other.txt
text1=whatever
text2=blah

The templates are text files with placeholders like %%data2%% (the placeholder form can be changed).

Does someone know a tool to automate this better than with a complicated shell script ?

Answer 1

You could also consider:

• the GNU tool called m4 which is a text processor that output the text you want taking as inputs a template with the parts to change. It would surely be simpler than shell script. (it works more or like a C preprocessor with #define macro IIRC).

• the GNU tool xsltproc that apply a transformation and give you the output. The template is in xml, and xslt is the format of the transformation things to do to your xml so as to output text.

Personnally I have a preference for xslt, but in your case though it wouldn't fit with fields in the form %DATA1% %DATA2%. It needs xml, so you would hate to change your templates.

Thus you should really have a look at m4.

• As another choice, I have been told Haskell programming language is really really good at transforming streams. I'm only considering this idea because Haskell lovers speak about the wonderous Parsec package, that allows natural parsing of string streams. Much better than xslt, which is already good. I only repeat them, because I'm just learning Haskell, and I have not the single idea how to transform text with it for now.

Answer 2

There are probably thousands of such template languages and associated software. A popular example is ERB, which is part of vanilla Ruby. After installing Ruby you can start up irb or an editor and simply paste the canonical example to get a feeling for it:

require 'erb'

x = 42
template = ERB.new <<-EOF
  The value of x is: <%= x %>
EOF
puts template.result(binding)

Answer 3

I don't know why you do that, but you have two templates here. One is your 'database' and one is your real template. Both are easy to handle with shtpl. (private project of mine, so not widely in use, but was developed to solve actually those kind of problems)

With shtpl you would do something like this:

Content of file 'configuration':

template_main=main.txt
template_other=other.txt
text1=whatever
text2=blah

Content of file 'database' (I assumed, that the delimiter is tab(\t)):

#% . "$CONFFile"
#% if [ -z "$template_main" ] || [ -z "$template_other" ] || \
#%    [ -z "$text1" ]         || [ -z "$text2" ]; then
#%   printf "database could not be generated!\n" > /dev/stderr
#%   exit 1
#% fi
#%# outputfile  template        data1   data2   data3
first.txt       $template_main  $text1  abcd    1234
second.txt      $template_main  $text2  efgh    5678
third.txt       $template_other $text1  ij      90

Content of generatetemplates.sh:

#!/bin/bash

if [ ! -s "$CONFFile" ]; then
 if [ ! -s "$1" ]; then
   printf "CONFfile is not set or empty!\n"
   exit 1
 else
   export CONFFile="$1"
 fi
fi

DB="$( bash -c "$( shtpl database )" )"
if [ -z "$DB" ]; then
  printf "Database is empty! Abort.\n"
  exit 2
fi
IFS=$'\t'
printf "%s" "$DB" | while read "Out" "In" "data1" "data2" "data3"; do

  data1="$data1" data2="$data2" data3="$data3" \
  bash -c "$( shtpl "$In" )" > "$Out"

done

Content of main.txt (other.txt is quite the same):

main.txt template
$data1
$data2
$data3

So executing generatetemplates.sh

$ bash generatetemplates.sh "./configuration"

generates us first.txt, second.txt and third.txt.

$ cat first.txt    | $ cat second.txt   | $ cat third.txt
main.txt template  | main.txt template  | other.txt template
whatever           | blah               | whatever
abcd               | efgh               | ij
1234               | 5678               | 90

Little explanation: In generatetemplates.sh is first the needed 'database' generated from your configuration-file. And secondly for every tupel in the database finally the corresponding Out-file from your In-template.

Note: An empty data[123] troubles read. So it is not possible with this approach.

So, hope this is simple enough for your needs.

Have fun!

Answer 4

I think that you would be better off looking at a real scripting language, such as PHP, Perl, or Python, to do something like this for you, especially if you really don't want to get into large scale complex shell scripts.

Answer 5

I recently published an open source project that accomplishs just that using a jinja-like template syntax. It's called cookie. Here's a demo:

Answer 6

Check out tcat.sh. Say you have a template file:

hello ${name}

then

$ export name=world # or load and export from a properties file.
$ ./tcat.sh template-file

Output:

hello world

Answer 7

In bash you can simply use parameter expansion to render the template:

rendered=${template@P}

The @P operator expands a variable using prompting substitution rules which substitute \h with the hostname, \d with the date and so on (see PROMPTING in man bash). But you probably don't need all this prompting expansion. In such case just replace \ with \\ in the template before rendering again using parameter expansion:

template=${template//\\/\\\\}

Thus prompting expansion will be disabled, but standard expansion will still take place. Full script:

# reading config:
source config.txt

# reading the template:
IFS= read -rd '' template < template.txt

# substituting '\' with '\\':
template=${template//\\/\\\\}

# rendering the template:
rendered=${template@P}

# printing rendered text:
printf "%s\n" "$rendered"

NOTE: For security reasons you might not want to allow $(...) expansion in the template, which allows running shell commands. In this case you might use standard utility envsubst:

envsubst < template.txt

Also, you might not want to allow all the shell variables expansion, in this case you can specify list of variables which will be expanded:

envsubst '$template_main$template_other$text1$text2' < template.txt

See also mako templates