I have to regularly create 100+ text files from templates.
I currently use an overly complicated shell script. I think there is a smarter way to handle this, but I don't know how.

I have a "database" :

# outputfile      template            data1   data2    data3
first.txt         $template_main      $text1  abcd     1234
second.txt        $template_main      $text2  efgh     5678
third.txt         $template_other     $text1  ij       90

And a configuration file :


The templates are text files with placeholders like %%data2%% (the placeholder form can be changed).

Does someone know a tool to automate this better than with a complicated shell script ?

  • Very difficult to judge what is complicated. Posting one of them will help us understand the situation. Maybe try php or perl script? Which have more powerful/easier way to handle string.
    – John Siu
    Commented Dec 27, 2012 at 23:45
  • 1
    Possible duplicate
    – l0b0
    Commented Dec 28, 2012 at 10:40
  • 1
    Another duplicate
    – l0b0
    Commented Dec 28, 2012 at 10:43

7 Answers 7


You could also consider:

  • the GNU tool called m4 which is a text processor that output the text you want taking as inputs a template with the parts to change. It would surely be simpler than shell script. (it works more or like a C preprocessor with #define macro IIRC).

  • the GNU tool xsltproc that apply a transformation and give you the output. The template is in xml, and xslt is the format of the transformation things to do to your xml so as to output text.

Personnally I have a preference for xslt, but in your case though it wouldn't fit with fields in the form %DATA1% %DATA2%. It needs xml, so you would hate to change your templates.

Thus you should really have a look at m4.

  • As another choice, I have been told Haskell programming language is really really good at transforming streams. I'm only considering this idea because Haskell lovers speak about the wonderous Parsec package, that allows natural parsing of string streams. Much better than xslt, which is already good. I only repeat them, because I'm just learning Haskell, and I have not the single idea how to transform text with it for now.

There are probably thousands of such template languages and associated software. A popular example is ERB, which is part of vanilla Ruby. After installing Ruby you can start up irb or an editor and simply paste the canonical example to get a feeling for it:

require 'erb'

x = 42
template = ERB.new <<-EOF
  The value of x is: <%= x %>
puts template.result(binding)

I don't know why you do that, but you have two templates here. One is your 'database' and one is your real template. Both are easy to handle with shtpl. (private project of mine, so not widely in use, but was developed to solve actually those kind of problems)

With shtpl you would do something like this:

Content of file 'configuration':


Content of file 'database' (I assumed, that the delimiter is tab(\t)):

#% . "$CONFFile"
#% if [ -z "$template_main" ] || [ -z "$template_other" ] || \
#%    [ -z "$text1" ]         || [ -z "$text2" ]; then
#%   printf "database could not be generated!\n" > /dev/stderr
#%   exit 1
#% fi
#%# outputfile  template        data1   data2   data3
first.txt       $template_main  $text1  abcd    1234
second.txt      $template_main  $text2  efgh    5678
third.txt       $template_other $text1  ij      90

Content of generatetemplates.sh:


if [ ! -s "$CONFFile" ]; then
 if [ ! -s "$1" ]; then
   printf "CONFfile is not set or empty!\n"
   exit 1
   export CONFFile="$1"

DB="$( bash -c "$( shtpl database )" )"
if [ -z "$DB" ]; then
  printf "Database is empty! Abort.\n"
  exit 2
printf "%s" "$DB" | while read "Out" "In" "data1" "data2" "data3"; do

  data1="$data1" data2="$data2" data3="$data3" \
  bash -c "$( shtpl "$In" )" > "$Out"


Content of main.txt (other.txt is quite the same):

main.txt template

So executing generatetemplates.sh

$ bash generatetemplates.sh "./configuration"

generates us first.txt, second.txt and third.txt.

$ cat first.txt    | $ cat second.txt   | $ cat third.txt
main.txt template  | main.txt template  | other.txt template
whatever           | blah               | whatever
abcd               | efgh               | ij
1234               | 5678               | 90

Little explanation: In generatetemplates.sh is first the needed 'database' generated from your configuration-file. And secondly for every tupel in the database finally the corresponding Out-file from your In-template.

Note: An empty data[123] troubles read. So it is not possible with this approach.

So, hope this is simple enough for your needs.

Have fun!


I think that you would be better off looking at a real scripting language, such as PHP, Perl, or Python, to do something like this for you, especially if you really don't want to get into large scale complex shell scripts.

  • I second this, when I have to do such text munging operations I grab Perl (but Python or Ruby should do equally well). Better use a regular, much used tool that you know well (even if not 100% suited to the job) than a specialized program you rarely use (and which sooner or later will fall sort somehow).
    – vonbrand
    Commented Mar 4, 2013 at 13:46

I recently published an open source project that accomplishs just that using a jinja-like template syntax. It's called cookie. Here's a demo:

cookie demo


Check out tcat.sh. Say you have a template file:

hello ${name}


$ export name=world # or load and export from a properties file.
$ ./tcat.sh template-file


hello world

In bash you can simply use parameter expansion to render the template:


The @P operator expands a variable using prompting substitution rules which substitute \h with the hostname, \d with the date and so on (see PROMPTING in man bash). But you probably don't need all this prompting expansion. In such case just replace \ with \\ in the template before rendering again using parameter expansion:


Thus prompting expansion will be disabled, but standard expansion will still take place. Full script:

# reading config:
source config.txt

# reading the template:
IFS= read -rd '' template < template.txt

# substituting '\' with '\\':

# rendering the template:

# printing rendered text:
printf "%s\n" "$rendered"

NOTE: For security reasons you might not want to allow $(...) expansion in the template, which allows running shell commands. In this case you might use standard utility envsubst:

envsubst < template.txt

Also, you might not want to allow all the shell variables expansion, in this case you can specify list of variables which will be expanded:

envsubst '$template_main$template_other$text1$text2' < template.txt

See also mako templates

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .