I have to regularly create 100+ text files from templates.
I currently use an overly complicated shell script. I think there is a smarter way to handle this, but I don't know how.

I have a "database" :

# outputfile      template            data1   data2    data3
first.txt         $template_main      $text1  abcd     1234
second.txt        $template_main      $text2  efgh     5678
third.txt         $template_other     $text1  ij       90

And a configuration file :


The templates are text files with placeholders like %%data2%% (the placeholder form can be changed).

Does someone know a tool to automate this better than with a complicated shell script ?

  • Very difficult to judge what is complicated. Posting one of them will help us understand the situation. Maybe try php or perl script? Which have more powerful/easier way to handle string.
    – John Siu
    Dec 27 '12 at 23:45
  • 1
    Possible duplicate
    – l0b0
    Dec 28 '12 at 10:40
  • 1
    Another duplicate
    – l0b0
    Dec 28 '12 at 10:43

There are probably thousands of such template languages and associated software. A popular example is ERB, which is part of vanilla Ruby. After installing Ruby you can start up irb or an editor and simply paste the canonical example to get a feeling for it:

require 'erb'

x = 42
template = ERB.new <<-EOF
  The value of x is: <%= x %>
puts template.result(binding)

You could also consider:

  • the GNU tool called m4 which is a text processor that output the text you want taking as inputs a template with the parts to change. It would surely be simpler than shell script. (it works more or like a C preprocessor with #define macro IIRC).

  • the GNU tool xsltproc that apply a transformation and give you the output. The template is in xml, and xslt is the format of the transformation things to do to your xml so as to output text.

Personnally I have a preference for xslt, but in your case though it wouldn't fit with fields in the form %DATA1% %DATA2%. It needs xml, so you would hate to change your templates.

Thus you should really have a look at m4.

  • As another choice, I have been told Haskell programming language is really really good at transforming streams. I'm only considering this idea because Haskell lovers speak about the wonderous Parsec package, that allows natural parsing of string streams. Much better than xslt, which is already good. I only repeat them, because I'm just learning Haskell, and I have not the single idea how to transform text with it for now.

I think that you would be better off looking at a real scripting language, such as PHP, Perl, or Python, to do something like this for you, especially if you really don't want to get into large scale complex shell scripts.

  • I second this, when I have to do such text munging operations I grab Perl (but Python or Ruby should do equally well). Better use a regular, much used tool that you know well (even if not 100% suited to the job) than a specialized program you rarely use (and which sooner or later will fall sort somehow).
    – vonbrand
    Mar 4 '13 at 13:46

I don't know why you do that, but you have two templates here. One is your 'database' and one is your real template. Both are easy to handle with shtpl. (private project of mine, so not widely in use, but was developed to solve actually those kind of problems)

With shtpl you would do something like this:

Content of file 'configuration':


Content of file 'database' (I assumed, that the delimiter is tab(\t)):

#% . "$CONFFile"
#% if [ -z "$template_main" ] || [ -z "$template_other" ] || \
#%    [ -z "$text1" ]         || [ -z "$text2" ]; then
#%   printf "database could not be generated!\n" > /dev/stderr
#%   exit 1
#% fi
#%# outputfile  template        data1   data2   data3
first.txt       $template_main  $text1  abcd    1234
second.txt      $template_main  $text2  efgh    5678
third.txt       $template_other $text1  ij      90

Content of generatetemplates.sh:


if [ ! -s "$CONFFile" ]; then
 if [ ! -s "$1" ]; then
   printf "CONFfile is not set or empty!\n"
   exit 1
   export CONFFile="$1"

DB="$( bash -c "$( shtpl database )" )"
if [ -z "$DB" ]; then
  printf "Database is empty! Abort.\n"
  exit 2
printf "%s" "$DB" | while read "Out" "In" "data1" "data2" "data3"; do

  data1="$data1" data2="$data2" data3="$data3" \
  bash -c "$( shtpl "$In" )" > "$Out"


Content of main.txt (other.txt is quite the same):

main.txt template

So executing generatetemplates.sh

$ bash generatetemplates.sh "./configuration"

generates us first.txt, second.txt and third.txt.

$ cat first.txt    | $ cat second.txt   | $ cat third.txt
main.txt template  | main.txt template  | other.txt template
whatever           | blah               | whatever
abcd               | efgh               | ij
1234               | 5678               | 90

Little explanation: In generatetemplates.sh is first the needed 'database' generated from your configuration-file. And secondly for every tupel in the database finally the corresponding Out-file from your In-template.

Note: An empty data[123] troubles read. So it is not possible with this approach.

So, hope this is simple enough for your needs.

Have fun!


I recently published an open source project that accomplishs just that using a jinja-like template syntax. It's called cookie. Here's a demo:

cookie demo


Check out tcat.sh. Say you have a template file:

hello ${name}


$ export name=world # or load and export from a properties file.
$ ./tcat.sh template-file


hello world

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.