I'm writing a little shell script to generate a directory listing. To make the output easily customizable, the script just builds a HTML-table and than should replace a specific token in a template file with that table and write it to stdout.

Template file:

<!doctype html>
    <head><title>Directory Listing</title></head>

{{LISTING}} should be replaced with the listing (which is stored in a shell variable).

The problem here is, that a simple sed 's/{{LISTING}}/$listing/' would be limited to the maximum argument length which doesn't work with long listings.

How can I replace a pattern in a file with a long string using POSIX utilities?

To add more clarification: My current solution is like that:

awk '{ gsub(A, B); print; }' A="{{LISTING}}" B="$listing" < $template

$listing gets expanded as awk is invoked, but this can exceed argument length limits, since $listing can be really long. For example, when generating a listing of /usr/lib, I get an error:

/bin/awk: Argument list too long

I'm thinking whether there is a solution to replace a pattern in a textfile with stdin.

  • @wlhim is there any issue with using awk and bash ? Commented Oct 12, 2013 at 18:48
  • I don't have a problem with them, I just couldn't come up with a solution without specifying the replacement text (which is in $listing) as an argument, since $listing can be too long and I get an error: "/bin/awk: Argument list too long". Commented Oct 12, 2013 at 18:59
  • I have updated the ans, please once check and let me know.. Commented Oct 12, 2013 at 19:00
  • bash: /bin/sed: Argument list too long Commented Oct 12, 2013 at 19:14
  • once check this link stackoverflow.com/questions/11289551/… Commented Oct 12, 2013 at 19:47

2 Answers 2


If there are 3 files:

  • old: the original data
  • replace: the large batch of text you want to insert
  • newfile: where the replacements will be saved

You should be able to do the following with a standard shell

for line in `cat old`; do
  if [ $line == "text to match" ]; then
    cat replace >> newfile
    echo $line >> newfile
  • There are two caveats with for in and while read: 1. Trailing characters included in IFS are stripped (for example trailing spaces) 2. The token has to be on its own line (not in the middle of text; this is only a minor issue for me) Commented Oct 12, 2013 at 21:15

How about using xargs since it knows about argument size limits:

cp "$template_file" "$output_file" || exit 1
echo "$listing" \
| xargs -I arg sed -i 's/{{LISTING}}/arg {{LISTING}}/' "$output_file" \
| sed -i 's/ {{LISTING}}//' "$output_file"

The caveat here is that xargs strips trailing new lines in file names.

  • The template file should be specified with a command line option: dir-listing.sh -t TEMPLATE Commented Oct 13, 2013 at 12:55
  • @wlhlm Edited answer. See if this helps.
    – Joseph R.
    Commented Oct 13, 2013 at 16:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .