I'm writing a little shell script to generate a directory listing. To make the output easily customizable, the script just builds a HTML-table and than should replace a specific token in a template file with that table and write it to stdout.

Template file:

<!doctype html>
    <head><title>Directory Listing</title></head>

{{LISTING}} should be replaced with the listing (which is stored in a shell variable).

The problem here is, that a simple sed 's/{{LISTING}}/$listing/' would be limited to the maximum argument length which doesn't work with long listings.

How can I replace a pattern in a file with a long string using POSIX utilities?

To add more clarification: My current solution is like that:

awk '{ gsub(A, B); print; }' A="{{LISTING}}" B="$listing" < $template

$listing gets expanded as awk is invoked, but this can exceed argument length limits, since $listing can be really long. For example, when generating a listing of /usr/lib, I get an error:

/bin/awk: Argument list too long

I'm thinking whether there is a solution to replace a pattern in a textfile with stdin.

  • @wlhim is there any issue with using awk and bash ? Oct 12, 2013 at 18:48
  • I don't have a problem with them, I just couldn't come up with a solution without specifying the replacement text (which is in $listing) as an argument, since $listing can be too long and I get an error: "/bin/awk: Argument list too long". Oct 12, 2013 at 18:59
  • I have updated the ans, please once check and let me know.. Oct 12, 2013 at 19:00
  • bash: /bin/sed: Argument list too long Oct 12, 2013 at 19:14
  • once check this link stackoverflow.com/questions/11289551/… Oct 12, 2013 at 19:47

2 Answers 2


If there are 3 files:

  • old: the original data
  • replace: the large batch of text you want to insert
  • newfile: where the replacements will be saved

You should be able to do the following with a standard shell

for line in `cat old`; do
  if [ $line == "text to match" ]; then
    cat replace >> newfile
    echo $line >> newfile
  • There are two caveats with for in and while read: 1. Trailing characters included in IFS are stripped (for example trailing spaces) 2. The token has to be on its own line (not in the middle of text; this is only a minor issue for me) Oct 12, 2013 at 21:15

How about using xargs since it knows about argument size limits:

cp "$template_file" "$output_file" || exit 1
echo "$listing" \
| xargs -I arg sed -i 's/{{LISTING}}/arg {{LISTING}}/' "$output_file" \
| sed -i 's/ {{LISTING}}//' "$output_file"

The caveat here is that xargs strips trailing new lines in file names.

  • The template file should be specified with a command line option: dir-listing.sh -t TEMPLATE Oct 13, 2013 at 12:55
  • @wlhlm Edited answer. See if this helps.
    – Joseph R.
    Oct 13, 2013 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.