Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm writing a little shell script to generate a directory listing. To make the output easily customizable, the script just builds a HTML-table and than should replace a specific token in a template file with that table and write it to stdout.

Template file:

<!doctype html>
    <head><title>Directory Listing</title></head>

{{LISTING}} should be replaced with the listing (which is stored in a shell variable).

The problem here is, that a simple sed 's/{{LISTING}}/$listing/' would be limited to the maximum argument length which doesn't work with long listings.

How can I replace a pattern in a file with a long string using POSIX utilities?

To add more clarification: My current solution is like that:

awk '{ gsub(A, B); print; }' A="{{LISTING}}" B="$listing" < $template

$listing gets expanded as awk is invoked, but this can exceed argument length limits, since $listing can be really long. For example, when generating a listing of /usr/lib, I get an error:

/bin/awk: Argument list too long

I'm thinking whether there is a solution to replace a pattern in a textfile with stdin.

share|improve this question
@wlhim is there any issue with using awk and bash ? – Rahul Patil Oct 12 '13 at 18:48
I don't have a problem with them, I just couldn't come up with a solution without specifying the replacement text (which is in $listing) as an argument, since $listing can be too long and I get an error: "/bin/awk: Argument list too long". – Wilhelm Schuster Oct 12 '13 at 18:59
I have updated the ans, please once check and let me know.. – Rahul Patil Oct 12 '13 at 19:00
bash: /bin/sed: Argument list too long – Rahul Patil Oct 12 '13 at 19:14
once check this link stackoverflow.com/questions/11289551/… – Rahul Patil Oct 12 '13 at 19:47

If there are 3 files:

  • old: the original data
  • replace: the large batch of text you want to insert
  • newfile: where the replacements will be saved

You should be able to do the following with a standard shell

for line in `cat old`; do
  if [ $line == "text to match" ]; then
    cat replace >> newfile
    echo $line >> newfile
share|improve this answer
There are two caveats with for in and while read: 1. Trailing characters included in IFS are stripped (for example trailing spaces) 2. The token has to be on its own line (not in the middle of text; this is only a minor issue for me) – Wilhelm Schuster Oct 12 '13 at 21:15

How about using xargs since it knows about argument size limits:

cp "$template_file" "$output_file" || exit 1
echo "$listing" \
| xargs -I arg sed -i 's/{{LISTING}}/arg {{LISTING}}/' "$output_file" \
| sed -i 's/ {{LISTING}}//' "$output_file"

The caveat here is that xargs strips trailing new lines in file names.

share|improve this answer
The template file should be specified with a command line option: dir-listing.sh -t TEMPLATE – Wilhelm Schuster Oct 13 '13 at 12:55
@wlhlm Edited answer. See if this helps. – Joseph R. Oct 13 '13 at 16:49

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.