The title says it all, I want to use a here-doc for sed commands and provide the file to be read and the output file.

I've looked at Here-Documents from Advanced Bash Scripting guide but it does not mention anything about regular arguments in using a here-doc. Is it even possible?

I'd like to achieve something like the following:

#!/bin/bash
OUT=/tmp/outfile.txt
IN=/my_in_file.txt

sed $IN << SED_SCRIPT
    s/a/1/g
    s/test/full/g

SED_SCRIPT 
> $OUT;

Any help is really appreciated.

up vote 12 down vote accepted

You can tell GNU sed to read the script from standard input with -f -, -f meaning to read the script from a file, and - meaning standard input as is common with a lot of commands.

sed -f - "$IN" > "$OUT" << SED_SCRIPT
    s/a/1/g
    s/test/full/g
SED_SCRIPT

POSIX sed also supports -f, but the use of - for standard input is not documented. In this case, you could use /dev/stdin on Linux systems (and I seem to recall Solaris has this too, but I cannot confirm that right now)

Using <<-SED_SCRIPT (with the '-' prefix) will allow the closing SED_SCRIPT tag to be indented.

  • Thank you, what you said worked, how ever I'd like to point out that the closing token cannot be indented, otherwise it causes an error in bash, I assume this is because it also includes the white-space in the closing one. – Tristian Aug 16 '12 at 2:29
  • Note that Apple OS X sed does not support reading a script from standard input. (At least not in that manner or any other documented manner.) – danorton May 24 '13 at 17:00
  • @danorton: I have added a solution which does not require support of -f - in sed but it requires process substitution: <( command ). – pabouk Nov 27 '14 at 2:09

In case sed does not support reading of a script from stdin (using -f -), you can use process substitution (available in bash, zsh, ksh93):

sed "$IN" > "$OUT" -f <( cat << SED_SCRIPT
    s/a/1/g
    s/test/full/g
SED_SCRIPT)

The closing parenthesis ) must follow the end delimiter (SEC_SCRIPT) immediately or after a newline. In the case of process substitution you can also use echo instead of a here document:

sed "$IN" > "$OUT" -f <( echo \
"    s/a/1/g
    s/test/full/g" )
  • 1
    On Mac OS X, it didn't like the input file ("$IN" or even testfile.txt) immediately following the sed command itself — it wanted a -e script or -f file. Re-arranging @pabouk 's answer worked for me: sed -f <( cat << ETC ... ETC) "$IN" > "$OUT" – Stephen P Oct 4 '16 at 22:21
  • On Max OSX with bash 4.2.45 and standard BSD sed, the closing parentheses had to be on a new line below the closing SED_SCRIPT. – Alexander Klimetschek Dec 8 '17 at 20:06

Maybe a nicer approach:

OUT=/dev/stdout

IN=my_in_file.txt
sed -s 's/in/out/' < $IN > $OUT
  -or-
sed -s 's/in/out/' > $OUT < $IN

IN=my_in_text
sed -s 's/in/out/' <<< $IN > $OUT
  -or-
sed -s 's/in/out/' > $OUT <<< $IN

:)

  • 1
    Interesting, but besides the point. The OP wants to pass sed commands via stdin. – muru Nov 27 '14 at 0:17
  • This approach allows both methods, why the downvote? – Magnus Nov 27 '14 at 0:21
  • Repeat: "pass sed commands via stdin". You answer doesn't do that by any method. – muru Nov 27 '14 at 0:23

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.