14

I want to use a here-doc for sed commands and provide the file to be read and the output file.

I've looked at Here-Documents from Advanced Bash Scripting guide but it does not mention anything about regular arguments in using a here-doc. Is it even possible?

I'd like to achieve something like the following:

#!/bin/bash
OUT=/tmp/outfile.txt
IN=/my_in_file.txt

sed $IN << SED_SCRIPT
    s/a/1/g
    s/test/full/g

SED_SCRIPT 
> $OUT;

Any help is really appreciated.

5 Answers 5

22

You can tell GNU sed to read the script from standard input with -f -, -f meaning to read the script from a file, and - meaning standard input as is common with a lot of commands.

sed -f - "$IN" > "$OUT" << SED_SCRIPT
    s/a/1/g
    s/test/full/g
SED_SCRIPT

POSIX sed also supports -f, but the use of - for standard input is not documented. In this case, you could use /dev/stdin on Linux systems (and I seem to recall Solaris has this too, but I cannot confirm that right now)

Using <<-SED_SCRIPT (with the '-' prefix) will allow the closing SED_SCRIPT tag to be indented.

4
  • Thank you, what you said worked, how ever I'd like to point out that the closing token cannot be indented, otherwise it causes an error in bash, I assume this is because it also includes the white-space in the closing one.
    – Tristian
    Aug 16, 2012 at 2:29
  • Note that Apple OS X sed does not support reading a script from standard input. (At least not in that manner or any other documented manner.)
    – danorton
    May 24, 2013 at 17:00
  • @danorton: I have added a solution which does not require support of -f - in sed but it requires process substitution: <( command ). Nov 27, 2014 at 2:09
  • Note: Redirection operator <<- in Bash Ref. Manual : "all leading tab characters are stripped from input lines and the line containing delimiter". If the (closing) delimiter is intended with spaces rather than tabs: bash: warning: here-document at line 21 delimited by end-of-file (wanted 'SED_SCRIPT')\bash: bash-script.sh: line 90: syntax error: unexpected end of file. (Git Bash v2.34.1; GNU bash, version 4.3.48(1)-release (x86_64-suse-linux-gnu)). Indentation of the sed commands works with tabs and spaces. Jun 24, 2022 at 10:13
7

In case sed does not support reading of a script from stdin (using -f -), you can use process substitution (available in bash, zsh, ksh93):

sed "$IN" > "$OUT" -f <( cat << SED_SCRIPT
    s/a/1/g
    s/test/full/g
SED_SCRIPT)

The closing parenthesis ) must follow the end delimiter (SEC_SCRIPT) immediately or after a newline. In the case of process substitution you can also use echo instead of a here document:

sed "$IN" > "$OUT" -f <( echo \
"    s/a/1/g
    s/test/full/g" )
2
  • 1
    On Mac OS X, it didn't like the input file ("$IN" or even testfile.txt) immediately following the sed command itself — it wanted a -e script or -f file. Re-arranging @pabouk 's answer worked for me: sed -f <( cat << ETC ... ETC) "$IN" > "$OUT"
    – Stephen P
    Oct 4, 2016 at 22:21
  • On Max OSX with bash 4.2.45 and standard BSD sed, the closing parentheses had to be on a new line below the closing SED_SCRIPT. Dec 8, 2017 at 20:06
0

Why not use multiple expressions with -e like this:

#!/bin/bash
OUT=/tmp/outfile.txt
IN=/my_in_file.txt

sed 
  -e "s/a/1/g" \
  -e "s/test/full/g" \
  $IN > $OUT

I find that slightly more readable for smaller inline sed scripts. BTW you could also use the in-place editing of sed:

sed -i.org
  -e "s/a/1/g" \
  -e "s/test/full/g" \
  $IN

This would save the original file under ${IN}.org and apply the changes to the input file.

0

There are some great answers on this page. I was looking to be as 'clean' as I could and came up with this version standing on the shoulders of the suggestions above. Comments ofr improvment or fixes are always welcome.

Sampe solution:

function sed_script()
{
    cat << SED_SCRIPT
  s/a/1/g;
  s/test/full/g;
SED_SCRIPT
}

Use as follows:

cat "$IN" | sed -f <(sed_script) > "$OUT" 

Worked on my use case, which had a lot of messy sed commands to run.

2
  • It makes sense to create a function that outputs a sed script if you need to use this sed script in many different places. However, it would probably never be useful to use it without invoking sed. It would therefore make more sense to also include the sed call in the function, so that you get sed_script "$IN" >"$OUT".
    – Kusalananda
    Feb 24, 2023 at 14:21
  • @Kusalananda ... Good point; although the thrus of making a function is primarily for maintainability and readability. Even when ''developing' script, one still needs to be able to quickly (human-)parse the intention and grok how it ought to apply/work-out. imho
    – will
    Feb 28, 2023 at 13:07
-1

Maybe a nicer approach:

OUT=/dev/stdout

IN=my_in_file.txt
sed -s 's/in/out/' < $IN > $OUT
  -or-
sed -s 's/in/out/' > $OUT < $IN

IN=my_in_text
sed -s 's/in/out/' <<< $IN > $OUT
  -or-
sed -s 's/in/out/' > $OUT <<< $IN

:)

3
  • 1
    Interesting, but besides the point. The OP wants to pass sed commands via stdin.
    – muru
    Nov 27, 2014 at 0:17
  • This approach allows both methods, why the downvote?
    – Magnus
    Nov 27, 2014 at 0:21
  • Repeat: "pass sed commands via stdin". You answer doesn't do that by any method.
    – muru
    Nov 27, 2014 at 0:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .