1

So I am loading in a properties file to define some variables. In this example imagine it is a string mark!slash\ending\ called INPUT.

INPUT is declared by loading in the properties file using:

. ./properties

where

INPUT=mark!slash\ending\

I need the final output to be a script run through SED of the form s!@output@!$INPUT!g such that when executed, @output@ is replaced in another file with the input from the properties file defined by the user (the variable, in this case, is INPUT). We are using ! as the delimiter in SED so this character will need to be escaped (as well as any \) before added to the SED template file.

If I try

echo SAFE_INPUT=$(printf '%s\n' 'mark!slash\ending\' | sed 's:[\!]:\\&:g')

I get the expected output

mark\!slash\\ending\\

If, however, I try

SAFE_INPUT=$(printf '%s\n' "${INPUT}" | sed 's:[\!]:\\&:g')

I get

mark\!slashending

I am assuming it is something to do with my declaration of string vs a variable but I am at a loss as to the difference and how to solve the issue right now.

Edit: On further testing INPUT as read using

. ./properties

is printed as

mark!slashending

So this is not an issue with SED but with reading in the property file.

  • 1
    how you are declaring INPUT? – msp9011 Feb 1 at 11:02
  • We really can't help if you don't show us how you assign a value to INPUT. The command you show works perfectly well if you just use INPUT='mark!slash\ending\'. – terdon Feb 1 at 11:31
  • Edited to show how INPUT is loaded in. – Gathris Feb 1 at 11:40
2

The problem is when you set variable INPUT in file properties like shown in the question

INPUT=mark!slash\ending\

and source it as a shell script.

The first \e is replaced by the shell with e and the \ at the end of the line will either be removed if there is some whitespace following it, otherwise it denotes that the line is continied in the next line. So the value of INPUT will already be modified.

You either have to define your input with quotes

INPUT='mark!slash\ending\'

like in Cornholio's answer or you have to parse your properties file in a different way.

With the following command you will get variable INPUT (and others if there are more lines in properties) set to the correct value.

eval $(sed -e 's:[\!]:\\&:g' properties)

This will let the shell evaluate the output of the sed command which replaces \ and ! and in the contents of properties. The result will be variables with the original names INPUT etc. This will work if your properties file contains one or more variable assignments. It may give strange result with other shell script code.

Note: If your input file contains other special characters, the peprocessing would be more difficult. So using eval this way without additional checking may be fragile or dangerous.

With your example the variable INPUT will have the intended value mark!slash\ending\

After this you can do

SAFE_INPUT=$(printf '%s\n' "${INPUT}" | sed 's:[\!]:\\&:g')

if you need variable SAVE_INPUT to contain mark\!slash\\ending\\.

A better way would be to read the input line by line with while read -r line and check and process the line to separate variable name and value.

  • Preprocessing the script to run it through eval seems a bit fickly. At least you'd need to add backslashes in front of a bunch of other characters too, not just ! and ` ($` would be an obvious one). Better to just quote the string in the original file. – ilkkachu Feb 1 at 14:40
  • im not sure quoting the string works, thats what im trying now, with apache configs. things would actually be really messed up if you have to escapce every wierd character. your readfile would then look like a big regex escaped mess. So it almost seems counterintuitive that it would be possible by readfile, however there must be a way. I need the readfile, when interepted by sed, to become the value of the variable, as opposed to literal $var. e.g. it needs to eval, but i dont like restorting to an eval() solution either.Is that really our only option here? – Brian Thomas Feb 9 at 5:17
1

If I do the following, with the value of INPUT quoted, it works:

> INPUT='mark!slash\ending\'
> SAFE_INPUT=$(printf '%s\n' "${INPUT}" | sed 's:[\!]:\\&:g')
> echo $SAFE_INPUT
mark\!slash\\ending\\

Note, it's an exact copy of your script, with INPUT= added by me.

  • Hey, this is an answer that actually works, no need to even think about posting it as a comment instead of a proper answer! – ilkkachu Feb 1 at 14:34
  • This may work, but how would you then, use vars in your input string, which what im trying to do. e.g. my input string is a file path, thats evaluated by sed read, as opposed to a var of data. once sed read reads the text, i want the var evaluqated when read parses the read. Is this possible? – Brian Thomas Feb 9 at 5:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.