0

I have a really large regex pattern string that I am using to delete INFO messages from a log file. When I use this all in one line like the following it works fine:

sed -r '/([12][[:digit:]]{3}-(0[1-9]|1[0-2])-(0[1-9]|[12][[:digit:]]|3[01]) [[:digit:]]{2}:[[:digit:]]{2}:[[:digit:]]{2},[[:digit:]]{3} INFO)/d' >> $2

But as you see this is difficult to read so I tried putting it in variable and breaking it into multiple lines. So I did the following:

regex='/([12][[:digit:]]{3}-(0[1-9]|1[0-2])-(0[1-9]|[12][[:digit:]]|3[01])
 [[:digit:]]{2}:[[:digit:]]{2}:[[:digit:]]{2},[[:digit:]]{3} INFO)/d'

cat copy.out | sed -r $regex >> tmp_log.txt

But I keep getting the error from sed:

sed: -e expression #1, char 67: unterminated address regex

When I use echo the regex string is printed without any line breaks.

echo $regex
/([12][[:digit:]]{3}-(0[1-9]|1[0-2])-(0[1-9]|[12][[:digit:]]|3[01]) [[:digit:]]{2}:[[:digit:]]{2}:[[:digit:]]{2},[[:digit:]]{3} INFO)/d

It seems I am missing something extremely basic here, what am I doing wrong ?

2
  • 3
    Always quote your variables. use "$regex" not $regex. I don't know if you can do that in a variable but you can split that long assignment into 2 partrs. eg '$regex1" and $regex2" then use $regex1$regex2". Maybe that is acceptable to you...
    – Jetchisel
    Commented Jan 18, 2020 at 11:55
  • 1
    The echo gets multiple args (because there is no quoting) and writes them out on one line. But sed gets two args, with half the RE in each. Personally, I would define D2=[[:digit:]]{2} etc and assemble the regex variable out of repeated components. Commented Jan 18, 2020 at 13:03

1 Answer 1

5

By using double quotes around your sed expression string, you can use ordinary line-continuation to break it into multiple lines:

expression="/([12][[:digit:]]{3}-(0[1-9]|1[0-2])-(0[1-9]|[12][[:digit:]]|3[01]) \
[[:digit:]]{2}:[[:digit:]]{2}:[[:digit:]]{2},[[:digit:]]{3} INFO)/d"

There must be nothing but a newline after the \ at the end of the first line, and the second line must not start with any extra characters. I moved the space in the string up to the first line to make it appear deliberate and not as an accidental indentation.

Alternatively, you may use the concatenation of two single-quoted strings:

expression='/([12][[:digit:]]{3}-(0[1-9]|1[0-2])-(0[1-9]|[12][[:digit:]]|3[01]) '\
'[[:digit:]]{2}:[[:digit:]]{2}:[[:digit:]]{2},[[:digit:]]{3} INFO)/d'

This will assign a single line string to the variable expression.

You would then use this as

sed -E -e "$expression"

Here I'm using -E here in place of -r as -E is more commonly supported (for enabling extended regular expressions in sed), and using -e to tell sed that the next argument is the sed expression to apply to the input. The sed expression itself needs to be double quoted as we don't want the shell to perform word splitting and filename globbing on its value.

6
  • I didn't quite get your use of -E in favour of -r. sed -r "$regex" works fine after I used double quotes as directed by you.
    – ng.newbie
    Commented Jan 18, 2020 at 13:18
  • @ng.newbie It's for portability. -r is GNU-sed only while -E works in [newer versions of] GNU sed and other seds, e.g. OSX sed.
    – Ed Morton
    Commented Jan 23, 2020 at 4:25
  • @Kusalananda - the assignment should still be in single quotes since you don't need double quotes and you wouldn't want the shell accidentally interpreting anything in that string. It was just the backslash that was missing in the assignment and then the double quotes where it's being passed to sed.
    – Ed Morton
    Commented Jan 23, 2020 at 4:29
  • 2
    @EdMorton No, sorry, on 2nd thought that won't work. The \ and the newline would end up as part of the variable's value if you used single quotes. If you wanted to use single quotes, you would need e='something'\<newline>'something' (two concatenated strings with a literal newline in-between).
    – Kusalananda
    Commented Jan 23, 2020 at 13:46
  • 1
    Ah, I see, yes you're right, it should be e='something'\<newline>'something'
    – Ed Morton
    Commented Jan 23, 2020 at 15:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .