5

Saw here a way to use sed to get text between two other strings in a line, like:

sed 's/.*starting_text\(.*\)ending_text.*/\1/'

but I'd like a simple command (like tr, but for string extraction) that just took two strings and would trim everything before the first string or after the second string, e.g.

grep something some_file | between message\"\:\" " with"

and would handle escaping characters.

  • 1
    Have you looked at this question? – Joseph R. Sep 27 '13 at 17:46
  • Just changed the question title to be clearer about the intent. I was looking for a simple command like tr that takes two strings, but that extracts the text between them. I'm going to integrate a modified version of the solution in that link into my answer though. Cool stuff! Thanks for the link! – Gary S. Weaver Sep 27 '13 at 18:04
  • 1
    This can be done with simple pattern matching in bash itself . man bash (longer text without images). – ott-- Sep 27 '13 at 18:13
  • @ott Thanks! A lot of info here on bash string extraction also, but not sure how to make a generic function like I'm talking about that will escape characters properly. If you could assist with a similar answer that would be awesome. – Gary S. Weaver Sep 27 '13 at 18:36
8

If the delimiters may appear several times per lines, you could use perl instead like:

between() {
  perl -Tlne 'BEGIN{$b=shift;$e=shift}
             print for /\Q$b\E(.*?)\Q$e\E/g' "$@"
}

And then for example:

$ echo "[b]test[e] foo [b]bar[e]" | between '[b]' '[e]'
test
bar

You can also use it as:

between BEG END file1 file2...
  • You can share an answer at the same time as asking to both share with others and get input back, so that is why I posted the Q&A- and because I wasn't sure if my solution was escaping characters to be fully safe from all types of input. Thanks for the Perl solution! Does that escape all characters in the two input strings properly? – Gary S. Weaver Sep 27 '13 at 19:08
  • 2
    @GaryS.Weaver, yes that's what \Q is for. – Stéphane Chazelas Sep 27 '13 at 19:24
  • Cool! Thanks for your help. I'm using your solution, but will leave the sed and grep solutions in case anyone's having perl trouble, I guess. – Gary S. Weaver Sep 27 '13 at 19:32
2

Doing this in sed generically would require escaping characters in the regexp used to find the substring which I found here (note: more info here if you run into problems).

Then, I found how to pipe into a function here.

Putting all that together into a function I can use in my .bashrc, looks like (though I don't have to set the a and b vars, but it makes it easier to read):

between(){
  a=$(printf '%s\n' "$1"|sed 's![\*.^$/[]!\\&!g')
  b=$(printf '%s\n' "$2"|sed 's![\*.^$/[]!\\&!g')
  sed "s/.*$a\(.*\)$b.*/\1/"
}

as Joseph R. mentioned, this answer shows how to use grep -oP to do similar. To escape Perl-compatible regexp I found this, so maybe the following would work also:

between(){
  a=$(printf '%s\n' "$1"|sed 's![]\*.^+?(){|$[]!\\&!g')
  b=$(printf '%s\n' "$2"|sed 's![]\*.^+?(){|$[]!\\&!g')
  grep -oP "(?=$a).*?(?=$b)"
}
  • The characters that need to be escaped are \^$[.* and the s command delimiter like / for BREs. Using a while read loop makes not sense here. – Stéphane Chazelas Sep 27 '13 at 18:56
  • @StephaneChazelas Thanks! 1. Could you clarify which part of the answer you mean? I'm escaping : in the first, because I'm in POSIX (I'll clarify that also), but for the second it is a PCRE so I thought I needed to escape .^$*+?()[{\|? For the second part of your comment, I'm piping into it which is the reason for while read. I'll clarify that in the answer. – Gary S. Weaver Sep 27 '13 at 19:03
  • It gets confusing to figure out how to be sure that the 2 input strings are fully escaped so they won't cause problems if the wrong set of characters were provided as input, so that is the primary reason I shared my answer to try to be sure that what I was doing was going to work or to see if there is a better solution. – Gary S. Weaver Sep 27 '13 at 19:10
  • : has only a significance inside [...] inside [...] like in [[:blank:]]. Once you've escaped all the [, there's no need to escape : or ]. – Stéphane Chazelas Sep 27 '13 at 19:13
  • 1
    Yes, see my suggested edit for an improvement. You should not use echo or function or a while read loop. – Stéphane Chazelas Sep 27 '13 at 19:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.