I have a text file and want to extract only the text beginning and ending with a certain strings using sed.

For example, in the line:

string>![TEST[Extract this string]>/string>

I want to get

Extract this string 

How would you implement this with sed? Basically I want to get text that start with the expression "string>![TEST[" and end with the expression "]>/string>".

  • It's ppossible with sed,awk , perl and so on, It's related to your job! – PersianGulf Mar 7 '15 at 19:27
  • 1
    this look like XML CDATA part. are you sure you dont want to use xml/xsl tools ? – Archemar Mar 7 '15 at 21:59
  • You really need sed? – Braiam Mar 8 '15 at 2:29
  • Why not use grep (or egrep) for this type of situation? – mdpc Mar 8 '15 at 3:14
sed -e 's/string>!\[TEST\[\(.*\)]>\/string>/\1/' file


sed -e 's|string>!\[TEST\[\(.*\)]>/string>|\1|' file


Extract this string

You need to tell the string not only what to match, but what to save as well:

sed -ne 's@string>!\[TEST\[\([^]]*\)\]>/string>@\1@gp'

The s command in sed takes two arguments: a regular expression and a replacement string. Typically, the / delimiter is used to separate the two, but you can use any character, in this case @. There are some special characters in the regular expressions, like [, ]. These would need to be quoted with \ if you want the real character, e.g. string>!\[. The \([^]]*\) captures everything between the square brackets. And the \1 replaces the who string with what matched the regular expression. At the end is @gp, which tells send to match multiple times on the line (g) and print the replaced line (after we tell sed to not print lines with the -n option.

  • To make a pattern shorter sed -ne 's@\(string>\)!\[TEST\[\([^]]*\)\]>/\1@\2@gp' – Costas Mar 7 '15 at 19:31
  • It captures everything which is not a right-side square bracket. – mikeserv Mar 7 '15 at 19:31

A simple approach with Awk:

awk -F'[][]' '{print $3}' file
  • Correct if there is the one string occurance in line. – Costas Mar 7 '15 at 19:32
  • @Costas Like I said, a simple approach (and based on the total amount of information provided by the OP)... – jasonwryan Mar 7 '15 at 19:34
  • 1
    So to be fully correct to OP it shoul be awk -F'string>\\!\\[TEST\\[|\\]' '$0 ~ FS{print $2}' – Costas Mar 7 '15 at 20:14
  • @Costas Sure, but that no longer qualifies as "simple". :) – jasonwryan Mar 7 '15 at 20:16
  • 1
    @mikeserv There was no need to defend your own: your familiarity with sed is truly impressive and my comment was a mixture of jest and admiration... – jasonwryan Mar 7 '15 at 22:03
sed '/\n/P;//D;y|]|\n|
    y|\n[|[\n|;D' <<\IN
    string>![TEST[][]Extract[ ]this[ ]string[][]>/string>

It maybe that you can specify that the square brackets are acceptable delimiters here, but, if so, it seems strange that the end delimiters would be so elaborate in that case. And anyway, as the question only states that you need to get text from between string>![TEST[ and ]>/string> and so that's what this tries to do - though it does fail if text should span newline boundaries.I

Anyway, it works by:

  1. y|]|\n| - It first translates all occurrences of ] on a line for a \newline.
  2. s|\n>/string>|]| - It next replaces the first occurring \newline which is followed immediately by your right end delimiter with ] (which makes it the only possible ] on a line at that time).
  3. y|[]\n|\n[]| - If the last substitution was successful that one ] is translated to a [ while all \newlines are translated back to ] and all [ are simultaneously translated to \newlines - the three character types are shifted, basically.
  4. s|string>!\nTEST\n\(.*\[\)|[\1| - If the left end delimiter is found preceding a [ at that time then it must be that both ends of the first occurrence of text have been found. That match is substituted for [.
  5. y|\n[|[\n| - And so in the last translation if there are any [ on a line at all they will become newlines and all newlines will become [.

At this point everything up to the first occurring newline (or the entire line if there are none at all) is Deleted. If anything remains it is sent to the top of the script. If the previous iteration resulted in two \newlines in pattern space - both ends of your delimited text then it is Printed up to the first occurring \newline. Else the pattern space already tested is cleared and the cycle continues.

And so the above example prints:

][]Extract[ ]this[ ]string[][

...and it will print each on a separate line as many similarly delimited strings as can be fully left and right delimited per line or nothing at all.

  • In the case cn be more simple sed -e 's#[^!]*string>!\[TEST\[\|\]>/string>[^!]*##g'? – Costas Mar 7 '15 at 20:34
  • @Costas - no. That doesn't work if there is more than one per line - or if the left-end delimiter occurs within text. – mikeserv Mar 7 '15 at 20:38
  • Agree. What you say re's#string>!\[TEST\[\|\]>/string>$#^#g;s/[^^]*^\([^^]*\)^[^^]*/\1 /g' If you do not sure in ^ free to change it by some more exotic. – Costas Mar 7 '15 at 20:45
  • @Costas - exotic doesn't work for unknown strings - it doesn't matter how rare it is - if it is a possibility then it is a bug. You have to sanitize it first, or use delimiters that cannot otherwise occur. – mikeserv Mar 7 '15 at 20:58

Through GNU grep,

$ echo 'string>![TEST[Extract this string]>/string> foo bar string>![TEST[Extract this string]>/string>' | grep -oP 'string>!\[TEST\[\K.*?(?=]>/string>)'
Extract this string
Extract this string

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.