0

I have a one line file like this Pastebin but way longer.

My goal is to filter only parts of the string that
example1: start with <a end with </a>
example2: start with PZ end with s16
and so in every case keep text in between the match without relying to an html entity

I am on a FreeBSD and I already have a workaround that rely on html entities

  1. Beautify to multiline tidy -i -m -w 160 -ashtml -utf8 ~/file
  2. Delete lines if not contains string sed -i '' '/\<\/a\>/!d' ~/file

By the way I am trying to run a direct filter without relying to an html entity. For now I can get only the exact beginning of a match but I don’t know how much is long the string contents I am filtering, so I can’t be precise with the end of the match, see unexpected result steps to reproduce

Steps to reproduce unexpected result

wget -O ~/file https://pastebin.com/raw/xbti369J

grep -E -o ".{0,0}PZ.{0,46}" ~/file

Result

We get wrong lines since we asked fixed lengths
PZ</td><td class="s15">€ 1.20</td><td class="s16
PZ</td><td class="s15"></td><td class="s16">A</t
Goal is to get lines result pattern no matter the length like this below
PZ</td><td class="s15">€ 1.20</td><td class="s16
PZ</td><td class="s15"></td><td class="s16
Improve this question
12
  • 1
    It sounds like you want a non-greedy match, anchored with lookarounds - something like perl's m/(?<=PZ)(.*?)(?=s16)/g
    – steeldriver
    Apr 13 at 0:48
  • 1
    Please give imput examples that are long enough to cover several edge cases, but not filled with useless junk, and give the expected output.
    – Philippos
    Apr 13 at 8:29
  • Hi @Philippos I added some steps to reproduce unexpected result, if you need more floor to answer I will be here
    – Ax_
    Apr 13 at 19:18
  • 1
    You seem to be writing a web scraper. Trying to do that using regular expressions alone is pretty much doomed to failure, you need an actual HTML parser. So, use a language with good web scraping libraries. Perl or Python, for example - the "best" choice is probably whichever language you are most familiar with that isn't shell. I mostly use the Web::Scraper perl module these days, it works well and it's fairly easy to knock up a custom scraper. The hardest/most time-consuming part is analysing the HTML structure to figure out what to extract.
    – cas
    Apr 14 at 2:42
  • 2
    @Philippos any of them. look for a td element's contents ending in PZ, then extract it and the next two td elements. Munge as required. More importantly, this question is probably at least partially an XY problem, and the extraction the OP is asking for is due to their mistaken belief that regexps are a good way to extract data from structured text like html, coupled with either lack of knowledge that html parsers exist and/or an unwillingness to use them. Most attempts to use regexps on html are due to a preference for quick and dirty hacks over doing it correctly and accurately.
    – cas
    Apr 14 at 12:33

3 Answers 3

Reset to default
2

You want to use an XML parser like xmllint.

Use the XPath expression below to filter out text in between a elements:

xmllint --html --xpath '//a/text()' <file>
Improve this answer
0
1

If you want to select each sniplet from a PZ to the nearest s16, you need a non-greedy match, which is not supported by grep (extended) regular expressions, but GNU grep has the -Perl option for Perl-style expressions:

grep -P -o "PZ.*?s16" ~/file

The Perl expression `.*?' stands for the shortest match of any characters that will make the whole expression match.

This is probably still not what you are looking for, because there are further PZ inside the matches, but as I understand your example, you only want those PZ that are followed by s16 without another PZin between. So let's get rid of unwanted stuff in a second step:

grep -P -o "PZ.*?s16" ~/file | sed 's/.*PZ/PZ/'
Improve this answer
1
  • grep -E -o "PZ.*?s16" ~/file | sed 's/.*PZ/PZ/' your -P is my -E
    – Ax_
    Apr 15 at 20:41
0

There are many ways you can do this.

1 GNU grep with PCRE enabled. Here we utilize non greedy regex *? in tandem with negative look ahead to discard any PZ occurring between PZ & s16.

grep -Po 'PZ(?:(?!PZ).)*?s16' file

2 In case you don't have access to such a version of grep, you can use the original, viz., Perl , which will have the regex support.

perl -lne 'print for /PZ(?:(?!PZ).)*?s16/g' file

3 We can use sed to do this. In the first pass, we mark the PZ & s16 as BOL & EOL. Then pass on this modified input to a second sed which shall select lines that begin with PZ, end with s16, and mustn't comprise PZ in the interior.

< file \
sed 's/PZ/\n&/g;s/s16/&\n/g' |
sed '/^PZ.*s16$/!d;/..*PZ/d' |
cat

4 We. will use just one invocation of sed in this. It requires GNU sed.

sed '/\n/{
  /^PZ[^\n]*s16/!D
  s//&\n/;P;D;}
  s/PZ/\n&/g;D
' file
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.