3

I have tons of html pages with a simlar content as follows:

<div class="wrapper">

<div class="aaa">
...lot of html1 here like div's/spans etc..
</div> <!-- end aaa -->

<div class="bbb">
...lot of html2 here
</div><!-- end bbb -->

</div>

I need to swap div's with aaa and bbb classes entirely with their content:

So it becomes:

<div class="wrapper">

<div class="bbb">
...lot of html2 here
</div><!-- end bbb -->

<div class="aaa">
...lot of html1 here
</div> <!-- end aaa -->

</div>

NOTES:

1) Empty lines between the blocks are optional.
2) HTMLs contain only one exact pair of aaa and bbb so, the case: aaa then bbb then aaa is not possible.

So could you please advise what arguments should I pass to sed. I'm using find to find all html files and then launch sed as exec param:

find . -iname "*.html" -exec sed -i '' 's/WHAT IS HERE / AND HERE /g' {} \;

If it is not possible with sed but let's say is possible, with awk I will be glad if you could share your thoughts on the best way to achieve the desired.

UPDATE:

Here is the link to real life example: http://pastebin.com/mdhJ9rtL

  • Please take a look closely. Example is already added :) – Sray Dec 24 '14 at 15:39
  • @Sigur, yup, exactly. – Sray Dec 24 '14 at 15:41
  • Is aaa and bbb meets in htmls together? Many times or just one? – Costas Dec 24 '14 at 15:44
  • Do you know that there are blank lines in between? – glenn jackman Dec 24 '14 at 16:59
1

If you want to put aaa section from before bbb just after it:

sed -i '/<div class="aaa">/{
        :1
        /<\/div> <\!-- end aaa -->/!{N;b 1}
        /<\/div> <\!-- end aaa -->/{N;h}
        d}
        /<\/div><\!-- end bbb -->/{n;G}' *html
  • Unfortunately, you code returns "extra characters at the end of p command". Could you please advise? – Sray Jan 19 '15 at 5:43
  • @Sray Is there some metacharacters in your classes content - especially / (because there is no p command in the script)? You can try to change each / in script to other some symbol which surely do not present in your content. For example \|<div class="aaa">| { ... – Costas Jan 19 '15 at 10:53
1

Here's another sed:

sed '/.*<div class="...">.*/{ h;s///;x;:n
     /<.div>/!N;/<!-- end/x;/<.div>/x;//!bn
    s/\(.*\).\(<div class=.*>\).*/\2\1/;x
     /<.div>[^>]*$/s/.//;H;x
}'

Starting from a class=.???. line and running through however many blocks you have, for each pair this swaps their positions. So, here are some examples:

If sed encounters a line that matches:

<div class=".\{3\}">

...while reading its in file it will ensure Hold space is completely clear and then begin pulling in every line until it encounters either a line that matches:

<.div>

...and...

<!-- end

...or just the former. If it matches both then sed saves the block in an alternate buffer and pulls in a second block before swapping their positions on output.

If just the former it does not affect the blocks position. In this way misatched pairs are left alone.

Given as input...

<div class="wrapper">
<div class="aaa"> first </div> <!-- end aaa -->
between
<div class="bbb"> swap two </div> <!-- end bbb -->
blocks
<div class="ccc"> mismatched </div> <!-- end ccc --> 
the end         
</div>

It prints...

<div class="wrapper">
<div class="bbb"> swap two </div> <!-- end bbb -->
between
<div class="aaa"> first </div> <!-- end aaa -->
blocks
<div class="ccc"> mismatched </div> <!-- end ccc -->
the end
</div>

...if given:

<div class="wrapper">
<div class="aaa"> first </div> <!-- end aaa -->
between
<div class="bbb"> swap two </div> <!-- end bbb -->
blocks
<div class="ccc"> matched </div> <!-- end ccc --> 
the end
<div class="ddd"> now matched </div> <!-- end ddd -->
</div>

It prints...

<div class="wrapper">
<div class="bbb"> swap two </div> <!-- end bbb -->
between
<div class="aaa"> first </div> <!-- end aaa -->
blocks
<div class="ddd"> now matched </div> <!-- end ddd -->
the end
<div class="ccc"> matched </div> <!-- end ccc -->
</div>

And, though the examples are all crunched up like that for space's sake, it has no real concern for whether or not the <div class= begin and the <.div> <!-- end sections fall on the same line or not:

<div class="wrapper">
<div class="aaa">

the first
block is here

</div> <!-- end aaa -->

these lines were
between aaa and bbb

<div class="bbb">

this is the second block
it should be swapped with the first

</div> <!-- end bbb -->

more
blocks
follow

<div class="ccc"> this is matched </div> <!-- end ccc -->
not the end
<div class="ddd">

this last block
is matched with the ccc line
</div> <!-- end ddd -->

this is the end
</div>

Gets...

<div class="wrapper">
<div class="bbb"> 

this is the second block
it should be swapped with the first

</div> <!-- end bbb -->

these lines were
between aaa and bbb

<div class="aaa"> 

the first
block is here

</div> <!-- end aaa -->

more
blocks
follow

<div class="ddd"> 

this last block
is matched with the ccc line
</div> <!-- end ddd -->
not the end
<div class="ccc"> this is matched </div> <!-- end ccc -->

this is the end
</div>
  • Thank you, sir. This works if blocks contain only texts. But in case they contain other divs, this script just pulls out one of the children divs before the parent instead of changing the places of two parent divs. – Sray Jan 17 '15 at 13:41
  • @Sray - this won't do nested instances. In fact - such a thing is not possible with regular expressions because language with contextual grammar ceases to be regular. You can recurse the parse though - you just have stack each level. – mikeserv Jan 17 '15 at 13:58
  • I'm sorry if I did not stand out. But this script just takes a child <div> let's say with class ccc and places it before div with class aaa. So it becomes: <div class="ccc">some content</div><div class="aaa"></div><div class="bbb"></div> Instead of <div class="bbb"></div><div class="aaa"><div class="ccc"></div></div> I don't need to process internal content, just swap aaa and bbb without touching the children divs/spans etc.. – Sray Jan 17 '15 at 14:08
  • @Sray - and how are these delimited? The same way as the internal content? There's no clear cut way for skipping over the internals without a top-level delimiter. – mikeserv Jan 17 '15 at 14:16
  • No, ccc is the internal content of aaa. It should stayed untouched. Just swap aaa and bbb. Please take a look at the real life example: pastebin.com/mdhJ9rtL – Sray Jan 17 '15 at 14:29
1

This is not work for sed, unless you are a glutton for punishment. At least in a more general case, where the beginning of the block is more then one line (or whee the tags are split across several lines, which is well possible in XML/HTML).

If you really must do this with anything else than an XML parser (yes, either fixing the input or cutting out the broken parts would generally be a better idea), use something like awk at least - it's much more suitable for a task like that*). General idea is to:

  1. print input lines until the beginning of the first block;
  2. accumulate lines of the first block to swap;
  3. accumulate lines between the blocks;
  4. print lines of the second block;
  5. print lines of the part between the blocks accumulated in step 3;
  6. print lines of the first block accumulated in step 2;
  7. print the rest.

Also remember to check the canonical SO Q&A.

* Why I claim that: sed is line-oriented and intended for simple (your mileage may vary) text transformations. While this holds true for AWK (and in some degree for Perl) as well, writing more complex scripts is simpler in the latter (easier access to multiple variables, automatic splitting into fields etc.). Thus unless you only need to swap two extremely well delimited blocks and will never need to extend the script to handle differently formatted input, a more complex language will likely be a better tool. That said, Perl has a XML parser readily available as a module.

  • I'm not familiar with awk. I will be grateful if you could provide me with example. – Sray Dec 24 '14 at 17:01
  • 1
    How is awk more suitable? That is an unfounded assertion. People often say things like that but I think it is because they just know how to use one better than the other - which may speak to their preference, but not the capability of the tool. This is very easily done with sed by the way. – mikeserv Dec 24 '14 at 17:07
  • @mikeserv more suitable in general case - try matching three following lines as a beginning of block. It might be easy in this particular case but if it gets a bit more complicated sed is not the way to go. (And neither is awk, but that's only repeating the answer.) – peterph Dec 24 '14 at 20:56
  • Also try to use sed in case there is something between the two blocks. You can do it (sed is Turing complete), but it will stop being comprehensible to most people. – peterph Dec 24 '14 at 21:02
  • No, these things are simple with sed - and they always are - sed is extremely simple. sed is very simple - if most people don't understand how it works, it is because most people make little effort to understand data flow, and require instead that a computer needlessly buffer to better accommodate their level of understanding. I will post an answer that demonstrates this with sed - though there is an excellent one here already, perhaps I can show another way - as there are several. Just a few minutess – mikeserv Dec 24 '14 at 22:07
0

Parsing HTML with regex is clearly discouraged.

Instead, you can use & if your source files are valid XHTML :

xmlstarlet edit -L -u "//div[@class='a']" -v 'some inner HTML' file.xhtml

If it's not valid XHTML, try to adapt the following perl code :

use strict;
use warnings;
use 5.008;

use File::Slurp 'read_file';
use HTML::TreeBuilder;

sub replace_keyword
{
  my $elt = shift;

  return if $elt->is_empty;

  $elt->normalize_content;      # Make sure text is contiguous

  my $content = $elt->content_array_ref;

  for (my $i = 0; $i < @$content; ++$i) {
    if (ref $content->[$i]) {
      # It's a child element, process it recursively:
      replace_keyword($content->[$i])
          unless $content->[$i]->tag eq 'a'; # Don't descend into <a>
    } else {
      # It's text:
      if ($content->[$i] =~ /here/) { # your keyword or regexp here
        $elt->splice_content(
          $i, 1, # Replace this text element with...
          substr($content->[$i], 0, $-[0]), # the pre-match text
          # A hyperlink with the keyword itself:
          [ a => { href => 'http://example.com' },
            substr($content->[$i], $-[0], $+[0] - $-[0]) ],
          substr($content->[$i], $+[0])   # the post-match text
        );
      } # end if text contains keyword
    } # end else text
  } # end for $i in content index
} # end replace_keyword


my $content = read_file('foo.shtml');

# Wrap the SHTML fragment so the comments don't move:
my $html = HTML::TreeBuilder->new;
$html->store_comments(1);
$html->parse("<html><body>$content</body></html>");

my $body = $html->look_down(qw(_tag body));
replace_keyword($body);

# Now strip the wrapper to get the SHTML fragment back:
$content = $body->as_HTML;
$content =~ s!^<body>\n?!!;
$content =~ s!</body>\s*\z!!;

Borrowed from https://stackoverflow.com/questions/3900870/how-can-i-modify-html-files-in-perl

  • It's not valid XHTML. Moreover, the inner content of both 'aaa' and 'bbb' div's is quite a big chunk of code. – Sray Dec 24 '14 at 15:47
  • The inner content should not be a very big deal, but malformed HTML is. – Gilles Quenot Dec 24 '14 at 15:53
  • OK, could you please explain how it works? Honestly, never heard about this command. – Sray Dec 24 '14 at 16:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.