How to check well-formedness of 4GB XML file?

Question

I want to check well-formedness of a big XML file. (about 4GB.)

However, when I try xmlwf, all it tells me is

filename.xml: Value too large for defined data type

What to do with it? Is there any other way to check it?

(I am using debian linux and gentoo linux)

Answer 1

You might like to try dtdgen, a program I wrote many years ago to generate a DTD for a document. It not only tells you whether a large file is well-formed, it also tells you what's in it (I wrote it because I wanted to know both).

Answer 2

xmllint --noout 4GB.xml

That sort of works.

It goes out of memory, too, but at least it checks something before it dies.

Answer 3

Not try it myself, but try this out :

xmllint --valid 4GB.xml

Answer 4

It's an older question, but as I haven't seen it suggested yet:

Perl with XML::Twig can handle large XML files thanks to having a 'purge' method, which discards in memory data as you go.

use strict;
use warnings;

use XML::Twig;

my $twig = XML::Twig->new(
    twig_handlers => {
        _all_ => sub { $_->purge }
    }
)->parsefile( 'my_xml_file.xml' );

The _all_ handler is triggered each element of the twig, and discards in memory data. That's important on a 4G file, because the memory footprint of XML is about 10x. But it'll throw an alert and abort if the XML is not well formed:

mismatched tag at line 12, column 27, byte 274 at C:/Perl/lib/XML/Parser.pm line 187.

(but bear in mind because it aborts, it'll only show you the first error it encounters).

Works on my (much smaller than 4G) sample data anyway.