Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I want to check well-formedness of a big XML file. (about 4GB.)

However, when I try xmlwf, all it tells me is

filename.xml: Value too large for defined data type

What to do with it? Is there any other way to check it?

(I am using debian linux and gentoo linux)

share|improve this question
I'd guess any XML parser would work, as aren't they required to reject documents which aren't well-formed? A quick suggests checking if xmlstarlet does what you want. – derobert Feb 22 '13 at 18:34
From man xmlwf: "-r Normally xmlwf memory-maps the XML file before parsing; this can result in faster parsing on many platforms. -r turns off memory-mapping and uses normal file IO calls instead. Of course, memory-mapping is automatically turned off when reading from standard input." By the way, I assume you are using a 64-bit setup... – Deer Hunter Feb 22 '13 at 19:22
up vote 2 down vote accepted

You might like to try dtdgen, a program I wrote many years ago to generate a DTD for a document. It not only tells you whether a large file is well-formed, it also tells you what's in it (I wrote it because I wanted to know both).

share|improve this answer
xmllint --noout 4GB.xml

That sort of works.

It goes out of memory, too, but at least it checks something before it dies.

share|improve this answer
does xmlint ship with most distros or do you have to install it separately? – amphibient Feb 22 '13 at 20:39
@amphibient If not installed you have to install expat. – TheodorosPloumis Jan 14 '15 at 14:40

Not try it myself, but try this out :

xmllint --valid 4GB.xml
share|improve this answer
I don't want to try if it's valid. I want to try if it's well-formed. – Karel Bílek Feb 22 '13 at 18:10
Can you spot me the difference ? – Gilles Quenot Feb 22 '13 at 18:15
basically, it can't be valid if it doesn't have DTD. And I haven't yet wrote the DTD :) – Karel Bílek Feb 22 '13 at 18:18

It's an older question, but as I haven't seen it suggested yet:

Perl with XML::Twig can handle large XML files thanks to having a 'purge' method, which discards in memory data as you go.

use strict;
use warnings;

use XML::Twig;

my $twig = XML::Twig->new(
    twig_handlers => {
        _all_ => sub { $_->purge }
)->parsefile( 'my_xml_file.xml' );

The _all_ handler is triggered each element of the twig, and discards in memory data. That's important on a 4G file, because the memory footprint of XML is about 10x. But it'll throw an alert and abort if the XML is not well formed:

mismatched tag at line 12, column 27, byte 274 at C:/Perl/lib/XML/Parser.pm line 187.

(but bear in mind because it aborts, it'll only show you the first error it encounters).

Works on my (much smaller than 4G) sample data anyway.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.