This is how I did to extract the first 100000 lines from my big xml file (2gb):

head source.xml -n 100000 > part.xml

How can I keep splitting them to 100000 line (or specific file size chunks) until the whole file is separated?

  • you want this file to split in multiple files of 100000 lines right??
    – Hackaholic
    Nov 9 '14 at 16:24
  • @Hackaholic Yes, 100000 is random number but I would like to limit the file size by line numbers.
    – Badr Hari
    Nov 9 '14 at 16:27

You could use

split -l lines_per_file --additional-suffix=.xml source.xml part

This will read the file source.xml and split it into chunks of lines_per_file lines each. The result will be written into a series of files partaa.xml, partab.xml, partac.xml, ...

If you want to use another number of suffix characters, you can use the -a option to specify a number, eg. -a 1 to name the files parta.xml, partb.xml, partc.xml, ...

If you want to split to file size chunks instead of line counts, you can use -b size_in_bytes instead of -l lines_per_file.

Please note that the resulting files will most likely be invalid XML files (unless you happen to get one file in return, ie. your input had too few lines/bytes to get split).

  • 1
    Thank you, this is exactly what I needed and it did it very fast. And yes, I'm aware that it will break the files in random places and will result invalid xml.
    – Badr Hari
    Nov 9 '14 at 16:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.