How do I break a large, +4GB file into smaller files of about 500MB each.

And how do I re-assemble them again to get the original file?

up vote 70 down vote accepted

You can use split and cat.

For example something like

$ split --bytes 500M --numeric-suffixes --suffix-length=3 foo foo.

(where the input filename is foo and the last argument is the output prefix)

The same command with short options:

$ split -b 100k -d -a 3 foo foo.

The split commands generate pieces named: foo.000, foo.001 ...

For re-assembling the generated pieces again you can use e.g.:

$ cat foo.* > foo_2

(assuming that the shell sorts the results of shell globbing - and the number of parts does not exceed the system dependent limit of arguments)

You can compare the result via:

$ cmp foo foo_2
$ echo $?

(which should output 0)

Alternatively, you can use a combination of find/sort/xargs to re-assemble the pieces:

$ find -maxdepth 1 -type f -name 'foo.*'  | sort | xargs cat > foo_3
  • 2
    Try this command: man split cat md5sum – Kevin M Sep 4 '10 at 19:13
  • 5
    When assembling, I recommend cat foo.{000..NNN} where NNN is the last expected piece. That way you get an error message if one of the pieces is missing. But note that -d to get numeric suffixes is specific to GNU split; on other platforms you have to make do with foo.aaa, foo.aab, etc. – Gilles Oct 17 '10 at 11:16
  • 1
    And bear in mind that, for split, KB = 1000, K = 1024, MB = 1000*1000, M = 1024*1024 etc. – Zorawar Nov 29 '12 at 18:05
  • Shouldn't this ... cat > foo_3 be ... cat >>foo_3? – alk Jul 8 '15 at 12:10
  • 1
    If you decide to ease pain by using a utility. rar and 7zip are often used in making such splits easier to reassemble cross-platform – infixed Jun 3 '16 at 18:05

You can also do this with Archive Manager if you prefer a GUI. Look under 'Save->Other Options->Split into volumes of'.

  • 4
    i tagged it 'command-line', but thanks for the answer :) – Stefan Sep 5 '10 at 19:25

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.