72

How do I break a large, +4GB file into smaller files of about 500MB each.

And how do I re-assemble them again to get the original file?

2

2 Answers 2

88

You can use split and cat.

For example something like

$ split --bytes 500M --numeric-suffixes --suffix-length=3 foo foo.

(where the input filename is foo and the last argument is the output prefix). This will create files like foo.000 foo.001 ...

The same command with short options:

$ split -b 100k -d -a 3 foo foo

You can also specify "--line-bytes" if you wish it to split on line boundaries instead of just exact number of bytes.

For re-assembling the generated pieces again you can use e.g.:

$ cat foo.* > foo_2

(assuming that the shell sorts the results of shell globbing - and the number of parts does not exceed the system dependent limit of arguments)

You can compare the result via:

$ cmp foo foo_2
$ echo $?

(which should output 0)

Alternatively, you can use a combination of find/sort/xargs to re-assemble the pieces:

$ find -maxdepth 1 -type f -name 'foo.*'  | sort | xargs cat > foo_3
7
  • 2
    Try this command: man split cat md5sum
    – Kevin M
    Sep 4, 2010 at 19:13
  • 6
    When assembling, I recommend cat foo.{000..NNN} where NNN is the last expected piece. That way you get an error message if one of the pieces is missing. But note that -d to get numeric suffixes is specific to GNU split; on other platforms you have to make do with foo.aaa, foo.aab, etc. Oct 17, 2010 at 11:16
  • 2
    And bear in mind that, for split, KB = 1000, K = 1024, MB = 1000*1000, M = 1024*1024 etc.
    – Zorawar
    Nov 29, 2012 at 18:05
  • 1
    Shouldn't this ... cat > foo_3 be ... cat >>foo_3?
    – alk
    Jul 8, 2015 at 12:10
  • 1
    If you decide to ease pain by using a utility. rar and 7zip are often used in making such splits easier to reassemble cross-platform
    – infixed
    Jun 3, 2016 at 18:05
4

You can also do this with Archive Manager if you prefer a GUI. Look under 'Save->Other Options->Split into volumes of'.

1
  • 7
    i tagged it 'command-line', but thanks for the answer :)
    – Stefan
    Sep 5, 2010 at 19:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.