I would like to merge efficiently binary files using a shell command and I quickly found classical ways like this one :

cat file1 file2 > file3

Not bad but :

  1. It's slow. IO access are slow.
  2. It needs extra space. I don't want to copy the files. Just concatenate them.

File systems are great to handle file fragmentation. Can't we just merge files using this mecanism?


You can do this:

cat file2 file3 [...] filen >> file1

This will concatenate file2, file3,..., filen to the end of file1 in-place. The >> operator tells the shell to write to the end of the file.

You want to leverage the file system to "handle file fragmentation". Unfortunately, there isn't any general way to do this. It's because "file systems" are much more general than file systems on disks — for example, you have NFS, FUSE, and many other mechanisms which let you expose any kind of resource (not just block devices such as hard disks) as a file system hierarchy. Even for block-device-based file systems, there isn't a standard mechanism to do this, and I don't know any implementation-specific ones either.

  • Thanks for your answer. I'm investigating virtual file systems! – zz64 Feb 19 '16 at 14:44
  • The only "easy" solution I have found to my challenge is a virtual file that encapsulate my files. I can't do an operation on my hard disk in the name abstraction. So I put even more abstraction to manage this ! :P Ironic isn't it ? – zz64 Feb 19 '16 at 16:00
  • The first part of your answer is about copying files. I suggest to remove it and to keep only the last part related to manipulating the file system. It would then be my best answer! – zz64 Feb 22 '16 at 10:28
    dd if=firstfile.raw > completedfile.raw
    dd if=nfile.raw    >> completedfile.raw
    dd if=lastfile.raw >> completedfile.raw
  • Uhhh, ooooh, what an epic explanation... – user62916 Mar 20 '18 at 19:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.