0

I'm thinking a way of storing large files on vfat file system. Obviously, the only way to store files larger than 4GiB is to split them. I'm aware that we can use split and cat commands for splitting and merging files. However, to use those commands, more space is needed for resulting file(s). If I'm understanding how file systems work correctly, no file is actually deleted and deallocated until they're closed. To use as little space as possible, it would take funky technique that involves truncate() and reversing the file content(reading and shrinking the original file as it goes).

Is there any kernel module that creates a loop device out of multiple split files? Or a util command that does the exact same idea as mine?

Or I'll start making one of them. Oh! Can I do that with mdadm? The linear level?

3
  • btw, I know how to do things like cat file.tar.00 file.tar.01 file.tar.02 | tar xf -
    – user210007
    Feb 2, 2017 at 0:52
  • Then what is your practical use that you can't do by yourself? Do note that truncating a file as you read it would imply rewriting the whole remains of the file after every segment you read, thus extremely slow. Feb 2, 2017 at 0:55
  • If the split files don't have odd sizes, you can do it loop devices and concatenate them with device mapper linear target. If they do have odd sizes, you need a user space block device or filesystem - fuse and nbd can be used for those. mdadm build linear might work too, never tried it. Feb 2, 2017 at 1:55

1 Answer 1

0

While I dont understand your need to use a fat file system, this seems a duplicate of https://unix.stackexchange.com/a/17173/108069

In your case each of your dd files would be 4gb each.

2
  • OP mentioned a need to avoid using more disk space and there is absolutely no advantage doing things that way instead of using a pipe, quite the opposite. Feb 2, 2017 at 1:37
  • The advantage to loop mounting+mdadm over pipe is you dont need to create an entire second file somewhere, either in memory or on disk every time you want to access it. As long as it's mounted the entire file / contents are always available.
    – jammin
    Feb 6, 2017 at 1:40

You must log in to answer this question.