3

I try to use the AmazonCloud-Storage using acd_cli to put my media files into the cloud. Unfortunately acd_cli don't work with large files. Therefore I'm search for a (virtual) filesystem which split large files into configurable peaces on the storage side.

E.g. mount.splitfs If the user stores a large file into the the data is only written into but files exceeding the configured size will be stored as file.{1..n}.

The complete setup will be like:

unionfs --+-- local_encfs (RW) ----- local_splitfs ----- local_storage
          |
          +-- acd_encfs (RO) ----- acd_splitfs ----- acd_cli_storage

The local and acd path is necesarry as acd_cli only support writing a file in sequence but encfs writes in random positions. New files will be written localy and a cron job moves them from the local_storage into the acd_cli_storage if they are older than x days.

I got most of all working but the splitfs is missing. Any ideas how the realize the splitfs?

Kind regards, Rüdiger

1
  • glusterfs? (Not sure if it can do that, and even if it does I don't know if it's appropriate for your use case) Oct 14, 2016 at 22:12

1 Answer 1

0

chunkfs.

ChunkFS is a FUSE based filesystem that allows you to mount an arbitrary file or block device as a directory tree of files that each represent a chunk of user-specified size of the mounted file.

More info at http://www.linux-mag.com/id/7437/

2
  • Chunkfs devides the complete fs into chunks. The critical size of the amazon cloud drive is 10GiB. I have to store 20TB. This causes the creation of > 2000 chunk files and mount them together. But in the 20TB are much less than this count of files which must be spitt into 2 ore 3 pieces. Oct 15, 2016 at 9:05
  • I tested with some localy created files (dd, losetup, mkfs, mount, unionfs) like describet in the referenced articel. Unfortunately I can't create a file larger than one chunk in the union. Oct 15, 2016 at 9:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .