I have a Music directory, and an MP3 player with two SD card slots. The contents of the directory is too big for one card, so I'd like to split it over two.

I like to have stuff in alphabetical order. If I filled up SD1 and left SD2 partly empty, whenever I add new music I'd either have to break the alphabetical order by putting new stuff on SD2, or re-jig everything around.

The other thing is that my directories are grouped by Artist, so if I got a new album by an artist on SD1 it wouldn't fit.

In short, I want the two cards filled up roughly equally.

Right now my solution is actually OK:

  1. List the directories and output the list to a file
  2. Open the file and find the halfway point (I'm sure there's a clever way to do this but it's quick to do manually in vim)
  3. Cut the bottom half of the list and paste it to a second file
  4. Run rsync --files-from= using one file on SD1 and the next on SD2. Job done!

This got me thinking though, is there a more accurate way to do this so that the two SD cards get filled more evenly? How would you go about this?

I guess I would have to maybe

  1. Run du on the Music dir to find the totalSize.
  2. Then loop through every Artist dir to get their sizes and keep a running total.
  3. Once that total surpasses the totalSize * 0.5, output a list of all the dirs you just counted to get the first half and put the rest in a second file to get the other half?
  4. Do the same rsync from-file

Anyone got any better ideas?

  • 1
    Is there anything preventing you from getting larger SD cards? – Mio Rin Apr 18 '18 at 16:21
  • @Mioriin They are 400GB each, my collection is quite large. – benoliver999 Apr 18 '18 at 16:26
  • 1
    Perhaps you should start sorting through what music you need readily available and keep the rest on larger storage somewhere (2-4TiB HDDs are fairly cheap these days). – Mio Rin Apr 18 '18 at 16:31
  • @Mioriin I have it all stored on a server at home, that's not the issue. My SD cards are plenty big enough to take the whole lot so I don't see why I wouldn't, I was just wondering if anyone had any bright ideas about splitting it up in a smart way. – benoliver999 Apr 18 '18 at 16:38

So, somewhat ironically, this would be really easy to do if those two SD cards where two separate computers on the same network, as there are all kinds of cluster storage solutions that do exactly what you're asking.

Given that that's not a very realistic option however (using a clustered filesystem is overkill for this), your best bet is probably a program called mhddfs. The official homepage is in Russian, but there's a decent post here about it, and it's packaged a number of different distributions. In essence, it does file-level RAID. The big advantage here is that once you get the mhddfs mount set up, you can literally just call rsync once. Each file will end up entirely on exactly one filesystem, and it will intelligently manage distribution across the filesystems. Additionally, it stores no state on the underlying devices (it just looks at the data distribution across the devices to figure out where to put any newly created files, and properly checks all of them for existing files), so it won't waste any space either.

| improve this answer | |
  • Wow this is exactly what I need. Very clever stuff! Sort of like a RAID array but one that lets you use the drives individually afterwards - I like it! The only thing it doesn't do is keep directories together (from what I can tell?) so at the split between drive 1 and 2 there might be some duplication in directory names? Doesn't matter though because if this works it would be so easy to use that it's really not an issue anymore. – benoliver999 Apr 19 '18 at 6:17
  • Ah the only catch, which to be fair I didn't mention in my post, is that I would need both SDs connected at once! I think I have a second SD reader somewhere but for people in the future reading, bear this in mind. – benoliver999 Apr 19 '18 at 6:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.