Create symlink tree in existing directories

Question

I carry all of my music around on a 1TB USB drive and have a udev rule to symlink it to $HOME/Music/ when I plug it in to one of my laptops, which it does.

The issue I have is that this works fine where the directory does not exist on the laptop, but it doesn't create the requisite tree where there is a pre-existing directory of the same name (Artist/Album/*.flac) on the laptop.

The script I currently run is this one:

#!/usr/bin/env bash
# repopulate music links when drive plugged in

shopt -s nullglob
export DISPLAY=:0
export XAUTHORITY=/home/jason/.Xauthority

music=(/media/Apollo/Music/*)

find /home/jason/Music -type l -exec rm {} \;
for dirs in "${music[@]}"; do
  ln -s "$dirs" /home/jason/Music/ 2>/dev/null
done
status1=$?

mpc update &>/dev/null

status2=$?
if [[ "$status1" -eq 0 && "$status2" -eq 0 ]]; then
  printf "%s\n" "Music directory updated" | dzen2 -p 3
fi

How can I ensure that where a directory exisits on both the laptop and the USB drive, but the contents are slightly different, the files are correctly symlinked? For example:

USB drive:

Music -- Matthew Shipp -- Patoral Composure -- Track 1
                                            -- Track 2 etc...
                       -- Strata            -- Track 1
                                            -- Track 2 etc...
                       -- Equilibrium       -- Track 1
                                            -- Track 2 etc...

Laptop:

Music -- Matthew Shipp -- Patoral Composure -- Track 1
                                            -- Track 2 etc...

In this case, no symlinks to the albums Strata or Equilibrium will be created, presumably because the parent directory (Matthew Shipp) exists.

I would prefer not to use rsync to copy the actual data across as I have limited space on the laptops and with mpd able to follow symlinks, I have no need to copy the files across.

Is it possible to tweak my script to propagate symlinks into pre-exisiting directories on the laptop?

Answer 1

Since your primary aim is to have a combined view of your local and external Music folder, I think a union mount via overlayfs could be used, especially if the files are not being written to.

The basic command is, in older kernel versions (<3.18):

mount -t overlayfs -o lowerdir=/read/only/directory,upperdir=/writeable/directory overlayfs /mount/point

For example:

$ ls Documents
374620-63301.pdf        My Kindle Content   scan0005.jpg
BPMN2_0_Poster_EN.pdf   scan0003.jpg        StrongDC++
$ ls devel
cse           ossec     ubuntu-14.04-desktop-amd64-ssh.iso
nexus         scripts   zsh-syntax-highlighting
$ sudo mount -t overlayfs -o lowerdir=$PWD/Documents,upperdir=$PWD/devel overlayfs ~/Documents
$ ls Documents
374620-63301.pdf        scan0003.jpg           
BPMN2_0_Poster_EN.pdf   scan0005.jpg    
cse                     scripts
My Kindle Content       StrongDC++
nexus                   ubuntu-14.04-desktop-amd64-ssh.iso
ossec                   zsh-syntax-highlighting

One drawback is the need for sudo, which can perhaps be taken care of using a careful NOPASSWD rule.

---

In light of Jason's blog post, the mount command for newer kernels changes to using overlay as the filesystem, instead of overlayfs, and using an additional workdir. The kernel documentation now codifies this:

At mount time, the two directories given as mount options "lowerdir" and "upperdir" are combined into a merged directory:

mount -t overlay overlay -olowerdir=/lower,upperdir=/upper,\
  workdir=/work /merged

The "workdir" needs to be an empty directory on the same filesystem as upperdir.

Answer 2

You can also use gnu stow, a symlink farm manager.

Assume the following layout:

.
├── drive
│   ├── a
│   │   ├── b
│   │   │   └── bar
│   │   └── c
│   │       └── baz
│   └── b
└── music
    └── a
        └── b
            └── foo

Execute: $ stow --target music --dir drive .

Result:

.
├── drive
│   ├── a
│   │   ├── b
│   │   │   └── bar
│   │   └── c
│   │       └── baz
│   └── b
└── music
    ├── a
    │   ├── b
    │   │   ├── bar -> ../../../drive/a/b/bar
    │   │   └── foo
    │   └── c -> ../../drive/a/c
    └── b -> ../drive/b

Answer 3

Another option is to use Xorg's lndir:

lndir source target

Note that the source needs to be relative to the target. If you were in a directory with music and drive, that'd mean:

lndir ../drive music

Because drive is at ../drive if you were in music.