1

To save space and time I copied a large project tree on a network drive as hard links, i.e.

cp -a -r --link proj proj_B

(background: it's huge, needs to be rebuilt from two incompatible environments, and doesn't have good support for specifying intermediate and product locations. So this was a quick hack to get a rebuild in environment "B": after copying clean and rebuild from "proj_B/obj". Both environments are under LinuxMint 16)

The problem with this approach is that edits won't be (reliably) shared between these trees, e.g. saving an edit to "proj/foo.cpp" will leave it pointing to a new inode and "proj_B/foo.cpp" will still point to the old one (maybe from the loss-avoidance pattern of "save temp; mv orig temp2; mv temp orig; rm temp2").

For sharing source I guess I need symbolic links for the source directories (but not simply a symlink of the project root, since the binary directories need to be kept apart), e.g. something like:

cp -a -r --symbolic-link proj proj_B

followed by unlinking the binary directories (except that recursive symlink copying fails with "can make relative symbolic links only in current directory". But something similar could be done with "find -exec", or just capitulating and writing a script)

But before doing that I wanted a sanity check: is there a better tool for this all along (e.g. some warlock-grade combination of rsync flags)? Or is this sharing approach doomed to end in tears and lost data and I should resign myself to using two copies (and lots of cursing when I find I forgot to push/pull latest changes between them)?

Improve this question

3 Answers 3

Reset to default
1

I wouldn't use hard links. Some editors break hard links when they save files, others don't, and some can be configured. However, preserving hard links when saving a file implies that the file is written in place, which means that if the system crashes during the write, you will be left with an incomplete file. This is why the save-to-new-file-and-move-in-place is preferable — but this breaks hard links.

In particular, most version control software breaks hard links. So hard links are out.

A forest of symbolic links doesn't have this problem. You need to ensure that you point your editor to the master copy or that the editor follows symlinks.

You can create the symlink forest with cp -as. However, if you create new files, cp -as is inconvenient to create the corresponding symlinks (it'll do the job, but drown you in complaints that the target already exists). You can use a simple shell loop.

for env in environement1 environment2 environment3; do
  cd "../$env"
  find ../src \
       -type d -exec mkdir {} + \
       -o -exec sh -c 'ln -s "$@" .' _ {} +
done
Improve this answer
1
  • thanks for laying out that reasoning and for the model answer, which I ended up butchering into a different form as a separate answer (I wanted to be able to give arbitrary non-sibling paths and didn't find a syntax for substituting {} within -exec to trim off the known paths). But I'm taking yours as "the answer" since mine is just dumb Bash footwork on top of it :-)
    – Tom Goodfellow
    Jun 2, 2014 at 14:00
0

The problem with this approach is that edits won't be (reliably) shared between these trees, e.g. saving an edit to "proj/foo.cpp" will leave it pointing to a new inode and "proj_B/foo.cpp" will still point to the old one.

This is incorrect. If you have two hardlinked files, they are always have same contents. Edits shouldn't change this.

If you need to edit files - it seems you need something like central storage for both your dirs. May be better would be to put your project under version control (like git)?

Improve this answer
4
  • With emacs each save creates a new file, e.g.: after ln foo bar ls -i shows the same inode for both files. Now edit bar and save, ls -i now shows different inodes. It makes sense that an editor doesn't just overwrite the existing file, since that risks losing existing contents if there's an error during the operation - for interest why not try it with your favourite editor and see how that behaves?
    – Tom Goodfellow
    May 28, 2014 at 9:52
  • I'd prefer to avoid even having "both my dirs" since that then introduces synchronisation between them (and the opportunity for me to forget...) - hence preferring some form of link
    – Tom Goodfellow
    May 28, 2014 at 9:57
  • Hm, yes. This is configurable in emacs: kb.iu.edu/data/acxl.html Thought second time about it. It seems my answer doesn't really adds a lot, symlinks seems the best solution.
    – gena2x
    May 28, 2014 at 10:11
  • Thanks for the steer on backup-by-copying-when-linked - as always with emacs "there's another way to do it" (and since that handles anything but new files I might well survive with it). I also checked gedit and as a pleasant surprise it preserves the inode - beneath it all in glib/gfile is handle_overwrite_open() which explicitly checks for hard links along with other issues that prevent the rename strategy) and if so makes an explicit backup copy (presume it tries copying it back upon a write failure, but I didn't dig through the code that far). So editors using glib should be ok too.
    – Tom Goodfellow
    May 28, 2014 at 12:54
0

A converted form of the answer from @Gilles, allowing for a more cp-like syntax (my Bash scripting is laughably elementary so I couldn't figure out how to package the path substitution into "-exec {}")

#/usr/bin/bash
if [ $# -ne 2 ] ; then  echo "Args: <original tree> <new tree of symlinks>" ; exit 1; fi
mkdir -p "$2"
srclen=${#1}
find "$1" | while read line
do
    sub=${line:$srclen}
    if [ -d "$line" ]
    then
        mkdir "$2$sub"
    else
        ln -s "$1$sub" "$2$sub"
    fi
done
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .