Use cases for hardlinks? [closed]

Question

In what situations would one want to use a hard-link rather than a soft-link? I personally have never run across a situation where I'd want to use a hard-link over a soft-link, and the only use-case I've come across when searching the web is deduplicating identical files.

Answer 1

Aside from the backup usage mentioned in another comment, which I believe also includes the snapshots on a BTRFS volume, a use-case for hard-links over soft-links is a tag-sorted collection of files. (Not necessarily the best method to create a collection, a database-driven method is potentially better, but for a simple collection that's reasonably stable, it's not too bad.)

A media collection where all files are stored in one, flat, directory and are sorted into other directories based on various criteria, i.e.: year, subject, artist, genre, etc. This could be a personal movie collection, or a commercial studio's collective works. Essentially finished, the file is saved, not likely to be modified, and sorted, possibly into multiple locations by links.

Bear in mind that the concept of "original" and "copy" are not applicable to hard-links: every link to the file is an original, there is no "copy" in the normal sense. For the description of the use-case, however, the terms mimic the logic of the behavior.

The "original" is saved in the "catalog" directory, and the sorted "copies" are hard-linked to those files. The file attributes on the sorting directories can be set to r/o, preventing any accidental changes to the file-names and sorted structure, while the attributes on the catalog directory can be r/w allowing it to be modified as needed. (Case for that would be music files where some players attempt to rename and reorganize files based on tags embedded in the media file, from user input, or internet retrieval.) Additionally, since the attributes of the "copy" directories can be different than the "original" directory, the sorted structure could be made available to the group, or world, with restricted access while the main "catalog" is only accessible to the principal user, with full access. The files themselves, however will always have the same attributes on all links to that inode. (ACL could be explored to enhance that, but not my knowledge area.)

If the original is renamed, or moved (the single "catalog" directory becomes too large to manage, for example) the hard-links remain valid, soft-links are broken. If the "copies" are moved and the soft-links are relative, then the soft-links will, again, be broken, and the hard-links will not be.

Note: there seems to be inconsistency on how different tools report disk usage when soft-links are involved. With hard-links, however, it seems consistent. So with 100 files in a catalog sorted into a collection of "tags", there could easily be 500 linked "copies." (For an photograph collection, say date, photographer, and an average of 3 "subject" tags.) Dolphin, for example, would report that as 100 files for hard-links, and 600 files if soft-links are used. Interestingly, it reports that same disk-space usage either way, so it looks like a large collection of small files for soft-links, and a small collection of large files for hard-links.

A caveat to this type of use-case is that in file-systems that use COW, modifying the "original" could break the hard-links, but not break the soft-links. But, if the intent is to have the master copy, after editing, saved, and sorted, COW doesn't enter the scenario.

Answer 2

Hard links are useful for cases where you don't want to tie the existence of both files. Consider this:

touch a
ln -s a b
rm a

Now b is useless. (And these steps may happen quite far apart, be done by different people, etc.)

Whereas with a hard link,

touch a
ln a b
rm a

b is still present and correct.

There are a number of scenarios where this is useful. One which is commonly encountered is de-duplication:

• within a file system, e.g. in your home directory, de-duplication tools can replace copies with hard links; this preserves some of the “independence” of the two files, in that one can still be deleted without affecting the other, but it does mean that changes to one will be visible in the other (unless the tool making the change uses the “save to new file and rename” strategy)
• within a backup system, hard links are often used to maintain multiple snapshots: if a given file is backed up every day, the backup system can hard link all the copies (but not the original), which reduces the space requirements while still ensuring that each snapshot is self-supporting at the file system level.

Another use-case is to make a file available in some other location. For example, if you have a script which you maintain in a “development” location, you may want to hard-link it to a location on your path; this ensures that the script remains available even if you remove it in your development directory. (There are a number of caveats in this scenario however; you may find it more appropriate to “install” a specific version in your path, or use a symlink.)

Answer 3

A single program may change its behavior depending on what name it is launched as:

$ ls -li `which pgrep` `which pkill`
208330 -r-xr-xr-x  2 root  bin  19144 Jul 26  2016 /usr/bin/pgrep
208330 -r-xr-xr-x  2 root  bin  19144 Jul 26  2016 /usr/bin/pkill

Which over in the source is decided via something like

if (strcmp(__progname, "pgrep") == 0) {
    action = grepact;
    pgrep = 1;
} else {
    action = killact;

though the exact details wil vary depending on the OS and language involved.

This allows (mostly) identical code to not have to be compiled out to two (mostly) identical binaries. Bear in mind unix dates to days when disk space was super expensive, though according to Stevens in APUE chapter 4 symlinks were implemented in BSD4.2 (1983) to replace various limitations of hardlinks. A test program to check whether the symlink name is used as the program name might look something like:

#include <stdio.h>
#include <stdlib.h>
int main(int argc, char *argv[])
{
    printf("called as '%s'\n", *argv);
    exit(0);
}

And tested via:

$ cc -o myname myname.c 
$ ln -s myname alias
$ ./myname
called as './myname'
$ ./alias
called as './alias'
$ 

Answer 4

When my P2P software finishes downloading a certain file, the file is placed in a specific directory. Downloaded files hardly ever need to be edited. The common case is I make a hardlink in a different directory where I need the file to be.

Advantages:

• I still share the file in P2P network as I should even if I rm or mv the "copy".
• The file is also at the path where I need it; most of such locations are not shared.
• I can rm the "original" to stop sharing the file; this operation doesn't affect the "copy" in desired place.
• My diskspace is used just once.

The main point: if I knew in advance which file I would rm first, I might go with a symlink; but I never know.

Answer 5

Filesystems are a simple and yet an efficient way to organize and classify files (this is their very primary reason for existence). Hardlinks allow a higher degree of flexibility in this matter.

As mentioned, there is no concept of original and copies when dealing with hardlinks, all directory entries (hardlinks) are simply references to the existence of the file (point to its inode) with no precedence, hence there are also no broken hardlinks...

So here there are some of the use cases that hardlinks attend but softlinks don't:

1. Imagine you have a collection of movies or music or other media and want to have different classification criteria applied, like songs classified by artist in a branch (each artist has its own sub-directory); by genre in another branch (each in a different sub-directory), etc.. Still you don't want to duplicate the files nor to decide where to put the "original" so that you have the freedom to reclassify without having to "manage" and re-link files when moving in order to avoid broken links.

2. Another reason is to avoid the waste of storage space that would be required for having multiple copies of the same file and yet allow the chroot syscall to benefit from a subset of files in the "master" filesystem root (symbolic links could never reference files from outside the chroot sandbox, even if they have relative paths).

3. Another very important but rarely mentioned reason for hardlinks to exist are the .. subdirectories. The .. directories actually are (in most unix fs implementations) hardlinks to the parent directory, without hardlinks this has to be implemented in a completely different way, while the existence of hardlinks makes this very easy to be implemented.

Answer 6

Very common, real-world example that needs hardlinks:

git clone --reference <repository>

This clones from an local Git repo with nearly zero copying. Instead of copying the object files (immutable files used by Git for its "database"), it simply hardlinks them.

Any repo can remove an object, but the inode stays valid for the rest of the repos. And if an object is removed from all repos, it's deleted from disk. Hard links make for a beautifully robust and fast solution. Very common in CI servers.

---

There is a non-hard-link version: git clone --shared <repository>. This, however, is fickle and has many more caveats since everyone is working on the same directory.

Answer 7

I had recently a use case for a somewhat safe update procedure for U-Boot based systems where uImage is a soft-link pointing to the image to boot, the idea was that a power outage should pose no problems, no matter at which point in the process it happens (assuming the file system plays along):

ln image.bin backup_image.bin
ln -sf backup_image.bin uImage

// replace image.bin

ln -sf image.bin uImage
rm backup_image.bin

Without hardlinks it wouldn't be that simple.

/edit:

Thanks to the comments I now know that it would be better to do:

ln image.bin backup_image.bin
ln -sf backup_image.bin uImageNew
mv uImageNew uImage || rm -rf uImage && mv uImageNew uImage

// replace image.bin

ln -sf image.bin uImageNew
mv uImageNew uImage || rm -rf uImage && mv uImageNew uImage
rm backup_image.bin

(The rm is here in order to be able to better escape a strange state, e.g. if uImage is something unexpected which would make mv fail [but not necessarily the previous ln -sf solution].)

Answer 8

One use that I have had for hard links is when downloading or uncompressing a broken file. The program that does the downloading or uncompressing (such as unzip or unrar) will often automatically remove the incomplete file when it encounters an error, and there is usually no option to keep it. If I want to keep the file, I can make a hard link to it.

Answer 9

BackupPC is a backup system that uses hard links on the servers to provide file-level deduplication.

Files are first stored in a "pool" directory tree based on their md5 hash. Any backup that makes use of that file makes a hard link to the pool file. As backups expire/are deleted, their hard links are removed from the filesystem.

Hard links are superior to soft links here because they provide automatic reference counting. A cron job periodically deletes any files in the pool directory that don't have more than one link.

This method has some disadvantages (principally, that it is difficult to use filesystem-based tools to replicate the backup store), but it's proven to be quite robust in practice.

---

Another use case: the tomcat java web application server treats file names as metadata. A java "war" file must be named based on its path on the web server.

e.g.: foo.war is the java code that serves the url /foo

Unfortunately, it resolves symlinks before making this decision.

So, say you want to deploy an application build, and give it a descriptive file name (e.g., with a release number or date). You can't make a symlink to the file with the "real" name - you have to make a hardlink.

foo.war symlinked to foo-20170129.war doesn't work

foo.war hardlinked to foo-20170129.war works.

i don't like this tomcat behavior, but hardlinks give me a way around it.