6

Is it possible for Ext4 file systems (or maybe another one understood by Linux) to assign different metadata (like modification time, owner, access rights) to several hard-links that refer to the same data. (I know that for the size this is impossible, of course).

Rationale: I want to create a simple de-duplicated archive. It can (and often will) happen, that files contain precisely the same data, but the metadata differ. My idea was to create a hard-link per file and then set the metadata accordingly. But changing one hard-link seems to also change the others. (Or maybe there's another solution to my problem?)

3

For Ext4 a hard-link simply refers to an inode, which contains all the metadata.

Therefore you cannot have different metadata using hard links.

I'm not sure what exactly you are trying to do, but you could have a look at git, zip, or soft-links. All of them have ways of handling duplicated data.

  • Thanks, I suspected that. What I am trying to do is moving the files to a repository, retaining hard-links at their original position. For the user it should look like nothing happened to the file (except for the hard-link count). I want the result to be small and I don't want versioning (so, no git) and I still want to see it "normally" in the filesystem (so no zip). Attic may be a solution. Although, I would still need to mount the archive with Attic and the mounted archive then will probably not be writable. As for soft-links, they do also use the original's metadata, right? – Xlea Mar 26 '15 at 10:20
  • The idea of git is to hash all content and instead of referring to the content simply refer to the hash. This way having multiple files with the same content are (per definition) only stored once. - Also think about fuse, which will allow you to easily build your own file system, if you have special needs. – michas Mar 26 '15 at 22:12
  • As for git your mostly right - except that the working copy is duplicated, which is not acceptable in my case. The suggestion regarding fuse is a good one, if very complex. I will have a look at bindfs suggested by Gilles, may that's what I'm looking for. Thanks again! – Xlea Mar 27 '15 at 13:14
3

It is impossible to dissociate the contents of the file from the inode which contains the file's metadata (timestamps, owner, permissions, etc.). Most metadata would be problematic if there was more than one set: not just the size (which obviously needs to match the content), but the modification time (which would have to be updated in every inode when the file is written), the owner (who controls the metadata if there are multiple owners?), etc.

If you need more flexible permissions than user-write-group, you can use access control lists, which ext4 supports out of the box.

You can use bindfs to present a view of a directory tree with some different metadata. You can't change everything that way, but you can remap users and groups and change permissions.

If you really need different files with unrelated metadata but with identical content and you don't want to duplicate the file contents due to space concerns, you can use a filesystem with deduplication. At the application, you mostly do nothing and let the filesystem sort things out. Ext4 doesn't do deduplication. ZFS can do it, but with a cost in RAM and reliability. Btrfs can also do it, with similar concerns.

  • Well, thanks a lot for the clarification, for your suggestions and for your remarks regarding de-duplication file systems. I certainly shall have a look at bindfs. – Xlea Mar 27 '15 at 13:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.