4

In order to debug my repeatedly crashing Ubuntu 16.04 I want to backup /var/log from my HDD to an external USB medium. I am running an live OS (Ubuntu 16.04) and using the --archive option:

$ sudo rsync --archive /media/ubuntu/2f1ea741-cc93-4b0e-ad74-1e21d56d27a9/var/log/* /media/ubuntu/transcend/DebuggingSystemCrash@box3/var_log_2016-07-04-0939

Expected behavior

Archive mode equals -rlptgoD, so modification times should be preserved (which I want), since

-t, --times                 preserve modification times

Observed behavior

The time stamps of the files (atime and mtime) are not being preserved, see screenshot. mtime of folders is preserved.

Screenshot with source on the left and destination on the right

Since both source and target are mounted this question is probably related to how to tell rsync to preserve time stamp on files when source tree has a mounted point.

I have also read:

Am I missing something?

UPDATE

Not running rsync as sudo produces the expected behavior regarding mtime, but will skip some files.

comparison of rsync output: $ sudo rsync -a vs $ rsync -a

  • 1
    I can tell you the "mounted point" theory is incoherent, as all filesystems are mounted. So if it seems significant, you need to post the specific filesystem options. findmnt | grep /media/ plz. – sourcejedi Jul 6 '16 at 12:25
  • E.g. if the filesystem type is vfat, we should look into the behaviour described by the allow_utime= option. lxr.free-electrons.com/source/Documentation/filesystems/… This could explain your results with sudo but I don't know why there would be a difference between files and directories. – sourcejedi Jul 6 '16 at 12:39
  • Your rsync command looks fine. Therefore it's something else that's causing the issue. What filesystem type is the target (backup) device? How is it mounted, i.e. what mount options? (Both answers can be obtained from the mount command, but you probably know that.) – roaima Jul 6 '16 at 13:23
2

Difference with sudo could be due to the extract below from Documentation/filesystems/vfat.txt. (Requires knowledge that your target filesystem is of type vfat & who owns the directory^Wfilesystem - determined by mount option in the case of vfat).

I went ahead and posted this anyway because it shows how important the filesystem types are. The general conclusion was that using FAT on Linux is pain (and there has been a special issue with rsync on FAT as well).

I'm certain at least one of the issues listed there has been solved with different defaults (possibly in udisks as used by the GUI, as opposed to manual mount commands). In any case I suggest you'd be wasting your time if what you're trying to do is copy parts of your Linux OS into FAT, and you demand specific results in file metadata (timestamps), or even file names. In that case you should use a native Linux filesystem, ideally the same type as the source.

Or, instead of copying individual files, create an archive using a native Linux tool such as tar. Think Zip file - but specifically designed for backing up Linux *nix files - their names, and the most common metadata. For example, tar -c -f out.tar input-directory/ (omits compression).

Reference doc for setting FAT timestamps on Linux

allow_utime=### -- This option controls the permission check of mtime/atime.

                  20 - If current process is in group of file's group ID,
                       you can change timestamp.
                   2 - Other users can change timestamp.

                 The default is set from `dmask' option. (If the directory is
                 writable, utime(2) is also allowed. I.e. ~dmask & 022)

                 Normally utime(2) checks current process is owner of
                 the file, or it has CAP_FOWNER capability.  But FAT
                 filesystem doesn't have uid/gid on disk, so normal
                 check is too unflexible. With this option you can
                 relax it.

Community confirmation and other potential issues to avoid

So the blog post I found confirms "only the owner of a FAT32 mount can set file time stamps on it". I'd call this a design bug (root should be able to do it as well), which the reference documentation almost spells out, but I'm too lazy to submit a patch for it.

The blog also provides an explanation for a file/directory difference. Unfortunately it seems to be doubly opposite what you describe. I think the file/directory difference described below should have prevented your normal user from setting timestamps correctly on directories - whereas what you've showed is that if you're root, you can set the correct timestamps on directories but not files.

It sounds to me like the --modify-window=1 is a workaround for an rsync-specific issue. So using cp to copy files instead might give you another data point.

Even the owner of a FAT32 mount can't set directory times reliably

Although I was now able to copy over the original file time stamps correctly I noticed that all of the directory modification times were set to the time I ran the rsync command, which is not what I wanted at all. A little googling uncovered this forum posting about this problem: http://ubuntuforums.org/showthread.php?t=886048

This forum posting suggested adding the option "--modify-window=1", which gives 1 second slack on how closely file and directory times have to match before rsync will see them as different, and someone said that worked to correctly preserve original directory timestamps.

  • Filesystem of the target device is vfat, you were right with your guess. (I will check the mountpoint next time I run live OS.) Would you mind elaborating as to how to use archiving tool (tar) for this purpose? “[...] unfortunately it seems to be the other way round from what you describe.” Not sure what you mean by that. From what I describe it seems to be the other way round?! – marianoju Jul 7 '16 at 10:29
  • Attempted. I mean I found a weird difference between files and directories, when rsync sets mtime on vfat, but I don't think it actually matches your screenshots. I.e. not only is this a massive nightmare (just look at how long the 2008 blog post is), I think you're hitting some other constraint, one which that very scientific blog post looking into the same scenario, didn't manage to document. – sourcejedi Jul 8 '16 at 13:09
  • I'm all about digging into obscure detail, but also noticing when you're doing obscure stuff, why you want to do it the obscure way and what the simpler approach would be :). – sourcejedi Jul 8 '16 at 13:14
  • Even though your answer did not solve my problem, it did help in finding out more about the reasons behind my problem. Sorry for not accepting your answer earlier, and thanks again for your help! – marianoju Sep 11 '18 at 10:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.