19

I get loads of warnings when backing up my running Postfix and Courier server files like:

file has vanished: /var/kunden/mail/username/name@mymail.de/tmp/courier.lock

How do I suppress those warnings from rsync when running it from Cron /usr/bin/rsnapshot hourly?

Can I somehow exclude those dirs?

/var/kunden/mail/*/*/tmp/

the tmp folder can be deeper as well, for example:

file has vanished: /var/kunden/mail/username/name@mymail.de/.Presse/tmp/1353871473.M716135P32214_imapuid_36.test.de
file has vanished: /var/kunden/mail/username/name@mymail.de/.Presse/tmp/courier.lock
8

You can use rsync's exclude switch (--exclude):

$ rsync -avz --exclude '**/tmp/' source/ destination/

Specified this way --exclude '**/tmp/' will ignore any path which includes the string /tmp/. You can provide patterns to this arguments as well.

Example

$ rsync -avz --exclude '/path/to/*/tmp/' source/ destination/

Will exclude on paths of the form: /path/to/*/tmp/.

4
  • so I have to try rsync -a --exclude '/var/kunden/mail/*/*/tmp/' --exclude '/var/kunden/mail/*/*/*/tmp/' /var/kunden/ $SERVER/var_kunden/ ? – rubo77 Aug 15 '13 at 16:02
  • @rubo77 - it will ignore any directories that include /tmp/ in them as I wrote it. If you want to be more explicit then yes you can specify it as you've written. – slm Aug 15 '13 at 16:11
  • @rubo77 --exclude='/tmp/' excludes only the directory called tmp at the root of the copy. To exclude all subdirectories called tmp anywhere, use --exclude='**/tmp/'. – Gilles 'SO- stop being evil' Aug 15 '13 at 22:16
  • 1
    ...or use a file system with snapshots... it seems problematic to restore a backup where files disappear halfway through... unless they are inconsequential, in which case, yes, they should probably be excluded... – Erk Dec 15 '19 at 5:00
22

Unfortunately, unlike what is described in SWdream solution, --ignore-missing-args has no impact on vanished files. It will simply ignore source arguments that doesn't exist.

See man rsync:

  --ignore-missing-args
          When rsync is first processing the explicitly  requested  source
          files  (e.g. command-line arguments or --files-from entries), it
          is normally an error if the file cannot be found.   This  option
          suppresses  that  error,  and does not try to transfer the file.
          This does not affect subsequent vanished-file errors if  a  file
          was initially found to be present and later is no longer there.

The "official" way of ignoring vanished file is to use this script from the official rsync source repository: https://git.samba.org/?p=rsync.git;a=blob_plain;f=support/rsync-no-vanished;hb=HEAD

which is very similar to what @kenorb and @gilles-quenot said.

16

The reason is these files were existing while rsync is building the list of files to transfer but they are removed before transferring.

It is a warning massage, not an error. However, you should try to find out why these file was deleted, it is maybe important.

To ignore this warning, you can use --exclude option as above question or use -ignore-missing-args rsync option, it makes rsync ignores vanished files: --ignore-missing-args ignore missing source args without error it maybe helps.

4
  • This seems definitely better than checking for a specific return code and overriding it to 0. – Xiong Chiamiov Sep 28 '16 at 17:52
  • Except in rare cases, files vanishing during a live backup are perfectly normal (a lot of applications create short-lived temporary files). This is especially true in the case of a mail server, where files containing e-mail messages are constantly moved from one directory to another, so IMHO this answer is more adequate than the one accepted by OP. – MoonSweep Feb 11 '17 at 1:04
  • 1
    Unfortunately this option doesn't exist in all versions of rynsc. In particular, 3.0.6, which ships with CentOS 6. – jph Jun 15 '17 at 12:51
  • 4
    this option only effect files named in the arguments, those found during recursive search still get warned. – Jasen Jan 18 '18 at 22:40
7

The error means that rsync can't find the files anymore which were existing while building the list to transfer. These vanished-file errors happens when a file was initially found to be present and later is no longer there. In some cases it also happens when the source files are corrupted or have invalid characters in the name (so fsck is advised).

Basically this is a warning, not an error, so nothing to worry about, since the state of each destination file reflects a state that the corresponding source files during the run.

If it is causing problem because of exit value is non-zero, this can be solved by the following wrapper script (source):

#!/bin/bash
(rsync "$@"; if [ $? == 24 ]; then exit 0; else exit $?; fi) 2>&1 | grep -v 'vanished'

or by the following workaround script (source):

#!/bin/sh
OUT=`/usr/bin/snapback2 2>&1`
RET=$?
if [ "$RET" != "23" -a "$RET" != "0" -a "$RET" != 24 ]; then
    echo "$OUT"
    exit $RET
fi

which basically exists with the same error codes as rsync if rsync fails.

This is further discussed in: Bug 3653 - Reduce the need for the "vanished files" warning

6

Or simply (with modern ):

#!/bin/bash

/usr/bin/rsync "$@" 2> >(grep -Ev '(file has |rsync warning: some files )vanished')
ret=$?
((ret==24)) && exit 0 || exit $ret
2
  • 1
    Wouldn't that suppress warning of all kinds of files that contain the string vanished in it also? – rubo77 Mar 29 '16 at 0:43
  • Sure, post edited accordingly – Gilles Quenot Mar 29 '16 at 18:50
2

My answer might be a special use case but I felt it was worth noting, so people don't lose data if they fall under this case.

I was getting that message for several files whilst performing a routine backup of drives via rsync.

I performed a System Check on the disk and it turns out there are issues with the drive (allocation/file corruption etc) and recommended a backup + restore + reformat.

So before you go out an suppress or ignore the message outright, it might be wise to just run a health check on your drive just to be safe.

1
  • Drive failures would not selectively only happen in the tmp directories. – user62916 Mar 3 '18 at 1:39
0

Use this parameter:

--exclude-from="./exclude.ini"

to put the list in a file. In the exclude.ini file write something like:

Cache
cache2/*
*.lock
/temp

Where:

  1. excludes ALL folders named "Cache" and his content;
  2. excludes the content of all "cache2" named folders but include folders hierarchy so you will get all subfolders but empty;
  3. excludes all files terminating with .lock (eg. some lock files of Firefox and derivates);
  4. excludes the content of /temp folder.

Remember that if file system is case-dipendent you may have to account for this.

-1

Try mount your backup disk with nosuid,nodev,nofail,x-gvfs-show options.

Not sure which system you're using but I believe this is related to the mount options of your disk. On Linux this happens if I set the mount options to User Session Default. It resolves when I disable it and rsync completes without any errors.

enter image description here

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.