7

NB: This is a 2-part question: why does an error happen, and how to work around the error. (The two parts are related, inasmuch as the answer to the "why"-question may shed light on the "how"-question.)


An rsync -avuz /some/source/directory . command I'm trying is giving several errors (roughly) of the form

rsync: symlink "/target/directory/foo/bar/baz" -> "../../bar/baz" failed: Read only file system (30)

...where /target/directory is the current directory.

I'm puzzled by this error. It is true that the rsync process does not have permission to write to ../.., but I don't see why it would need to in the first place: it definitely has permission to write to /target/directory/foo/bar/baz.

Be that as it may, in the case at hand, the number of files with this problem represent a very tiny fraction of the total number of files being copied, and therefore it would be fine if those problematic symlinks were "copied to" regular files whose names have been changed in some way to indicate their special nature, and whose contents consist only of the target of the original symlink. For example, the symlink

/some/source/directory/foo/bar/baz -> ../../bar/baz

shown earlier would be "copied to" the regular file

/target/directory/foo/bar/baz-PSEUDOSYMLINK

...consisting of exactly the one line shown below

../../bar/baz

I've seen something similar done when copying Unix directory trees to Windows systems.

Is there a simple way to do this?

1

1 Answer 1

5

Issue

I am using rsync to backup my music collection (/mnt/Music/), via crontab, to an older (hence, sometimes error-prone) hard drive (/mnt/Music_Backups).

Upon manual inspection of that backup partition, I've noticed that these backups sometimes (silently, if run via crontab) fail, as do manual backup attempts:

sudo rsync -aq --delete /mnt/Music/ /mnt/Music_Backups

    rsync: [generator] failed to set times on \
    "/mnt/Music_Backups/ROCK & POP": Read-only file system (30)

Evaluation

Here are my system particulars.

  • Operating system: Arch Linux x_86.
  • rsync version v3.2.3
  • /etc/crontab entry:
# /etc/crontab entry
0 5 * * * root nice -n 19 rsync -aq --delete /mnt/Music/ /mnt/Music_Backups

Verify device, partition, mount point:

lsblk | grep -B1 -i music_back

    sdd      8:48   1   1.8T  0 disk 
    └─sdd1   8:49   1   1.8T  0 part /mnt/Music_Backups

blkid | grep sdd1

    /dev/sdd1: LABEL="Music_Backups" \
    UUID="b90047d3-147c-491c-a17d-17763f79a12a" \
    BLOCK_SIZE="4096" TYPE="ext4" \
    PARTLABEL="Music_Backups" \
    PARTUUID="98425256-c626-41f1-a85e-6f5feaf412d6"

Cause

While the destination drive permissions are OK: I got the same rsync error after doing the following:

sudo chown -R root:victoria /mnt/Music_Backups/

... and the partition looked fine in a disk partition utility (Gparted), the command-line fsck utility indicated that in fact there were errors present:

sudo umount /mnt/Music_Backups/

## Note: if any programs, e.g. a file manager, are using / viewing
## that partition you won't be able to `umount` it until you navigate
## away from anything accessing `/mnt/Music_Backups/`

## Update/CAUTION: do not run fsck on a Network Attached Storage (NAS) 
## hard drive (commonly sold on AMZN ...); these require special 
## software; fsck will kill your HDD.

sudo fsck /dev/sdd1

    fsck from util-linux 2.36
    e2fsck 1.45.6 (20-Mar-2020)
    Music_Backups: recovering journal
    Music_Backups contains a file system with errors, check forced.
    Pass 1: Checking inodes, blocks, and sizes
    Inode 85460709 extent tree (at level 1) could be shorter.  Optimize<y>? yes
    Pass 1E: Optimizing extent trees
    Pass 2: Checking directory structure
    Pass 3: Checking directory connectivity
    Pass 4: Checking reference counts
    Pass 5: Checking group summary information

    Music_Backups: ***** FILE SYSTEM WAS MODIFIED *****
    Music_Backups: 164762/122101760 files (19.8% non-contiguous), 423271337/488378368 blocks

## Recheck:

sudo fsck /dev/sdd1

    fsck from util-linux 2.36
    e2fsck 1.45.6 (20-Mar-2020)
    Music_Backups: clean, 164762/122101760 files, 423271337/488378368 blocks

sudo mount /dev/sdd1 /mnt/Music_Backups/

sudo rsync -aq --delete /mnt/Music/ /mnt/Music_Backups

$ ## completed normally

Conclusion

Based on this, I believe that when my operating system detects errors on a partition, it blocks read-write on that partition until those errors are resolved -- hence blocking rsync from completing its task.


Notes

rsync:

  • -a [archive] : equivalent to -rlptgoD. It is a quick way of saying you want recursion and want to preserve almost everything ... [see https://linux.die.net/man/1/rsync]

  • -q [quiet] : suppress non-error messages


Addendum 1: Notification of Failure

As mentioned, the etc/crontab-scheduled backup silently failed. Here is a solution to that issue (added to /etc/crontab).

DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1000/bus"

# add this as one line (broken here, for readability;
# substitute your user name for "victoria"):

0  5  *  *  *  victoria  nice -n 19
  rsync -aq --delete /mnt/Music/ /mnt/Music_Backups ; EXIT_STATUS=$?;
  if [ $EXIT_STATUS -ne 0 ]; then
  notify-send -i warning -t 0 "/etc/crontab for rsync'd Music Backups failed"
  --icon=dialog-information ;
  mutt -s "/etc/crontab for rsync'd Music Backups failed" <your_email_address>; fi

With this, I in future I should be notified by onscreen notification and email if that backup fails.


Note that there are a few issues when using notify-send and mutt in /etc/crontab:

  • When running as root, as a regular user (e.g. me: "victoria") you will not see the notification; and, you'll need to configure a mutt email account for root (user).

  • It is therefore much easier to run the crontab entry as a regular user. I recursively chown'd my backup location as root:victoria, so running that crontab entry as "victoria" is not an issue.

  • Indicated above, executing notify-send via /etc/crontab requires this line near the top of /etc/crontab:

DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1000/bus"

Execute echo $UID as a regular user [e.g., "victora" (me)] in a terminal to get that number ("1000", above).

For details, see:

With all of that, my crontab entry (above) runs at 5 am daily and notifies me (onscreen; email) if that backup fails for any reason.

You can test those commands with this crontab entry (runs every minute; "apples" is not a program, so guaranteed fail):

*  *  *  *  *  <your_user_name>  nice -n 19
  apples -aq --delete /mnt/Music/ /mnt/Music_Backups ; EXIT_STATUS=$?;
  if [ $EXIT_STATUS -ne 0 ]; then
  notify-send -i warning -t 0 "/etc/crontab for rsync'd Music Backups failed"
  --icon=dialog-information ;
  mutt -s "/etc/crontab for rsync'd Music Backups failed" <your_email_address>; fi

Addendum 2: Notice of Failure

mutt-emailed notification of rsync backup failure (Viewed in Claws Mail)

[victoria@victoria ~]$ date
  Wed Jan  6 08:48:38 AM PST 2021

[victoria@victoria ~]$ lsblk

  NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
  ...
  sdd      8:48   1   1.8T  0 disk 
  └─sdd1   8:49   1   1.8T  0 part /mnt/Music_Backups
  ...

[victoria@victoria ~]$ sudo umount /mnt/Music_Backups/

## Update/CAUTION: do not run fsck on a Network Attached Storage (NAS) 
## hard drive (commonly sold on AMZN ...); these require special 
## software; fsck will kill your HDD.

## Repair disk partition [accept default Y/n suggestions, if present]:

[victoria@victoria ~]$ sudo fsck /dev/sdd1
  fsck from util-linux 2.36.1
  e2fsck 1.45.6 (20-Mar-2020)
  Music_Backups: recovering journal
  Music_Backups contains a file system with errors, check forced.
  Pass 1: Checking inodes, blocks, and sizes
  Pass 2: Checking directory structure
  Pass 3: Checking directory connectivity
  Pass 4: Checking reference counts
  Pass 5: Checking group summary information
  Music_Backups: 165909/122101760 files (20.0% non-contiguous), 425759721/488378368 blocks

## Repeat until no errors:

[victoria@victoria ~]$ sudo fsck /dev/sdd1
  fsck from util-linux 2.36.1
  e2fsck 1.45.6 (20-Mar-2020)
  Music_Backups: clean, 165909/122101760 files, 425759721/488378368 blocks

[victoria@victoria ~]$ sudo mount /dev/sdd1 /mnt/Music_Backups/

## /etc/crontab entry (broken over lines here for readability):

  0    5    *    *    *    victoria    nice -n 19
  rsync -aq --delete /mnt/Music/ /mnt/Music_Backups ; STATUS=$? ;
  if [ $STATUS -ne 0 ]; then notify-send -i warning -t 0 "/etc/crontab
  for rsync'd Music Backups failed" --icon=dialog-information ;
  mutt -s "/etc/crontab for rsync'd Music Backups failed" [email protected] ;
  fi

## Check: manually run Music_Backup command

[victoria@victoria ~]$ time rsync -aq --delete /mnt/Music/ /mnt/Music_Backups
  0:39.91

[victoria@victoria ~]$ 

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .