0

I just got a new backup drive today, so I went to use it with rsync and everything is completely fine, the backup appears and my system appears fine, but then I went to use apt-get and I got this error:

root@cloud7-media:~# apt-get install sl
W: Not using locking for read only lock file /var/lib/dpkg/lock
E: Unable to write to /var/cache/apt/
E: The package lists or status file could not be parsed or opened.

So I think to myself that its OK and probably just a bug, so I reboot. Then my system wasnt online. I go onto my network panel to check if my system is up and running, and its not. I plug in a monitor and accept to automatically fix changes because problems were detected on the drive, and it's booting again.

This is the rsync code I used:

rsync -aAXv --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found","/BTSync","/home/cloud7/torrent"} / /mnt/backup/cloud7

On a side note, I can confirm rsync caused it, because I ran it again and rebooted the system, and its offline again.

  • If you didn't restore the backup, and the backup is a filesystem mounted from a different machine, then rsync "couldn't" break your system. – Thomas Dickey Sep 25 '16 at 15:43
  • "Unable to write to /var/cache/apt/" You need to find out why your filesystem is being mounted read-only. There is likely an explanation in /var/log/messages (or wherever your distribution puts its messages). Also, run dmseg & look for information. Like untold many others, I use rsync daily and rsync has no history of causing disk problems. – John1024 Sep 25 '16 at 18:31
  • I'm using Ubuntu server, would it be /var/log/syslog? – Noah Farmer Sep 25 '16 at 23:59
  • @NoahFarmer /var/log/syslog sounds good. – John1024 Sep 26 '16 at 2:02
  • Nothing was there, just information about rebooting the system (I did that) – Noah Farmer Sep 26 '16 at 6:49
1

You could try inspecting run-time logs following rsync. The kernel log probably says Remounting filesystem read-only. This happens automatically when encountering errors, even when reading only. The in-memory kernel log is retrieved with dmesg. If using systemd,journalctl -b may also continue working, by use of a tmpfs.

I see other people are commenting about logs as well. To be clear, when an error causes your filesystem to be re-mounted read-only, you can expect there will be no opportunity to write the error message(s) to the log file stored on the filesystem :).

The reason I'm so confident about this is the subsequent "automatically fix changes because problems were detected on the drive".

Also I know rsync can continue in the face of at least some errors, so it wouldn't necessarily have aborted early with a prominent error message. Instead it might end with a generic warning that there was an error transferring some files - I've overlooked this in the past. (Or rsync might not have been affected by the error at all, but I can't think of a situation which would cause this).

Without breaking your system again

Your hard disk is probably faulty

Please check its health using SMART. smartctl -H. Also smartctl -a and look specifically at the counters mentioning sectors. If there are sectors "Pending" or "Uncorrectable", it is strongly recommended to consider the drive faulty.

(Companies building large storage systems write use redundancy over drives, rewrite bad sectors, and write algorithms to guess whether the fault was transient or persistent. It doesn't sound like you're running a redundant system (RAID); in this case the risks of using the drive are generally much greater than any benefit from trying to recover the hardware fault).

| improve this answer | |
  • Making rsync not backup /var fixed it! But thats stupid, because I need to backup my web directory too :P. I heard that if the system is writing to a file, and rsync tries to copy it, it will get corrupted. How would I get around that? – Noah Farmer Sep 26 '16 at 1:33
  • The filesystem would not get corrupted in that case; the filesystem should not get corrupted by any user program simply accessing the filesystem. It would just leave you with inconsistent file contents in the backup copy. If I'm right, what this means is the already-existing errors in your hard disk or filesystem are located in the files in /var. – sourcejedi Sep 26 '16 at 8:52
  • w.r.t consistent system backups, what you're asking for is a snapshot. It should be possible to set up with if you use LVM, but it's annoying and mistakes might be risky... it should be asked as a separate question please :). – sourcejedi Sep 26 '16 at 8:57
  • please post the results of smartctl as suggested, you already have a backup copy of all the files which could be recovered, now you should ask your disk whether it is failing or not, because that is the single most likely cause I can see from what you've said. – sourcejedi Sep 26 '16 at 9:08
  • Sorry for the wait, my smartctl results are in a pastebin link: pastebin.com/YujvzWKp - take a look at my load cycle rate! Way too high, but it was a WD Caviar Green, so it has Intellipark which is honestly the stupidest thing ever. I've disabled that now but does that mean the disk needs replacing? (wd greens are rated at 300,000 load cycles.) – Noah Farmer Sep 27 '16 at 9:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.