Should you run automatic updates

Question

I'm running production Centos web servers. I want to know what is best practice for running updates. Should I automate this with yum-cron or yum-updatesd? Or is there a danger of updates breaking sites so it would be better to update on a test server and then run updates manually weekly?

The majority of the servers are using only the official repositories but some have the atomic repository for PHP modules that are not available otherwise. What's best in this case? can I set yum to only use Atomic for the PHP modules ? I don't want everything updating to the bleeding edge stuff in Atomic, I would trust rather Centos (or really Red Hat) to keep my servers stable and secure.

Answer 1

There are pros and cons to both ways, and you really need to work through your system architecture to know which way is best for you. Whichever route you take, you should understand why you picked that method and what the disadvantages are so that you can compensate for them.

Here are some things to consider:

• Security updates should always be applied one way or another, and sooner rather than later. If your server isn't getting security updates applied in a timely manner for whatever reason, fix your sysadmin habits.

• Distro updates are usually good, especially if they are from official sources. If you feel uncomfortable applying them perhaps you are not using the best distro for your needs. You should be using a distro that matches your methodology. In other words, one that you can trust the updates to be in your best interest. If they move too fast or make software changes that break your stuff, you should probably be using a different distro.

• Updates can break your stuff. A move to newer software could break your code if you used deprecated features or just wrote bad stuff. You should consider a system architecture that allows you to test your product on updates before running them on production systems.

• If you use auto-update features, you should always have a way to roll back to known working configurations. Full system snapshots can be a life saver if some update breaks your product on a production system.

This are not an exhaustive list, just a few things to get you thinking. In the end, this isn't a decision anybody else can make for you. You're the admin. Know your systems. Know your distro.

Answer 2

I agree 100% with what Caleb already said. Some more CentOS-specific notches from me:

The distribution is based on RedHat and its patches. Those patches are tested very good and in the last 4 years where we have been using CentOS 5 with 50+ servers there has never been a bad patch.

BUT: Although we apply all patches daily automatically to servers that run just distribution stuff, we boot production servers (after glibc or kernel update) only after we had our test systems running in that configuration a couple of days.

For other repositories we mirror these to a staging directory. These patches are applied to test-servers first. If it is proven that nothing breaks, we activate the staging directory and copy it over to a production repository.

Automatic patching has the side effect that patched components get often restarted - so if you misconfigured them after startup the restart will fail..