Why is my EC2 server's time off by ~10 seconds every day?

Question

I have what I believe is a system file, /etc/cron.daily/ntpupdate which runs
ntpdate ntp.ubuntu.com daily to sync with the network time. Every day it generates output very similar to this:

/etc/cron.daily/ntpupdate:
16 Jan 06:30:42 ntpdate[21446]:
step time server 91.189.94.4 offset -12.646804 sec

I'm not positive what the 91.189.94.4 means but I'm pretty sure -12.646804 sec means that my server is off by around 12 seconds. But I don't know why it is off by around the same amount every day. This is an Amazon EC2 instance running Ubuntu.

I can only guess that either it is losing / gaining 12 seconds per day, or something else is syncing the time with another clock that is off by 12 seconds and then I am re-syncing it.

What should I do to try and track this down further? I don't see any other cron jobs in the /etc/cron.* directories or in the users' cron jobs...

UPDATE

Just thought I'd share that I started running this hourly to see if there would be a big jump at a certain hour. This is what the hourly output is:

16 Jan 15:17:04 ntpdate[8346]:
adjust time server 91.189.94.4 offset -0.464418 sec

So apparently every hour the clock is off by around half a second, so that makes sense that each day (24 hours) the clock would be off by around 12 seconds. Guess the clock is just running fast! Thanks!

Answer 1

There are a number of factors that might make a software clock run slow or fast. Clocks on virtual servers are especially prone to a whole class of these problems. 12 seconds a day is pretty bad until you come across virtual boxes with clocks that run at 180–200% speed! Clocks on laptops that suspend can suffer from time-keeping issues too.

You should consider dropping ntupdate in favour of ntpd. The package name is ntp on Debian (and presumably Ubuntu too). The NTP daemon keeps your time in sync a lot more proactively than a cron job, synchronising with one or more other NTP servers and keeping your clock much more accurate. It's another implementation of the same protocol ntpdate uses, except ntpd monitors the time continuously.

If you don't want the (very small) overhead of ntpd, you might consider running ntpdate once an hour. Assuming you're 0.5s off every hour, that should be sufficient.

Answer 2

Answering the other half of your question, on why this is happening: Computer hardware clocks are notoriously inaccurate, so while a drift of 12 seconds in a day is unusual, it's not really all that unusual.

(This is probably because of the prevalence of usage of network time, so that a drift of even 12 seconds a day is a minor annoyance compared to what it would be in a watch -- and thus the hardware companies can use cheap clock chips. Physically, what's happening is probably that the oscillator in your clock chip isn't calibrated quite right, so it runs slightly-but-reliably slow.)

Answer 3

I suspect that ntp does not update your server's time because the difference is too small. I had a similar problem, I saw the same delay every day, until I found out that being a little difference ntp would not update until a specific threshold is present.

Check you configuration for minimum threshold for sync.