Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a cron that is setup to run every Monday at 1 AM

0 1 * * 1 /script/dir/script >> /script/dir/file.log

After two years of running as expected, it ran twice on June 18, with the second call occurring ½ - 1 second after the first. What could cause that to happen?

It wasn't run either the first or second time by a human.

The script is associated with a website but it is outside of the document root.

Is it possible that the clock was synchronized at that same moment, forcing it backwards 1 second and rerunning the cron at 1 AM?

share|improve this question
4  
Automatic clock adjustments with NTP and the like take care not to make the clock go backwards. If they notice that the clock is ahead of the real time, they slow it down until it's right. The first explanation that comes to me is that someone manually set the clock, or has set up a script that forcefully resets the clock. –  Gilles Jul 7 '12 at 23:41
    
I didn't know that about NTP. The server is hosted on AWS, and we certainly don't have a script that modifies the time. And as far as I know, Amazon is just running NTP and the EC2 instances use Xen to sync with the hardware clock. –  JustinY Jul 7 '12 at 23:48
    
What does the cron-logfile say about this? What OS or Linux distribution and version is this? Is cron patched up to date? –  Nils Jul 11 '12 at 20:38
    
@Nils It is using Ubuntu. Cron wasn't setup to have its own logfile so the logs went to system messages which has long since been rotated out and deleted. –  JustinY Jul 11 '12 at 23:33
2  
@Gilles The NTP docs say that NTP only avoids drifting backwards if run with the -x option. See the section titled "How NTP Operates". –  JustinY Jul 13 '12 at 16:36

3 Answers 3

It could be jippie is onto something after all: While the leap second was indeed inserted on July 1st, there's currently a bug in some GPS time systems (please read the full thread), that causes leap seconds to be announced continuously. In other words, the leap second flag has been announced from many top ranking time sources, every day since June 30th. It's being fixed as we speak, again ref the NTP questions mail thread. Numerous systems were hit on July 31st/August 1st due to this, just like on June 30th (Linux kernel crashes) / July 1st (Java CPU issues).

The strange thing is that it hit on June 18th. Did your kernel log have any "Leap second inserted" message around the same time? Is your server currently in UTC+1 (BST)? If you see a leap message in the kernel log, which ntpd version and kernel version are you running? In all ntpd versions I've come across, the leap is only ever propagated to the kernel on the last day of the month, so the entire leap second theory could be a red herring here.

(This should have been a comment, but I haven't gotten 50 reputation yet.)

share|improve this answer

My answer below must be wrong, I confused the date: The most recent leap second was inserted on June 30, 2012 at 23:59:60 UTC. Not June 18th.


The leap second that was introduced that day.

Leap seconds are not as predictable as leap years, they're usually caused due to large earth quakes, tsunami's, ... that sort of thing. Where a leap year is reasonably predictable (about every four years) a leap second is not. There is a commission of bright brained people (I believe at UN) that decide there should be introduced a leap second. This can only be predicted about 60 days in advance. Many systems don't like time to go from ...:59 to ...:60 before going to ...:00. Databases don't like time being set back a second, they're really picky about timestamps occuring only once.

https://en.wikipedia.org/wiki/Leap_second#Announcement_of_leap_seconds

share|improve this answer

I have been getting a double running cron also. Two simultaneous scripts reading and deleting error messages. Kept coming back to this thread to see if anyone had found something new!

But I just realised it was a double up entry of cron. I had an entry in /etc/crontab to run all scripts in /etc/cron.quarter, and I had an entry in /etc/cron.d also calling the same script at the same time.

Not quite as intellectually rigorous as debugging leap second issues!

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.