UNIX Seconds, TAI (SI) Seconds, Leap Seconds, and Real-World Code

Question

I've read a lot about UNIX Time lately, most of it incoherent, much of it contradictory. I am trying to reconcile conversions between UNIX Time (hereafter, for simplicity, UXT), TAI, and UTC, and to do this, I need to understand UXT properly. The trouble is, I can't seem to find anyone else who does.

The following represents my best-attempt explanation, reconstructed from innumerable sources by tedious research. It is also wrong somewhere. I am looking for a holistic analysis and point-by-point verification/refutation of the following. Essentially, fix the following so that it works.

---

1. TAI is a monotonically increasing time standard. It ticks SI seconds, and ignores DST and leap seconds.

2. UTC is the same as TAI, but corrected by an integer number of leap SI seconds (conversions to time strings reflect this as a 60th second) so as to be within 0.9 SI seconds of UT1, an astronomical time standard.

3. UXT is a count of UNIX seconds since 1970-01-01 00:00:00 UTC. There are always exactly 86400 seconds per day. Nevertheless, UXT is related to UTC.

4. How is this possible? Well, the UNIX second needs to be different from the SI second, and because leap seconds are not perfectly regular, UNIX seconds can't be a well-defined length of time.

5. The conversion from UTC to UXT in §4.15 of the UNIX spec aliases different UTC times to the same UXT timestamp, effectively making UNIX seconds the same as SI seconds (except for UNIX leap seconds, which are two SI seconds).
   
   In practice, what actually happens varies. Most computers synchronize based on a remote server, and so they handle leap second updates implicitly during the synchronization.

6. All of this means that, while each individual UXT timestamp can be converted to/from UTC easily (use gmtime or §4.15, respectively), you can't really do arithmetic to find out anything using them. In particular, difftime returns UNIX seconds, and so you can't do anything with it, including adding it to a different timestamp, unless you know where all the relevant leap seconds are.

I think I understand so far.

7. But now we look at actual code, which doesn't do any of this at all. I can understand people measuring durations using difftime and just sortof hoping that it's good enough (or not knowing there's a problem), but timekeeping libraries are wrong too.
   
   As one example, libtai provides a conversion (tai_now.c:7) to TAI from UXT as: TAI := 4,611,686,018,427,387,914 + UXT. Since TAI ticks SI seconds while UXT ticks UNIX seconds, you just can't do this. Yet, since libtai explicitly handles leapseconds, it doesn't seem reasonable that this is a careless mistake.
   
   It's not specific to libtai. You see this sort of thing all over.

So: points 1-6 are in disagreement with point 7. That is, tons of existing code is in contradiction with the time standards it supposedly represents. What went wrong?

Answer 1

A problem is that most documents do not use a vocabulary which can distinguish time scales without sentences of ambiguity. I suggest http://www.ucolick.org/~sla/leapsecs/picktwo.html as introduction to the problem that in historic use there are two unrelated kinds of seconds -- one which is a subdivision of a calendar day for residents of Earth, and one which is a constant duration as measured in a particular reference frame. Any time library which spans dates before and after 1970 is trying to make use of both kinds of second, and that ends up providing answers which are akin to a function that claims to provide arcsin(-2) -- which is to say that there are complexities involved that need careful explanations and definitions of exactly what is being considered important.

Answer 2

Well . . . future you here, writing for a sense of closure.

A few years after you wrote this question, but still a few years before today, you wrote up a huge tutorial that solves this. It constitutes a complete bootstrapping of modern time standards, from TAI through UTC to offsets and UXT.

---

As to your question, it appears you indeed got points 1–6 substantively correct. Your point 7 can just be summed up as "software cannot possibly be so bad at this, right?"

Spoiler alert: it can. The particular complaint about libtai isn't a good example—the library includes leap seconds, but only by composing the appropriate functions, and not just the single function you looked at. This feels like a huge (and demonstrably confusing) deficiency to us, even if it gets the answer correct in the end (who knows if it does), but it's no use harping on it because this sort of thing is incredibly common. Not only does most software actually get it wrong, software gets it wrong in different, incompatible ways!

Heck, the documentation for the fundamental UXT primitive, time(...), isn't even clear. The standard C documentation (ref.), (ref.), which defer to (part of) the POSIX Standard (ref.) say it's a count of seconds since the epoch. But elsewhere in the POSIX standard (ref.), it is not a count of seconds, but instead a count of seconds from the epoch, exclusive of leap seconds, which is the correct characterization. The idea is to make computing things simpler, but it clearly makes it harder because subtraction doesn't work when you introduce discontinuities in your number line. Any program that uses difftime(...), for example, is necessarily incorrect.

The key mental model here is that SI seconds and UNIX seconds are different. This is clearest in yet another place of the POSIX spec. (ref.). This section pegs UXT to UTC by means of a datetime conversion. UTC includes leap seconds, and yet this section also specifies that days are 86 400 seconds long. How can we reconcile this? The key realization is that some UNIX seconds are 2 SI seconds long, and some are 0 (and obviously, the vast majority are 1). Still confused? Go read the tutorial! It is quite unsurprising that there is such a mess with real-world code.

---

You regret you can't point the reader to code. You wrote your own timekeeping library, but it is as complicated as it is powerful, and it's not really production-ready.

I guess, to first order, call time(...) to get the POSIX-epoch-relative count of UNIX seconds, subtract 220 924 790, and add leap seconds (from a table) to get the TAI-epoch-relative count of SI seconds. Do it in reverse to go back. (Note: there's some ambiguity in what the TAI epoch is; I chose 1977-01-01 00:00:00 TAI (i.e., 1976-12-31 23:59:45 UTC) since this was the epoch that standardized TAI, JD_TAI, TCB, TCG, TT, and TDB (ref.).)

In C++20 we will have std::chrono::tai_clock, with epoch 1958-01-01 00:00:00 TAI, (i.e. 1957-12-31 23:59:50 UTC). Hopefully it will be right and we can put this whole sordid business mostly behind us.

Answer 3

You are right, and this mess is terrible. My conclusion:

UXT and UTC are the same, except during the leap second. There, UTC counts to 60 and UTX just 'hangs' for a second.

Answer 4

Best for your mental health is to assume that every day has 86,400 seconds, without exception, and that is good enough for most people. If you take the time right now, and subtract 5,000 times 86,400 seconds, you will get exactly the same hour, minutes and seconds, 5,000 days ago.

Now Posix time is UTC but with the leap seconds removed. That means it's always very close to UT1 (solar time), at most 0.9 seconds away. The best mental model is that every time a leap second is added or removed in UTC, in Posix time we will have either a second that is two seconds long, or a second that is 0 seconds long. You could observe that if you use a stop watch just exactly at the moment a leap second happens - some minute according to Posix would take 61 or 59 seconds according to your stopwatch. Or if your computer displays a clock with seconds, the seconds digit stands still for a second or jumps two seconds at some point. Outside the exact moment the leap seconds are removed you can't observe this. There is no trace in a Posix system of these changes.

When is this wrong? If there is a 100m race, just at the right point in time, and you record start and finish times, someone could finish the race 9.317 seconds after they started, according to Posix time. In reality that would be 10.317 seconds. So if you have physics data then time intervals could be wrong by a second.

How can you fix it? The only way to fix it is by having a table of when leap seconds were inserted, and use this to change your "UTC without leap seconds" to TAI.

Note that Posix time is not really UTC without leap seconds, but your systems best information about what UTC is. You might be able to change the clock on your computer "manually" and Posix time will be off. The clock on your computer may be a bit inaccurate and corrected by a time server, in that case it will drift maybe two seconds a day away from UTC and then be corrected back.

I had an application where this was a problem, and an iOS (probably generally in Posix) there is a sysctl call that returns the boot time. That function returns a changed date when your clock changes apart from the one second every second change. It also returns a changed date when you reboot your computer. So you can store the last known value, and ask a time server when the boot time changes.