Computer dates and times

From Helpful
Jump to: navigation, search
This article/section is a stub — probably a pile of half-sorted notes, is not well-checked so may have incorrect bits. (Feel free to ignore, fix, or tell me)

Broadcasted time synchronization

This article/section is a stub — probably a pile of half-sorted notes, is not well-checked so may have incorrect bits. (Feel free to ignore, fix, or tell me)


  • GPS - almost anywhere (not so easily indoor, though), but is a relatively pricy way of getting just time synchronization


Network time (NTP)

NTP (Network Time Protocol) is a networked way of synchronizing time.


Assuming a well-behaved network, such as a relatively idle broadband connection, you can easily keep the clock correct to within a few milliseconds of the server you point it at, and you can hope for 'within 0.2ms' in many cases.


Standards:

  • NTP v4 is in development
  • RFC 1305, "Network Time Protocol (Version 3) - Specification, Implementation and Analysis" (1992) (current)
  • RFC 1119, ""Network Time Protocol (Version 2) - Specification and Implementation"" (1989) (now obsoleted)
  • RFC-1059, "Network Time Protocol (Version 1) - Specification and Implementation" (1988) (now obsoleted)
  • RFC-958, "Network Time Protocol (NTP)" (1985) (now obsoleted)

There is also SNTP, a simplified, easier to implement variant that can be useful where high-accuracy timing is not necessary. It is seen in embedded devices and such (though simple NTP clients are also seen there).


ntpd (on unices) is a complete implementation of NTPv3


Some terminology

This article/section is a stub — probably a pile of half-sorted notes, is not well-checked so may have incorrect bits. (Feel free to ignore, fix, or tell me)


  • offset (sometimes 'phase')
    • the difference between the reference time and system clock(verify)
    • ...which can only be estimated, as it varies with your networking performance
    • since offset is a relatively instantaneous (non-smoothed) figure, it is not a perfect time adjustment, so you don't want to indiscriminately adjust to every offset you find. This is relevant when considering some behaviour and implementation details, and is major reason behind the slow adjustment. Read the NTP documentation if you want to know all the hairy details.
  • delay - networking round-trip time (a.k.a. ping time)
  • jitter (sometimes 'dispersion') - variation in received/calculated offsets (verify)
  • stratum - the amount of steps from a hardware clock. NTP is an effectively hierarchical system.
    • Stratum 0 are clock devices, which reference from an atomic clock, GPS, CDMA, WWV, DCF77, or such, but are not themselves networked devices.
    • Stratum 1 are the networked servers that are directly connected to stratum 0 devices (implicitly with very stable latency). They are the best networked reference you can get. There aren't many, and not all are public, but it's getting simpler and cheaper to run these yourself.
    • Servers that synchronize with Stratum 1 are themselves stratum 2, those that synch with stratum 2 are stratum 3, etc.
    • The upper limit for stratum is 15. Stratum 16 refers to nodes that a particular node has not synchronized with (or were never synchronized themselves(verify)).


NTP behaviour

This article/section is a stub — probably a pile of half-sorted notes, is not well-checked so may have incorrect bits. (Feel free to ignore, fix, or tell me)




slewing in this context means intentionally tweaking the clock speed a little so that the time will crawl towards the intended reference time.

This is the least disruptive for programs that do timing, since the time value increases monotonously, and the length of intervals while slewing are inaccurate by only a tiny amount.

Ideally, ntpd will only ever need to slew, once it is actively correcting.


stepping refers to just changing the time.

Anything looking at the time sees an instant change forwards or backwards in time. When things have timeouts that now time out early or late, this can cause some odd behaviour.

(does this mean going in small steps, or in one immediate step to the reference time?(verify))




Time-changing behaviour, at startup and while running, depends on the difference to the reference time.


Huge offsets (fifteen minutes) are assumed to be a strange situation that an admin needs to look at, which is why the time needs to be approximately correct for ntpd to even start working.

Servers with huge offsets will never be selected, and if a selected server's offset becomes huge, ntpd will quit.


When looking at servers with small-to-moderate offset, it will take some time to estimate the quality of the time source.

Once (and while) a server is selected, NTP seems to keep a local-to-server offset in mind for a while (rejecting this time) before doing anything (why?(verify)).

Once it wants to correct, ntpd will either step to it (for offsets >128ms), or slew to it (if smaller).

Ideally, once NTP is running, the offset will never become high enough for a step to be necessary, but it can happen.

NTP accuracy

This article/section is a stub — probably a pile of half-sorted notes, is not well-checked so may have incorrect bits. (Feel free to ignore, fix, or tell me)

Stratum-1 hosts can themselves be within ~10 microseconds of UTC, and each stratum above may add on the order of perhaps a millisecond. Exactly how much depends on a bunch of factors, but you should probably assume it's that much.


Since stratum 1 servers are relatively scarce, and each server can only serve so many hosts, it's much easier to find stratum 2 and 3 servers.


You can expect your clock to become correct to within approximately a millisecond of the reference you use. I've seen different values, many within 0.2 to 10ms. The variation comes from...

  • the stratum, because each client can be no more accurate than its server, and the further you are from a reference clock, the more small errors may add up. This is why 2 or 3 is generally preferable over higher strata.
  • network latency (and the jitter in that)
    • NTP tries to be clever about the variable-latency nature of many networks, but it's impossible to completely eliminate the network's effect.
    • network load tends to make network latency and jitter higher. On quiet networks you often get correction to under a millisecond. When you have high latency for long periods, it is possible to accumulate enough offset (order of ~100ms) that it will step your system time when it gets quieter.
  • your clock's instability - though typically this is small enough to be negligible once you have NTP running and actively correcting


Also, it may take hours to days for the offset to be as low and as stable as it can become. (verify)


Basic ntpd setup (unices)

Make sure your system's timezone is correct. Not really a requirement, but probably helpful for you.


Configure the time servers you want to use. See #Servers below. The defaults may be fine.


Start with a system clock that is approximately right, to within a few minutes. This because NTP clients typically refuse to adust when the time difference is more than about fifteen minutes, figuring that a difference this large indicates a problem a person needs to look at.


When you're fine with a sudden largeish jump (such as at the first installation on a server, boot time for devices without a hardware clock) you can force the time correction right now (with the ntpd service not yet running):

ntpd -g -q

(This used to be done via ntpdate, but ntpdate is now considered deprecated. -g allows big changes, -q will quit immediately after instead of running as a daemon.).


You can then start the ntpd service, check that it works, and then feel free to forget about it.


The simplest check that it works is probably to query localhost about its peers, using ntpq:

ntpq -pn localhost

It may take a few minutes to show useful statistics for all peers. (Note that strata 16 means servers we haven't synced with - including servers we cannot contact at all)

Once ntpq shows a
*
in front of one of the servers,

you can be certain ntpd has chosen a server and is actively synchronizing your clock to it. It may take a while for the offset to become as low as it is going to be.

If ntpq gives you:

  • localhost: timed out, nothing received
    , then you probably have an overly strict firewall keeping you from connecting to ntpd (e.g. not trusting localhost, and dropping the packets), though it can also indicate rejection by ntpd itself via its configuration.(verify)
  • ntpq: read: Connection refused
    , this may mean ntpd isn't running (possibly because you started it with a clock more than 15 minutes off), or that it is not configured to allow you.

Servers

This article/section is a stub — probably a pile of half-sorted notes, is not well-checked so may have incorrect bits. (Feel free to ignore, fix, or tell me)

The exchange of time can be done as

  • a client to a listed server (probably the most common)
  • within a network via broadcasts or multicasts
  • between a set of peers that will synchronize with each other (useful for fallback/redundancy)


Assuming you want to pick a server to synchronize with:

You can hand-pick low-stratum servers near you. Or you can be lazy and use the ntp.org pool servers for almost the same effect - ntp.org uses DNS tricks to resolve the names to geographically close hosts. Because they keep track of actual hosts, this means less time worrying about NTP servers that went offline.

In the case of ntpd, having the following is a simple start:

server 0.pool.ntp.org
server 1.pool.ntp.org
server 2.pool.ntp.org
server 3.pool.ntp.org

You can also use specific subdomains to narrow down the options to things that are on the same continent or on the same country. See pages like http://support.ntp.org/bin/view/Servers/NTPPoolServers

Note that these numbers are not strata.


See also

Common date formatting

mm/dd versus dd/mm

The tendency to write things like 02/06/2009 is internationally problematic.

A lot of the world would read it as June 2nd, 2009 (dd/mm/yy), yet the US, Canada and a handful of other nations may easily read it as february 6th, 2009 (mm/dd/yy).

This is only nonambiguous when day is the 13th or later, so it's ambiguous for ~40% of all dates (worse if the year is also 2-digit). So please don't do this.

ISO 8601

ISO 8601, often referred to as ISO date format, is one solution to this last problem. It uses an order previously mostly unused order, and requires a four-digit year. These ISO dates are always formatted YYYY-MM-DD, so identifiable as this format and non-ambiguous.

It is fairly generally accepted. For example, Canada uses it on official documents, and modern programming languages (e.g. .NET date functions) support it out of the box, and various people seem to like its date formatting clarity.

W3C Datetime is a W3 note/overview/profile of the basic and probably most useful parts and uses of ISO 8601.


ISO 8601 allows varied levels of detail:

# dates: years, months, days
YYYY
YYYY-MM
YYYY-MM-DD
#date&time, with time, to minute, to second, or using second with fraction:
YYYY-MM-DDThh:mmTZD      (e.g. 1997-07-16T19:20+01:00)
YYYY-MM-DDThh:mm:ssTZD   (e.g. 1997-07-16T19:20:30+01:00)
YYYY-MM-DDThh:mm:ss.sTZD (e.g. 1997-07-16T19:20:30.45+01:00)
...and, in fact, more, including a compact form like
19940203T141529Z


Notes:

  • The T is a literal T, which signals that a time follows.
    • When displaying these dates, the T is sometimes a space. (Can also be done in storage/communication, if "partners in information interchange" mutually agree)


  • TZD refers to being a Time Zone Designator, and should be one of:
    • Z ('Zulu') meaning UTC
    • +hh:mm
    • -hh:mm

Note that lexical sorting is also useful sorting (except for the timezone information).


RFC 822/1233

For example:

Thu, 11 Oct 07 12:38:29 GMT 
Thu, 11 Oct 2007 12:38:29 GMT 

RFC 822 first specified the format.

RFC 1123 updated this to require the year to always be four-digit (822 allowed 2-digit and 4-digit)


You can leave off the weekday, and timezones can be specified in a number of ways:

  • 4-digit offset: +0330, -0100 (preferred format)
  • pre-defined zones: UT (refers to UTC), GMT, EST, EDT, CST, CDT, MST, MDT, PST, PDT
  • Military: Z for 0, and A-Z except for J

In terms of strftime(): (if you've converted to GMT)

"%a, %d %b %Y %H:%M:%S GMT"

Or, if you're insane enough to like two-digit years

"%a, %d %b %y %H:%M:%S GMT"


RFC 2822

Looks equivalent to RFC1233. (verify)

Mostly a documentation thing because 2822 updates 822, but picks up the definition from 1233.

RFC 850/1036

Looks like (weekday optional):

Sunday, 06 Nov 94 08:49:37 GMT

Defined by RFC 1036 (which obsoletes RFC 850, in which the format was originally defined).

While the format sees relatively little use in standards, it is not uncommon to see real-world dates that should be formatted in the 822 way that look rather like this instead -- in part because this format is valid 822 format as well.

In strftime (assuming you've converted to GMT) (verify)

%A, %d %h %y %H:%M:%S GMT


Observed variation:

  • dashes in the date
  • four-digit year

asctime/ctime

The C library's asctime() and ctime() output: (verify)

Sat May 20 15:21:51 2000
Thu Feb  3 17:03:55 GMT 1994


Common logfile format

common logfile format uses something like:

03/Feb/1994:17:03:55 -0700

More notes

GMT / UTC, daylight savings

Most date formats do not allow for specification of daylight savings details.


This is a good reason reason to use UTC, as daylight savings does not apply to it -- that's the difference between UTC and GMT; daylight savings applies to GMT.

In fact, it is generally better to only ever store date-time in UTC, and only localize when you show it.


The military reference to Zulu time refers to UTC.


(Technically speaking, Universal Time (UT) contains a number of reference dafinitions. The most interesting is UT1 and its practical approximation, UTC.)

Used by...

These are primarily notes
It won't be complete in any sense.
It exists to contain fragments of useful information.


HTTP

HTTP has historically allowed three formats: RFC 822/1233 style, RFC 850/1036 style, and asctime style.

It seems HTTP1.1 says you should only use 822/1233 (it seems to add two details), but be prepared to parse/accept 850/1036.


MIME is also primarily 822/1233, though a stroll through spam will reveal dozens of types of abuse.


.NET seems to use the longest form of ISO8601

See also


  • RFC 3339 'Date and Time on the Internet: Timestamps'


  • RFC 822, 'Standard For The Format Of ARPA Internet Text Messages'
  • RFC 1123, 'Requirements for Internet Hosts - Application and Support'


  • RFC 850, 'Standard for Interchange of USENET Messages'
  • RFC 1036, 'Standard for Interchange of USENET Messages'


Date serialization/storage formats

Text

Human-readable, some are also easily computer-parsed, and some are unambiguous timezonewise (ISO8601 is probably best if you want all of that).

See e.g. #Common date formatting.

Unix time

  • Counting one-second seconds elapsed since the defined epoch, namely January 1, 1970 (UTC)
    • Initially a signed 32-bit number, which will overflow in 2038 (and extends back to ~1902)
    • modern systems are moving to 64-bit, which is epoch ± 293 billion years
    • http://en.wikipedia.org/wiki/Unix_time
    • note that using float means resolution varies with actual time. The resolution drops below a second earlier than the value range (read up on storing integers in floats). If you do this, do it in 64-bit floats (order of a hundred million years before res is sub-second(verify))


FILETIME

Windows FILETIME is a 64-bit int representing 100-nanosecond steps since 1601-01-01T00:00:00Z

So basically

filetime = (unixtime * 10000000) + 116444736000000000
unixtime = filetime/10000000. - 11644473600.


Where

that constant is the amount of nanoseconds between 01-01-1601 and 01-01-1970
you want to consider the details of floats and int64

Semi-sorted

  • Microsoft FILETIME
    • a 64-bit value counting 100-nanosecond intervals since January 1, 1601, UTC.
    • [1]


  • time value in UUIDs
    • a 60-bit time value counting 100-nanosecond intervals since 15 October 1582, midnight, UTC (the date of Gregorian reform)


Unsorted

In strftime, %Z sometimes outputs your current timezone. This may not be what you want.

If you want to output time with a timezone (e.g. for short-term cookies), it's often easiest/laziest to use a "now in GMT" function and hardcode 'GMT' into the string.