Python notes - date and time

From Helpful
(Redirected from Datetime)
Jump to navigation Jump to search

Syntaxish: syntax and language · type stuff · changes and py2/3 · decorators · importing, modules, packages · iterable stuff · concurrency · exceptions, warnings


IO: networking and web · filesystem

Data: Numpy, scipy · pandas, dask · struct, buffer, array, bytes, memoryview · Python database notes

Image, Visualization: PIL · Matplotlib, pylab · seaborn · bokeh · plotly


Tasky: Concurrency (threads, processes, more) · joblib · pty and pexpect

Stringy: strings, unicode, encodings · regexp · command line argument parsing · XML

Date and time: date and time


Varied use notes: pytest · pylint


Notebooks

speed, memory, debugging, profiling · Python extensions · semi-sorted


Conversions

Some code I've copy-pasted more than once:


from seconds-since-unix-epoch

# to '''datetime object'''
datetime.datetime.fromtimestamp( value ) # int or float

from elapsed seconds

# to timedelta
td = datetime.timedelta(seconds=int_or_float)

from timedelta

### to time difference in seconds
# py>=2.7 added
seconds = timedelta.total_seconds()
# py2.6 and earlier:
seconds = (td.microseconds + (td.seconds + td.days * 24 * 3600) * 10**6) / 10**6

from datetime

WARNING: strftime discards timezone.

# to ISO8601-like string
isostr = dtval.strftime('%Y-%m-%dT%H:%M:%S%z')

### to unix timestamp (float) 
# some variations. If you care about microseconds:
time.mktime(dtval.timetuple()) + (1e-6)*dtval.microsecond

from seconds-since-epoch (a.k.a. unix time, a.k.a. "what time.time() gives")

# to datetime
dt = datetime.datetime.fromtimestamp(int_or_float)


# to ISO8601 style string
isostr = datetime.datetime.fromtimestamp(int_or_float).strftime('%Y-%m-%dT%H:%M:%S%z')


From ISO8601 string

### to datetime
# easier and better than the below, but you have to install/include it
dateutil.parser.parse(s)

# quick and dirty independent hack
def iso8601_dt_hack(s):
    ''' This massages a string for consumption by strptime, 
        which is a "parse this string precisely according to this date string" thing.
          we strip timezone (if present) to make it easier to deal with
          but only sensible within the same timezone.
        If you want to deal with timezones correctly, 
         or want to deal with the compact format at all,
         then you probably want to use dateutil. '''
    d, t = s.split('T',1)
    if '-' in t:
        t=t[:t.index('-')]
    if 'Z' in t:
        t=t[:t.index('Z')]
    if '+' in t:
        t=t[:t.index('+')]
    if '.' in t: # also chops off the above. Separate because I could add fractional-sec handling later                                                                                         
        t=t[:t.index('.')]
    return datetime.datetime.strptime('%sT%s'%(d,t),
                                      "%Y-%m-%dT%H:%M:%S")

from standardish strings, and less standard things

If a specific set of data has a date formatted in a different but completely consistent way, then standard functions like strptime can help (comes from C, but also exposed in python and other higher level languages because it's quite useful).

Consider:

dt = datetime.datetime.strptime( dt_txt, "%Y-%m-%d-%H-%M-%S")


dateutil

When you may get structured-but-not-necessarily-standard strings, or varied free-form strings, then dateutil is nice, as a fallback or in general. (it also has various date-based logic you may not want to do yourself).

You probably want the dateutil.parser.parse() function

For example:

>>> dateutil.parser.parse('2015-06-26 23:00:41')
datetime.datetime(2015, 6, 26, 23, 0, 41)

>>> dateutil.parser.parse("Thu Sep 25 2003")
datetime.datetime(2003, 9, 25, 0, 0)

It seems to match on known patterns so will will work on many commonish/standardish things.

It doesn't like seeing parts it doesn't understand, apparently being cautious. It will be more permissive with fuzzy=True (e.g. it groks apache's date format only with fuzziness, apparently because of the unexpected : between date and time)

On ambiguous dates like 02-04-2012 (or worse, 02-04-12) you may have to guide it, see e.g. [1]

from apache log time

Apache uses date-time-with-timezone format like:

29/Nov/2013:14:21:20 +0100

dateutil.parser.parse with fuzzy=True will deal.

You can do with with a dozen lines of string manipulation, which may be slightly faster(verify) (TODO: add that)


timedelta

Timezones

python's timezones

This article/section is a stub — some half-sorted notes, not necessarily checked, not necessarily correct. Feel free to ignore, or tell me about it.


Python has two kinds of datetime:

  • datetime without timezone information, also known as timezone-naive datetime
  • datetime with timezone information, also known as timezone-aware


A lot of functions handle either of these, but you cannot mix them.

Which is good, because they aren't conceptually compatible to begin with, and this makes you think about what you need.


A naive datetime is just a "don't know, don't care", but that also means it can never compare or convert to an absolute or local time reference.

Even considering it as 'local time' is a dangerous idea, in that it suggests you personally do still softly implies a timezone, except you're specifically witholding that from the computer.

At best you can add it later, and you better only do it correctly. On that road many accidental mistakes lie.


So consider always going for timezone-aware

Ask yourself whether the thing you are building will ever need to pinpointing a precise time, to people in distinct timezones.

If yes, do everything with timezones.


"You convinced me to always store timezone-aware. How do I do this?"


Any existing datetime value that gets handed to you better have it already, so much of this question is 'how do I correctly create a new datetime with timezone information', which usually means using now()

Annoyingly, now() by default gives you a naive datetime -- this seems backwards compatibility with the very first versions of this.

So supply a timezone.

Yours, UTC, any other, it doesn't really matter.

You can also let it default to your computer's configured timezone (which has to be correct anyway - that is, if your wallclock time is correct but you set the wrong timezone, conversions will be incorrect)


The following three are practically mostly equivalent:

# in UTC:
datetime.datetime.now( tz=datetime.timezone.utc )                 
# in the timezone of the calling PC:
datetime.datetime.now( tz=datetime.timezone.utc ).astimezone()    
# in the timezone of the calling PC:
datetime.datetime.now( ).astimezone()

Notes:

  • These are not strictly equivalent (timezones are cursed), but close enough for most purposes.
  • Either way you depend on the PC's timezone information to be correct - nothing much you can do if it's not.
  • Ignore utcnow(), it's misleading and started to be deprecated in 3.12
the first of those two it's converting from one timezone (given) to another (defaulting to the calling PC's)
the second it's assuming this timezone-naive datetime should be contextualizedin the calling PC's.


Code to play around with:

import datetime
from dateutil import tz

utc = datetime.timezone.utc
buc = tz.gettz('Europe/Bucharest') 

# now() with a timezone effectively asks what time it is in that timezone
print( 'Naive               ',datetime.datetime.now() )
print( 'Now in UTC          ',datetime.datetime.now(tz=utc) )
print( 'Now in Bucharest    ',datetime.datetime.now(tz=buc) )

# These are all 'now', but only the last would be comparable.
#   Demonstrating that these "wallclock with timezone adjustment" _are_ the same time.
#   (The .replace(microsecond=0) is correcting for the fact that these are 
#    distinct now() calls some microseconds apart (and would still fail around second rollover)
#    but you get the idea)
print( datetime.datetime.now(tz=utc).replace(microsecond=0) ==  
       datetime.datetime.now(tz=buc).replace(microsecond=0) )


# For this example, the computer this is called on is in yet another timezone
# which is two hours ahead of UTC,  and one hour behind Bucharest:
print( 'Local with timezone ', datetime.datetime.now(tz=utc).astimezone() )
print( 'Local with timezone ', datetime.datetime.now(      ).astimezone() )
# Note: astimezone() without an argument will fetch the local timezone from the system
#       (and yes, there are ways to get that explicitly)
# To point out this too is again equivalent:
print( datetime.datetime.now(tz=ams).replace(microsecond=0) ==  
       datetime.datetime.now(tz=utc).replace(microsecond=0).astimezone() )

Might print:

Naive                2026-04-02 14:11:35.247848
Now in UTC           2026-04-02 12:11:35.247922+00:00
Now in Bucharest     2026-04-02 15:11:35.247960+03:00
True
Local with timezone  2026-04-02 14:11:35.248120+02:00
Local with timezone  2026-04-02 14:11:35.248230+02:00
True



Thoughts on storing


The choice of which timezone may matter when you want to store it, especially when that has its own rules.

For example, postgres has roughly the same idea as python (see timestamp and timestamptz), which means comparing date within SQL is well defined too.

For example sqlite leaves more up to you, so you want to think about this more, or even consider converting to unix time (which is considered to be referenced to UTC)








pytz

Unsorted

See also: