Python notes - date and time

Syntaxish: syntax and language · changes and py2/3 · decorators · importing, modules, packages · iterable stuff · concurrency IO: networking and web · filesystem Data: Numpy, scipy · pandas, dask · struct, buffer, array, bytes, memoryview · Python database notes Image, Visualization: PIL · Matplotlib, pylab · seaborn · bokeh · plotly Tasky: Concurrency (threads, processes, more) · joblib · pty and pexpect Stringy: strings, unicode, encodings · regexp · command line argument parsing · XML date and time Notebooks speed, memory, debugging, profiling · Python extensions · semi-sorted

Conversions

Some code I've copy-pasted more than once:

from seconds-since-unix-epoch

# to '''datetime object'''
datetime.datetime.fromtimestamp( value ) # int or float

from elapsed seconds

# to timedelta
td = datetime.timedelta(seconds=int_or_float)

from timedelta

### to time difference in seconds
# py>=2.7 added
seconds = timedelta.total_seconds()
# py2.6 and earlier:
seconds = (td.microseconds + (td.seconds + td.days * 24 * 3600) * 10**6) / 10**6

from datetime

WARNING: strftime discards timezone.

# to ISO8601-like string
isostr = dtval.strftime('%Y-%m-%dT%H:%M:%S%z')

### to unix timestamp (float) 
# some variations. If you care about microseconds:
time.mktime(dtval.timetuple()) + (1e-6)*dtval.microsecond

from seconds-since-epoch (a.k.a. unix time, a.k.a. "what time.time() gives")

# to datetime
dt = datetime.datetime.fromtimestamp(int_or_float)


# to ISO8601 style string
isostr = datetime.datetime.fromtimestamp(int_or_float).strftime('%Y-%m-%dT%H:%M:%S%z')

From ISO8601 string

### to datetime
# easier and better than the below, but you have to install/include it
dateutil.parser.parse(s)

# quick and dirty independent hack
def iso8601_dt_hack(s):
    ''' This massages a string for consumption by strptime, 
        which is a "parse this string precisely according to this date string" thing.
          we strip timezone (if present) to make it easier to deal with
          but only sensible within the same timezone.
        If you want to deal with timezones correctly, 
         or want to deal with the compact format at all,
         then you probably want to use dateutil. '''
    d, t = s.split('T',1)
    if '-' in t:
        t=t[:t.index('-')]
    if 'Z' in t:
        t=t[:t.index('Z')]
    if '+' in t:
        t=t[:t.index('+')]
    if '.' in t: # also chops off the above. Separate because I could add fractional-sec handling later                                                                                         
        t=t[:t.index('.')]
    return datetime.datetime.strptime('%sT%s'%(d,t),
                                      "%Y-%m-%dT%H:%M:%S")

from standardish strings, and less standard things

If a specific set of data has a date formatted in a different but completely consistent way, then standard functions like strptime can help (comes from C, but also exposed in python and other higher level languages because it's quite useful).

Consider:

dt = datetime.datetime.strptime( dt_txt, "%Y-%m-%d-%H-%M-%S")

dateutil

When you may get structured-but-not-necessarily-standard strings, or varied free-form strings, then dateutil is nice, as a fallback or in general. (it also has various date-based logic you may not want to do yourself).

You probably want the dateutil.parser.parse() function

For example:

>>> dateutil.parser.parse('2015-06-26 23:00:41')
datetime.datetime(2015, 6, 26, 23, 0, 41)

>>> dateutil.parser.parse("Thu Sep 25 2003")
datetime.datetime(2003, 9, 25, 0, 0)

It seems to match on known patterns so will will work on many commonish/standardish things.

It doesn't like seeing parts it doesn't understand, apparently being cautious. It will be more permissive with fuzzy=True (e.g. it groks apache's date format only with fuzziness, apparently because of the unexpected : between date and time)

On ambiguous dates like 02-04-2012 you may have to guide it, see e.g. [1]

from apache log time

Apache uses date-time-with-timezone format like:

29/Nov/2013:14:21:20 +0100

dateutil.parser.parse with fuzzy=True will deal.

You can do with with a dozen lines of string manipulation, which may be slightly faster(verify) (TODO: add that)

Unsorted

See also:

• http://seehuhn.de/pages/pdate
• Common date formats