How to Convert Strings to datetime in Python

How to Convert Strings to datetime in Python

Introduction

One of the many common problems that we face in software development is handling dates and times. After getting a date-time string from an API, for example, we need to convert it to a human-readable format. Again, if the same API is used in different timezones, the conversion will be different. A good date-time library should convert the time as per the timezone. This is just one of many nuances that need to be handled when dealing with dates and time.

Thankfully, Python comes with the built-in module datetime for dealing with dates and times. As you probably guessed, it comes with various functions for manipulating dates and times. Using this module, we can easily parse any date-time string and convert it to a datetime object.

Converting Strings Using datetime

The datetime module consists of three different object types: datetime, and datetime. Obviously the date object holds the date, time holds the time, and datetime holds both date and time.

For example, the following code will print the current date and time:

import datetime

print ('Current date/time: {}'.format(datetime.datetime.now()))

Running this code will print something similar to this:

$ python3 datetime-print-1.py
Current date/time: 2018-06-29 08:15:27.243860

When no custom formatting is given, the default string format is used, i.e. the format for “2018-06-29 08:15:27.243860” is in ISO 8601 format (YYYY-MM-DDTHH:MM:SS.mmmmmm). If our input string to create a datetime object is in the same ISO 8601 format, we can easily parse it to a datetime object.

Let’s take a look at the code below:

import datetime

date_time_str = '2018-06-29 08:15:27.243860'
date_time_obj = datetime.datetime.strptime(date_time_str, '%Y-%m-%d %H:%M:%S.%f')

print('Date:', date_time_obj.date())
print('Time:', date_time_obj.time())
print('Date-time:', date_time_obj)

Running it will print the date, time, and date-time:

$ python3 datetime-print-2.py
Date: 2018-06-29
Time: 08:15:27.243860
Date-time: 2018-06-29 08:15:27.243860

In this example, we are using a new method called strptime. This method takes two arguments: the first one is the string representation of the date-time and the second one is the format of the input string. Specifying the format like this makes the parsing much faster since datetime doesn’t need to try and interpret the format on its own, which is much more expensive computationally. The return value is of the type datetime.

In our example, "2018-06-29 08:15:27.243860" is the input string and "%Y-%m-%d %H:%M:%S.%f" is the format of our date string. The returned datetime value is stored in date_time_obj variable. Since this is a datetime object, we can call the date() and time() methods directly on it. As you can see from the output, it prints the ‘date’ and ‘time’ part of the input string.

You might be wondering what is the meaning of the format "%Y-%m-%d %H:%M:%S.%f". These are known as format tokens. Each token represents a different part of the date-time, like day, month, year, etc. Check out the strptime documentation for the list of all different types of format code supported in Python. For a quick reference, here is what we’re using in the code above:

  • %Y: Year (4 digits)
  • %m: Month
  • %d: Day of month
  • %H: Hour (24 hour)
  • %M: Minutes
  • %S: Seconds
  • %f: Microseconds

All of these tokens, except the year, are expected to be zero-padded.

So, if the format of a string is known, it can be easily parsed to a datetime object using strptime. Let me show you one more non-trivial example:

import datetime

date_time_str = 'Jun 28 2018 7:40AM'
date_time_obj = datetime.datetime.strptime(date_time_str, '%b %d %Y %I:%M%p')

print('Date:', date_time_obj.date())
print('Time:', date_time_obj.time())
print('Date-time:', date_time_obj)

From the following output you can see that the string was successfully parsed since it is being properly printed by the datetime object here:

$ python3 datetime-print-3.py
Date: 2018-06-28
Time: 07:40:00
Date-time: 2018-06-28 07:40:00

Here are a few more examples of commonly used time formats and the tokens used for parsing:

"Jun 28 2018 at 7:40AM" -> "%b %d %Y at %I:%M%p"
"September 18, 2017, 22:19:55" -> "%B %d, %Y, %H:%M:%S"
"Sun,05/12/99,12:30PM" -> "%a,%d/%m/%y,%I:%M%p"
"Mon, 21 March, 2015" -> "%a, %d %B, %Y"
"2018-03-12T10:12:45Z" -> "%Y-%m-%dT%H:%M:%SZ"

You can parse a date-time string of any format using the table mentioned in the strptime documentation.

Dealing with Timezones and datetime

Handling date-times becomes more complex while dealing with timezones. All above examples we have discussed are naive datetime objects, i.e. these objects don’t contain any timezone-related data. The datetime object does has one variable that holds the timezone information, tzinfo.

import datetime as dt

dtime = dt.datetime.now()

print(dtime)
print(dtime.tzinfo)

This code will print:

$ python3 datetime-tzinfo-1.py
2018-06-29 22:16:36.132767
None

The output of tzinfo is None since it is a naive datetime object. For timezone conversion, a library called pytz is available for Python. You can install it as described in these instructions. Now, let’s use the pytz library to convert the above timestamp to UTC.

import datetime as dt
import pytz

dtime = dt.datetime.now(pytz.utc)

print(dtime)
print(dtime.tzinfo)

Output:

$ python3 datetime-tzinfo-2.py
2018-06-29 17:08:00.586525+00:00
UTC

+00:00 is the difference between the displayed time and the UTC time. In this example the value of tzinfo happens to be UTC as well, hence the 00:00 offset. In this case, the datetime object is a timezone-aware object.

Similarly, we can convert date-time strings to any other timezone. For example, we can convert the string “2018-06-29 17:08:00.586525+00:00” to “America/New_York” timezone, as shown below:

import datetime as dt
import pytz

date_time_str = '2018-06-29 17:08:00'
date_time_obj = dt.datetime.strptime(date_time_str, '%Y-%m-%d %H:%M:%S')

timezone = pytz.timezone('America/New_York')
timezone_date_time_obj = timezone.localize(date_time_obj)

print(timezone_date_time_obj)
print(timezone_date_time_obj.tzinfo)

Output:

$ python3 datetime-tzinfo-3.py
2018-06-29 17:08:00-04:00
America/New_York

First, we have converted the string to a datetime object, date_time_obj. Then we converted it to a timezone-enabled datetime object, timezone_date_time_obj. Since we have set the timezone as “America/New_York”, the output time shows that it is 4 hours behind than UTC time. You can check this Wikipedia page to find the full list of available time zones.

Converting Timezones

We can convert timezone of a datetime object from one region to another, as shown in the example below:

import datetime as dt
import pytz

timezone_nw = pytz.timezone('America/New_York')
nw_datetime_obj = dt.datetime.now(timezone_nw)

timezone_london = pytz.timezone('Europe/London')
london_datetime_obj = nw_datetime_obj.astimezone(timezone_london)


print('America/New_York:', nw_datetime_obj)
print('Europe/London:', london_datetime_obj)

First, we created one datetime object with the current time and set it as the “America/New_York” timezone. Then using the astimezone() method, we have converted this datetime to “Europe/London” timezone. Both datetimes will print different values like:

$ python3 datetime-tzinfo-4.py
America/New_York: 2018-06-29 22:21:41.349491-04:00
Europe/London: 2018-06-30 03:21:41.349491+01:00

As expected, the date-times are different since they’re about 5 hours apart.

Using Third Party Libraries

Python’s datetime module can convert all different types of strings to a datetime object. But the main problem is that in order to do this you need to create the appropriate formatting code string that strptime can understand. Creating this string takes time and it makes the code harder to read. Instead, we can use other third-party libraries to make it easier.

In some cases these third-party libraries also have better built-in support for manipulating and comparing date-times, and some even have timezones built-in, so you don’t need to include an extra package.

Let’s take a look at few of these libraries in the following sections.

dateutil

The dateutil module is an extension to the datetime module. One advantage is that we don’t need to pass any parsing code to parse a string. For example:

from dateutil.parser import parse

datetime = parse('2018-06-29 22:21:41')

print(datetime)

This parse function will parse the string automatically and store it in the datetime variable. Parsing is done automatically. You don’t have to mention any format string. Let’s try to parse different types of strings using dateutil:

from dateutil.parser import parse

date_array = [
    '2018-06-29 08:15:27.243860',
    'Jun 28 2018 7:40AM',
    'Jun 28 2018 at 7:40AM',
    'September 18, 2017, 22:19:55',
    'Sun, 05/12/1999, 12:30PM',
    'Mon, 21 March, 2015',
    '2018-03-12T10:12:45Z',
    '2018-06-29 17:08:00.586525+00:00',
    '2018-06-29 17:08:00.586525+05:00',
    'Tuesday , 6th September, 2017 at 4:30pm'
]

for date in date_array:
    print('Parsing: ' + date)
    dt = parse(date)
    print(dt.date())
    print(dt.time())
    print(dt.tzinfo)
    print('n')

Output:

$ python3 dateutil-1.py
Parsing: 2018-06-29 08:15:27.243860
2018-06-29
08:15:27.243860
None

Parsing: Jun 28 2018 7:40AM
2018-06-28
07:40:00
None

Parsing: Jun 28 2018 at 7:40AM
2018-06-28
07:40:00
None

Parsing: September 18, 2017, 22:19:55
2017-09-18
22:19:55
None

Parsing: Sun, 05/12/1999, 12:30PM
1999-05-12
12:30:00
None

Parsing: Mon, 21 March, 2015
2015-03-21
00:00:00
None

Parsing: 2018-03-12T10:12:45Z
2018-03-12
10:12:45
tzutc()

Parsing: 2018-06-29 17:08:00.586525+00:00
2018-06-29
17:08:00.586525
tzutc()

Parsing: 2018-06-29 17:08:00.586525+05:00
2018-06-29
17:08:00.586525
tzoffset(None, 18000)

Parsing: Tuesday , 6th September, 2017 at 4:30pm
2017-09-06
16:30:00
None

You can see that almost any type of string can be parsed easily using the dateutil module.

While this is convenient, recall from earlier that having to predict the format makes the code much slower, so if you’re code requires high performance then this might not be the right approach for your application.

Maya

Maya also makes it very easy to parse a string and for changing timezones. Some simple examples are shown here:

import maya

dt = maya.parse('2018-04-29T17:45:25Z').datetime()
print(dt.date())
print(dt.time())
print(dt.tzinfo)

Output:

$ python3 maya-1.py
2018-04-29
17:45:25
UTC

For converting the time to a different timezone:

import maya

dt = maya.parse('2018-04-29T17:45:25Z').datetime(to_timezone='America/New_York', naive=False)
print(dt.date())
print(dt.time())
print(dt.tzinfo)

Output:

$ python3 maya-2.py
2018-04-29
13:45:25
America/New_York

Now isn’t that easy to use? Let’s try out maya with the same set of strings we have used with dateutil:

import maya

date_array = [
    '2018-06-29 08:15:27.243860',
    'Jun 28 2018 7:40AM',
    'Jun 28 2018 at 7:40AM',
    'September 18, 2017, 22:19:55',
    'Sun, 05/12/1999, 12:30PM',
    'Mon, 21 March, 2015',
    '2018-03-12T10:12:45Z',
    '2018-06-29 17:08:00.586525+00:00',
    '2018-06-29 17:08:00.586525+05:00',
    'Tuesday , 6th September, 2017 at 4:30pm'
]

for date in date_array:
    print('Parsing: ' + date)
    dt = maya.parse(date).datetime()
    print(dt)
    print(dt.date())
    print(dt.time())
    print(dt.tzinfo)

Output:

$ python3 maya-3.py
Parsing: 2018-06-29 08:15:27.243860
2018-06-29 08:15:27.243860+00:00
2018-06-29
08:15:27.243860
UTC

Parsing: Jun 28 2018 7:40AM
2018-06-28 07:40:00+00:00
2018-06-28
07:40:00
UTC

Parsing: Jun 28 2018 at 7:40AM
2018-06-28 07:40:00+00:00
2018-06-28
07:40:00
UTC

Parsing: September 18, 2017, 22:19:55
2017-09-18 22:19:55+00:00
2017-09-18
22:19:55
UTC

Parsing: Sun, 05/12/1999, 12:30PM
1999-05-12 12:30:00+00:00
1999-05-12
12:30:00
UTC

Parsing: Mon, 21 March, 2015
2015-03-21 00:00:00+00:00
2015-03-21
00:00:00
UTC

Parsing: 2018-03-12T10:12:45Z
2018-03-12 10:12:45+00:00
2018-03-12
10:12:45
UTC

Parsing: 2018-06-29 17:08:00.586525+00:00
2018-06-29 17:08:00.586525+00:00
2018-06-29
17:08:00.586525
UTC

Parsing: 2018-06-29 17:08:00.586525+05:00
2018-06-29 12:08:00.586525+00:00
2018-06-29
12:08:00.586525
UTC

Parsing: Tuesday , 6th September, 2017 at 4:30pm
2017-09-06 16:30:00+00:00
2017-09-06
16:30:00
UTC

As you can see, all of the date formats were successfully parsed.

But did you notice the difference? If we are not providing the timezone info then it automatically converts it to UTC. So, it is important to note that we must provide to_timezone and naive parameters if the time is not in UTC.

Arrow

Arrow is another library for dealing with datetime in Python. And like before with maya, it also figures out the datetime format automatically. Once interpreted, it returns a Python datetime object from the arrow object.

Let’s try this with the same example string we have used for maya:

import arrow

dt = arrow.get('2018-04-29T17:45:25Z')
print(dt.date())
print(dt.time())
print(dt.tzinfo)

Output:

$ python3 arrow-1.py
2018-04-29
17:45:25
tzutc()

And here is how you can use arrow to convert timezones using the to method:

import arrow

dt = arrow.get('2018-04-29T17:45:25Z').to('America/New_York')
print(dt)
print(dt.date())
print(dt.time())

Output:

$ python3 arrow-2.py
2018-04-29T13:45:25-04:00
2018-04-29
13:45:25

As you can see the date-time string is converted to the “America/New_York” region.

Now, let’s again use the same set of strings we have used above:

import arrow

date_array = [
    '2018-06-29 08:15:27.243860',
    #'Jun 28 2018 7:40AM',
    #'Jun 28 2018 at 7:40AM',
    #'September 18, 2017, 22:19:55',
    #'Sun, 05/12/1999, 12:30PM',
    #'Mon, 21 March, 2015',
    '2018-03-12T10:12:45Z',
    '2018-06-29 17:08:00.586525+00:00',
    '2018-06-29 17:08:00.586525+05:00',
    #'Tuesday , 6th September, 2017 at 4:30pm'
]

for date in date_array:
    dt = arrow.get(date)
    print('Parsing: ' + date)
    print(dt)
    print(dt.date())
    print(dt.time())
    print(dt.tzinfo)

This code will fail for the date-time strings that have been commented out, which is over half of our examples. The output for other strings will be:

$ python3 arrow-3.py
Parsing: 2018-06-29 08:15:27.243860
2018-06-29T08:15:27.243860+00:00
2018-06-29
08:15:27.243860
tzutc()

Parsing: 2018-03-12T10:12:45Z
2018-03-12T10:12:45+00:00
2018-03-12
10:12:45
tzutc()

Parsing: 2018-06-29 17:08:00.586525+00:00
2018-06-29T17:08:00.586525+00:00
2018-06-29
17:08:00.586525
tzoffset(None, 0)

Parsing: 2018-06-29 17:08:00.586525+05:00
2018-06-29T17:08:00.586525+05:00
2018-06-29
17:08:00.586525
tzoffset(None, 18000)

In order to correctly parse the date-time strings that I have commented out, you’ll need to pass the corresponding format tokens to give the library clues as to how to parse it. For example, “MMM” for months name, like “Jan, Feb, Mar” etc. You can check this guide for all available tokens.

Conclusion

In this article we have shown different ways to parse a string to a datetime object in Python. You can either opt for the default Python datetime library or any of the third-party libraries mentioned in this article, among many others.

The main problem with the default datetime package is that we need to specify the parsing code manually for almost all date-time string formats. So, if your string format changes in the future, you will likely have to change your code as well. But many third-party libraries, like the ones mentioned here, handle it automatically.

 

Python Example for Beginners

Two Machine Learning Fields

There are two sides to machine learning:

  • Practical Machine Learning:This is about querying databases, cleaning data, writing scripts to transform data and gluing algorithm and libraries together and writing custom code to squeeze reliable answers from data to satisfy difficult and ill defined questions. It’s the mess of reality.
  • Theoretical Machine Learning: This is about math and abstraction and idealized scenarios and limits and beauty and informing what is possible. It is a whole lot neater and cleaner and removed from the mess of reality.

 

Data Science Resources: Data Science Recipes and Applied Machine Learning Recipes

Introduction to Applied Machine Learning & Data Science for Beginners, Business Analysts, Students, Researchers and Freelancers with Python & R Codes @ Western Australian Center for Applied Machine Learning & Data Science (WACAMLDS) !!!

Latest end-to-end Learn by Coding Recipes in Project-Based Learning:

Applied Statistics with R for Beginners and Business Professionals

Data Science and Machine Learning Projects in Python: Tabular Data Analytics

Data Science and Machine Learning Projects in R: Tabular Data Analytics

Python Machine Learning & Data Science Recipes: Learn by Coding

R Machine Learning & Data Science Recipes: Learn by Coding

Comparing Different Machine Learning Algorithms in Python for Classification (FREE)

Disclaimer: The information and code presented within this recipe/tutorial is only for educational and coaching purposes for beginners and developers. Anyone can practice and apply the recipe/tutorial presented here, but the reader is taking full responsibility for his/her actions. The author (content curator) of this recipe (code / program) has made every effort to ensure the accuracy of the information was correct at time of publication. The author (content curator) does not assume and hereby disclaims any liability to any party for any loss, damage, or disruption caused by errors or omissions, whether such errors or omissions result from accident, negligence, or any other cause. The information presented here could also be found in public knowledge domains.  

Google –> SETScholars