Using Future Statements to Prepare for the Future


Note: This article was first published the June 2008 issue of Python Magazine

Mark Mruss

With the release of Python 3.0 only a few months away many Python programmers have visions of compatibility problems dancing in their heads. This article will introduce the concept of future statements including two future statements that you can use to help prepare your code for version 3.0.

Introduction

Python 3.0 (or Python 3000 as many people know it) is something that many Python programmers have started thinking about. Besides being the next major version of our beloved programming language, version 3.0 of Python will break backwards compatibility with the current 2.0 branch of Python. This means that a some of the code that you are writing right now in Python 2.X won’t immediately work in Python 3.0. Since Python 3.0 has a scheduled release date of September 2008 [1] now may be a good time to start thinking about future migration.

Of course there’s no reason to be alarmed yet. Guido, himself, has said that there is no rush to switch over to Python 3.0. [2] According to his PyCon 2008 essay “Python 3000 and You” you should switch when the following are true: “1. You’re ready 2. All your dependencies have been ported.” [3]

In the same essay Guido also says that Python programmers should be prepared and that they should “start writing future-proof” code for 2.5″ [4] In this sprit, this article will introduce “future statements” and two ways that you can use them now in order to make the migration process from the Python 2.0 branch to the Python 3.0 branch as smooth as possible.

The rest of this article, and the examples within, will assume that you are working with Python 2.5.

Import the Future

A very important step in getting your code ready for Python 3.0 is to make use of the __future__ module. The __future__ module allows you to both import changes that are going to be made in future version of Python and use them in your current version of Python.

For example, if feature X is slated to change to feature Y in a future version of Python, there is a chance that it may be supported by the __future__ module. If feature Y is supported by the __future__ module, you can import feature Y from the __future__ module in the version of Python that is still using feature X. This means that instead of having to alter your code when feature X is deprecated and feature Y is implemented, you can write your code now using feature Y and not worry about when the switch will occur.

Future Statements

When you use the __future__ module you will be creating future statements. Future statements, and the purpose of the __future__ module are defined quite well in the Python documentation:

“A future statement is a directive to the compiler that a particular module should be compiled using syntax or semantics that will be available in a specified future release of Python. The future statement is intended to ease migration to future versions of Python that introduce incompatible changes to the language. It allows use of the new features on a per-module basis before the release in which the feature becomes standard.” [5]

Future statements are what will import the new features from the __future__ module. Future statements must be at the top of your module; only comments, blank lines, or other future statements can be located before them.

Let’s take a look at a future statement that will let us work with the changes that will happen to the division operation (explained later):

from __future__ import division

This will change the implementation of the division operation, in whatever module the future statement occurs, from its current integer form to its future float form.

Floating Division

Now that we know what a future statement is and how to use the __future__ module, let’s look at the changes that will be made to the division operation. In the current branch of Python the division operation can be somewhat confusing. It can be integer division (returning the floor of the division, i.e. only the integer quotient) or floating point division (returning the result as a float including the remainder).

Confusion within the current state of Python’s division operation can be seen in the following example:

print 3/2
print 3/2.0

This results in:

1
1.5

Here Python is looking at the type of the operands in the operation. If they are both of type int, integer division will be performed as shown in the first result. If one (or both) of the values is a float, the operation will return a float that will include the remainder (if there is one).

Note that division of this sort (where the result type is dependent on the operands) is common in other programming languages.

While the correct type of division to be performed may be obvious when literal numbers are being used, unexpected results can occur when you are working with variables whose type may be unknown:

def divide_by_three(value):
    return 3/value

This will result in both integer and float return values depending on the type of ‘value’.

As a result of this confusion, in Python 3.0 changes are being made to what the division operation returns. All division using a single slash (/) will return a floating point number including the remainder. In order to perform integer division, you will need to use two slashes (//). The type of division performed now depends on the number of slashes used and not the type of the operands. The only gotcha to be aware of is that if one of the operands in the division operation is a float, the return value will also be a float with 0 as the remainder.

Listing 1 uses a future statement to import the changes to the division operation and replicates our previous division test. If you were to run the code found in Listing 1 the output would be as follows:

1.5
1.0

If you are writing code where you perform division of any sort, I’d suggest importing division from __future__ so that when the move to Python 3.0 occurs, you won’t find yourself with any strange results.

Note that the // division operation has been in Python since version 2.2 so you can already use it without importing division from __future__. Importing division from __future__ changes the functionality of the / division operation.

The future of Imports

In Python 3.0 all imports will be absolute. This means that when you perform an import, you will import “a module or package reachable from sys.path.”[6] While at first glance this may seem to make importing local modules impossible, it is important to remember that when you launch a Python script file directly, the directory where that script file is located will be added to the start of sys.path in position 0.

To illustrate the reason for the change to absolute imports, let’s take a look at a silly example. Let’s say we had a project that used a tokenizer package using the directory tree shown in Figure 1 where string.py is a helper module (with an unfortunate name) that the parse.py module needs to use.

In this example my_proj.py won’t do anything besides import the tokenizer package and create an instance of a Parse object:

import tokenizer

if __name__ == '__main__':
    parser = tokenizer.Parse()

The __init__.py modules in the tokenizer package imports the Parse class from the parse module. This is done so that the Parse class can be accessed as tokenizer.Parse as opposed to tokenizer.parse.Parse:

from parse import Parse

Finally, parse.py is where all of the magic happens:

import string

class Parse(object):
    def __init__(self):
        print(string)

Well it’s not really magic! All that the parse.py file does is import the string module. When a Parse instance is created, a string representation of the string module that was imported is printed out to the command line.

In other words, what happens here is:

  1. my_proj.py imports the tokenizer package, which then imports Parse from parse.py.
  2. my_project.py then creates an instance of the Parse object.
  3. The Parse instance will then print out a string representation of the string module it imported.

When we run my_proj.py, we get the following:

<module 'tokenizer.string' from '/home/project/tokenizer/string.py'>
</module>

As we can see, our local string module was imported by the parse.py module. This is a relative import which means that the “import statement first looks in the containing package before looking in the standard module search path.” [7]

So What’s the Problem?

I’m sure that many of you realize that there is a string module built into Python, and that our string module and the built-in string module share the same name. As long as everything in the tokenizer package only needs the tokenizer.string module, and not the built-in string module, everything will be fine.

But what happens a few months down the road when some new module in the tokenizer package needs to import the built-in string module? Whenever it tries to import string it gets tokenizer.string instead. The obvious solution is to make sure that your module does not share the same name as a built-in module. But this does not fully solve the problem, as we can’t anticipate the names of future modules that will be added to the standard distribution.

Absolute Imports

PEP 328 describes the issue explained in our tokenizer example and the solution that the Python team has put in place:

“In Python 2.4 and earlier, if you’re reading a module located inside a package, it is not clear whether import foo refers to a top-level module or to another module inside the package. As Python’s library expands, more and more existing package internal modules suddenly shadow standard library modules by accident. It’s a particularly difficult problem inside packages because there’s no way to specify which module is meant. To resolve the ambiguity, it is proposed that foo will always be a module or package reachable from sys.path. This is called an absolute
import.” [8]

As stated earlier, once Python gets to version 3.0 all import statements will be absolute imports. We can already use this functionality in Python 2.5 by using a future statement to import absolute_import from the __future__ module:

from __future__ import absolute_import

When we add a future statement to parse.py that imports the absolute_import feature, it will look like the following:

from __future__ import absolute_import
import string

class Parse(object):
    def __init__(self):
        print(string)

When we run my_proj.py, the output will be something similar to the following:

<module 'string' from '/usr/lib/python2.5/string.pyc'></module>

As you can see, enabling absolute imports changes import string from a relative import into an absolute import. This solves our problem when we want import string to import the built-in module, but what about when we want to import the local string module in the tokenizer package? The solution is to use an absolute import to import the local module. The following version of parse.py will import the local string module:

from __future__ import absolute_import
from tokenizer import string

class Parse(object):
    def __init__(self):
        print(string)

We can also use the following absolute import to import the local string module:

from __future__ import absolute_import
import tokenizer.string

class Parse(object):
    def __init__(self):
        print(tokenizer.string)

Relative Imports in an Absolute World

Once you start importing the absolute_import feature, you may still want to use relative imports. Relative imports generally are not needed in smaller packages. But if you have a very large package that contains many sub-packages, you might not want to rewrite your import statements every time the internal structure of the main package changes. Fortunately, it is still possible to perform relative imports in Python 3.0 or with absolute_imports enabled. If you want to perform a relative import, you will need to use the following syntax:

from . import foo

Notice that a dot is used to indicate that the import is a relative import. “A single leading dot indicates a relative import, starting with the current package. Two or more leading dots give a relative import to the parent(s) of the current package, one level per dot after the first.”[9] The “dot syntax” is only available to imports that use the “from” syntax. “import . foo” is not allowed.

To import our local string module in the parse.py module with absolute imports turned on, we do the following:

from __future__ import absolute_import
from . import string

class Parse(object):
    def __init__(self):
        print(string)

Conclusion

I hope I have convinced you that importing division and absolute_imports from the __future__ module is an easy way to add a little “3.0 protection” to your current Python projects. Deciding to make these changes as you create new code, will make migrating your code to Python 3.0 an easier process when the big day comes.

Of course these are not the only changes that you should start making to your code to prepare for Python 3.0. (For example, some of you may have noticed that I was using the print function in this article and not the print statement. Since the print statement is no longer going to be available in Python 3.0, I’m trying to get used to using the print function.) If you want a more detailed description of all the upcoming changes to Python 3.0, and steps you can take to prepare your code now, please see the Python wiki. [10]

You don’t have to worry about Python 3.0 too much right now since the true migration path will be through Python 2.6 and the tools that will be made available. In fact for some people full migration to Python 3.0 may be many years away. But like Guido said it’s best to be prepared and start writing future-proof code now. [11]

[1] http://www.python.org/dev/peps/pep-0361/
[2] http://www.python.org/doc/essays/ppt/pycon2008/Py3kAndYou.pdf
[3] http://www.python.org/doc/essays/ppt/pycon2008/Py3kAndYou.pdf
[4] http://www.python.org/doc/essays/ppt/pycon2008/Py3kAndYou.pdf
[5] http://docs.python.org/ref/future.html
[6] http://www.python.org/dev/peps/pep-0328/#rationale-for-absolute-imports
[7] http://docs.python.org/tut/node8.html#SECTION008420000000000000000
[8] http://www.python.org/dev/peps/pep-0328/#rationale-for-absolute-imports
[9] http://www.python.org/dev/peps/pep-0328/#guido-s-decision
[10] http://wiki.python.org/moin/FutureProofPython
[11] http://www.python.org/doc/essays/ppt/pycon2008/Py3kAndYou.pdf

selsine

del.icio.us del.icio.us

3 Responses to “Using Future Statements to Prepare for the Future”

  1. Rob
    Says:

    Already out

  2. PythonLover
    Says:

    Hi,

    I’m still using evryday “old” Python 2.7 in several projects and I start to see Python 3 everywhere, what is your thought about Python 3 adoption by the community ?

    Should I jump as soon as possible in Python 3 or can I safely stay with the regular Python syntax I already know ?

    Regards

  3. lyxint
    Says:

    hi, where is Figure 1?

Leave a Reply

 

Popular Posts