Just a quick note that MemeToad 1.0.5 has been released. It now has over 100 images included and also automatically scales the font based on the image and the amount of text entered.

Please let me know what you think of the app and if you like it don’t forget to leave a review. Thanks!

Today I released my newest Android App: MemeToad it’s a meme generator for your Android phone and/or tablet. I’ve been working on this app for the last while in the evenings.

It’s tough to find the time when you’ve got a family and a full-time job but I’m happy to have stick with it and put it out there.

If you have an Android device and you are into meme’s check it out it’s free. I know it’s not Python related, but it’s all I’ve got.

Cheers.

Well it’s been over a year and there have been no updates to this blog. Sorry about that. I have been very busy with my family (my wife and I had another baby) and my day to day work. (Why is it that no one every told me how much of my free time would be spent with my young children? Rewarding? Yes. Time consuming? Yes.)

I do have something to announce to anyone that still reads this blog, and anyone that may be doing Android development. We have recently launched AirBop a GCM based push notification service for Android apps.

I won’t spend too much time talking about it, instead I’ll let the features page do that for me, but I will tell you that it’s a great way to add push notifications to your app. We’ve been working really hard on AirBop and we’re very proud of it. If you are an Android developer and you do check it out (free to sign up and register your first 1000 devices, no charge, no credit card, etc.) please let me know what you think. I can answer any questions about the service that you want.

We’ve also got a sample client up on GitHub: airbop-client that can help get you started with the implementation details.

Hey Everyone, or anyone that’s still around sorry it’s been so quiet around here lately, I’ve just had so much on my plate. Moving, baby, wife, work, a new homebrewing passion, reading, writing, and so on. All of this has led to very little time spent on this blog.

I have some things in mind for this blog, but I never seem to get around to writing the posts or doing the work. Hopefully I’ll be able to get back into it soon.

At work we’ve delved into the Android platform and created an online app builder called andromo, which means I’ve been writing a lot of Java and hanging out on smart phones and tablets. If you are interested you can see some of the apps that have been built and added to the Android Market in our showcase.

You can keep up with what I’m doing on twitter: @MarkAtAndromo or add me to a circle on google+ under Mark Mruss and yes I follow Guido. If you are not on Google plus and need an invite let me know in the comments.

Oh and I also write on the Andromo blog from time to time as well.

Hey everyone, just a quick note to let you know that Bohdan has translated the Creating a GUI using PyGTK and Glade tutorial in to Belorussian/Belarusian. You can view the translation here: Belorussian translation.

Many thanks to Bohdan and all the other who have translated some of these tutorials over the years.

Note: This article was first published the June 2008 issue of Python Magazine

Mark Mruss

With the release of Python 3.0 only a few months away many Python programmers have visions of compatibility problems dancing in their heads. This article will introduce the concept of future statements including two future statements that you can use to help prepare your code for version 3.0.

Introduction

Python 3.0 (or Python 3000 as many people know it) is something that many Python programmers have started thinking about. Besides being the next major version of our beloved programming language, version 3.0 of Python will break backwards compatibility with the current 2.0 branch of Python. This means that a some of the code that you are writing right now in Python 2.X won’t immediately work in Python 3.0. Since Python 3.0 has a scheduled release date of September 2008 [1] now may be a good time to start thinking about future migration.

Of course there’s no reason to be alarmed yet. Guido, himself, has said that there is no rush to switch over to Python 3.0. [2] According to his PyCon 2008 essay “Python 3000 and You” you should switch when the following are true: “1. You’re ready 2. All your dependencies have been ported.” [3]

In the same essay Guido also says that Python programmers should be prepared and that they should “start writing future-proof” code for 2.5″ [4] In this sprit, this article will introduce “future statements” and two ways that you can use them now in order to make the migration process from the Python 2.0 branch to the Python 3.0 branch as smooth as possible.

The rest of this article, and the examples within, will assume that you are working with Python 2.5.

Import the Future

A very important step in getting your code ready for Python 3.0 is to make use of the __future__ module. The __future__ module allows you to both import changes that are going to be made in future version of Python and use them in your current version of Python.

For example, if feature X is slated to change to feature Y in a future version of Python, there is a chance that it may be supported by the __future__ module. If feature Y is supported by the __future__ module, you can import feature Y from the __future__ module in the version of Python that is still using feature X. This means that instead of having to alter your code when feature X is deprecated and feature Y is implemented, you can write your code now using feature Y and not worry about when the switch will occur.

Future Statements

When you use the __future__ module you will be creating future statements. Future statements, and the purpose of the __future__ module are defined quite well in the Python documentation:

“A future statement is a directive to the compiler that a particular module should be compiled using syntax or semantics that will be available in a specified future release of Python. The future statement is intended to ease migration to future versions of Python that introduce incompatible changes to the language. It allows use of the new features on a per-module basis before the release in which the feature becomes standard.” [5]

Future statements are what will import the new features from the __future__ module. Future statements must be at the top of your module; only comments, blank lines, or other future statements can be located before them.

Let’s take a look at a future statement that will let us work with the changes that will happen to the division operation (explained later):

from __future__ import division

This will change the implementation of the division operation, in whatever module the future statement occurs, from its current integer form to its future float form.

Floating Division

Now that we know what a future statement is and how to use the __future__ module, let’s look at the changes that will be made to the division operation. In the current branch of Python the division operation can be somewhat confusing. It can be integer division (returning the floor of the division, i.e. only the integer quotient) or floating point division (returning the result as a float including the remainder).

Confusion within the current state of Python’s division operation can be seen in the following example:

print 3/2
print 3/2.0

This results in:

1
1.5


Here Python is looking at the type of the operands in the operation. If they are both of type int, integer division will be performed as shown in the first result. If one (or both) of the values is a float, the operation will return a float that will include the remainder (if there is one).

Note that division of this sort (where the result type is dependent on the operands) is common in other programming languages.

While the correct type of division to be performed may be obvious when literal numbers are being used, unexpected results can occur when you are working with variables whose type may be unknown:

def divide_by_three(value):
    return 3/value

This will result in both integer and float return values depending on the type of ‘value’.

As a result of this confusion, in Python 3.0 changes are being made to what the division operation returns. All division using a single slash (/) will return a floating point number including the remainder. In order to perform integer division, you will need to use two slashes (//). The type of division performed now depends on the number of slashes used and not the type of the operands. The only gotcha to be aware of is that if one of the operands in the division operation is a float, the return value will also be a float with 0 as the remainder.

Listing 1 uses a future statement to import the changes to the division operation and replicates our previous division test. If you were to run the code found in Listing 1 the output would be as follows:

1.5
1.0


If you are writing code where you perform division of any sort, I’d suggest importing division from __future__ so that when the move to Python 3.0 occurs, you won’t find yourself with any strange results.

Note that the // division operation has been in Python since version 2.2 so you can already use it without importing division from __future__. Importing division from __future__ changes the functionality of the / division operation.

The future of Imports

In Python 3.0 all imports will be absolute. This means that when you perform an import, you will import “a module or package reachable from sys.path.”[6] While at first glance this may seem to make importing local modules impossible, it is important to remember that when you launch a Python script file directly, the directory where that script file is located will be added to the start of sys.path in position 0.

To illustrate the reason for the change to absolute imports, let’s take a look at a silly example. Let’s say we had a project that used a tokenizer package using the directory tree shown in Figure 1 where string.py is a helper module (with an unfortunate name) that the parse.py module needs to use.

In this example my_proj.py won’t do anything besides import the tokenizer package and create an instance of a Parse object:

import tokenizer

if __name__ == '__main__':
    parser = tokenizer.Parse()

The __init__.py modules in the tokenizer package imports the Parse class from the parse module. This is done so that the Parse class can be accessed as tokenizer.Parse as opposed to tokenizer.parse.Parse:

from parse import Parse

Finally, parse.py is where all of the magic happens:

import string

class Parse(object):
    def __init__(self):
        print(string)

Well it’s not really magic! All that the parse.py file does is import the string module. When a Parse instance is created, a string representation of the string module that was imported is printed out to the command line.

In other words, what happens here is:

1. my_proj.py imports the tokenizer package, which then imports Parse from parse.py.
2. my_project.py then creates an instance of the Parse object.
3. The Parse instance will then print out a string representation of the string module it imported.

When we run my_proj.py, we get the following:

<module 'tokenizer.string' from '/home/project/tokenizer/string.py'>
</module>

As we can see, our local string module was imported by the parse.py module. This is a relative import which means that the “import statement first looks in the containing package before looking in the standard module search path.” [7]

So What’s the Problem?

I’m sure that many of you realize that there is a string module built into Python, and that our string module and the built-in string module share the same name. As long as everything in the tokenizer package only needs the tokenizer.string module, and not the built-in string module, everything will be fine.

But what happens a few months down the road when some new module in the tokenizer package needs to import the built-in string module? Whenever it tries to import string it gets tokenizer.string instead. The obvious solution is to make sure that your module does not share the same name as a built-in module. But this does not fully solve the problem, as we can’t anticipate the names of future modules that will be added to the standard distribution.

Absolute Imports

PEP 328 describes the issue explained in our tokenizer example and the solution that the Python team has put in place:

“In Python 2.4 and earlier, if you’re reading a module located inside a package, it is not clear whether import foo refers to a top-level module or to another module inside the package. As Python’s library expands, more and more existing package internal modules suddenly shadow standard library modules by accident. It’s a particularly difficult problem inside packages because there’s no way to specify which module is meant. To resolve the ambiguity, it is proposed that foo will always be a module or package reachable from sys.path. This is called an absolute
import.” [8]

As stated earlier, once Python gets to version 3.0 all import statements will be absolute imports. We can already use this functionality in Python 2.5 by using a future statement to import absolute_import from the __future__ module:

from __future__ import absolute_import

When we add a future statement to parse.py that imports the absolute_import feature, it will look like the following:

from __future__ import absolute_import
import string

class Parse(object):
    def __init__(self):
        print(string)

When we run my_proj.py, the output will be something similar to the following:

<module 'string' from '/usr/lib/python2.5/string.pyc'></module>

As you can see, enabling absolute imports changes import string from a relative import into an absolute import. This solves our problem when we want import string to import the built-in module, but what about when we want to import the local string module in the tokenizer package? The solution is to use an absolute import to import the local module. The following version of parse.py will import the local string module:

from __future__ import absolute_import
from tokenizer import string

class Parse(object):
    def __init__(self):
        print(string)

We can also use the following absolute import to import the local string module:

from __future__ import absolute_import
import tokenizer.string

class Parse(object):
    def __init__(self):
        print(tokenizer.string)

Relative Imports in an Absolute World

Once you start importing the absolute_import feature, you may still want to use relative imports. Relative imports generally are not needed in smaller packages. But if you have a very large package that contains many sub-packages, you might not want to rewrite your import statements every time the internal structure of the main package changes. Fortunately, it is still possible to perform relative imports in Python 3.0 or with absolute_imports enabled. If you want to perform a relative import, you will need to use the following syntax:

from . import foo

Notice that a dot is used to indicate that the import is a relative import. “A single leading dot indicates a relative import, starting with the current package. Two or more leading dots give a relative import to the parent(s) of the current package, one level per dot after the first.”[9] The “dot syntax” is only available to imports that use the “from” syntax. “import . foo” is not allowed.

To import our local string module in the parse.py module with absolute imports turned on, we do the following:

from __future__ import absolute_import
from . import string

class Parse(object):
    def __init__(self):
        print(string)

Conclusion

I hope I have convinced you that importing division and absolute_imports from the __future__ module is an easy way to add a little “3.0 protection” to your current Python projects. Deciding to make these changes as you create new code, will make migrating your code to Python 3.0 an easier process when the big day comes.

Of course these are not the only changes that you should start making to your code to prepare for Python 3.0. (For example, some of you may have noticed that I was using the print function in this article and not the print statement. Since the print statement is no longer going to be available in Python 3.0, I’m trying to get used to using the print function.) If you want a more detailed description of all the upcoming changes to Python 3.0, and steps you can take to prepare your code now, please see the Python wiki. [10]

You don’t have to worry about Python 3.0 too much right now since the true migration path will be through Python 2.6 and the tools that will be made available. In fact for some people full migration to Python 3.0 may be many years away. But like Guido said it’s best to be prepared and start writing future-proof code now. [11]

[1] http://www.python.org/dev/peps/pep-0361/
[2] http://www.python.org/doc/essays/ppt/pycon2008/Py3kAndYou.pdf
[3] http://www.python.org/doc/essays/ppt/pycon2008/Py3kAndYou.pdf
[4] http://www.python.org/doc/essays/ppt/pycon2008/Py3kAndYou.pdf
[5] http://docs.python.org/ref/future.html
[6] http://www.python.org/dev/peps/pep-0328/#rationale-for-absolute-imports
[7] http://docs.python.org/tut/node8.html#SECTION008420000000000000000
[8] http://www.python.org/dev/peps/pep-0328/#rationale-for-absolute-imports
[9] http://www.python.org/dev/peps/pep-0328/#guido-s-decision
[10] http://wiki.python.org/moin/FutureProofPython
[11] http://www.python.org/doc/essays/ppt/pycon2008/Py3kAndYou.pdf

The results are in for the first annual LearningPython.com Python version quiz:

By a landslide version 2.6 is the winner, with 3.1 and 2.5 following far behind. A grand total of 1084 people voted and 700 of those still use Python 2.6, 187 use 3.1 and 182 use 2.5. While not the largest sampling of users the wide margin of victory probably means that most Python programmers are still targeting 2.6, or at the very least the newest version of the 2.x branch. Now what is installed on our end user’s system…that’s another matter all together.

Thanks again to everyone that voted. I’m happy that I was able to attract 1000+ people to vote in this poll. My next idea for a poll is an IDE poll, partially because I’m curious as to what people are using, and partially because there is a good chance that I may have overlooked other programs on my way to selecting Geany.

Note: Python 2.7 was released (July 3rd, 2010) after the poll was created (Mach 4, 2010) this means that the value for 2.7 is probably larger now and 2.6 is probably slightly smaller.

Hey Everyone, just an update on the last post. I was able to dig through the wordpress files and find where the hack was and it appears to be fixed now.

Again if you notice anything strange happening here, redirects to external sites and whatnot, please let me know.

Also, does anyone here use any of the python web frameworks? I’d like to try my hand at them, but I’m having difficulty deciding between Pylons, Django, and Zope. I’m a total n00b when it comes to web programming, so it’s something that I’d like to be better at, so one of my requirements is being easy for beginners to learn.

If anyone has any thoughts or advice please post a comment.

Just to let everyone who still reads this blog that it looks as though I have been hacked again.

Please don’t believe the rumours, I am not selling, nor will I ever, levitra online.

If you notice anything strange in the text on this blog, or you end up redirected to some strange site, please let me know.

Note: This article was first published the May 2008 issue of Python Magazine

Introducing Descriptors and Properties

Mark Mruss

New-style classes were introduced to Python with the release of Python 2.2. And with these new-style classes came descriptors and properties. This article will introduce the descriptor protocol, descriptors, and properties.

Introduction

New-style classes were introduced to Python with the release of Python 2.2. A new-style class is any class that is derived from the object base class. New-style classes give Python programmers many new (and initially confusing) features. One such feature is the descriptor protocol, and more specifically descriptors themselves.

Descriptors give Python programmers the ability to easily and efficiently create “managed attributes”. Managed attributes can be thought of as attributes that are not accessed directly. Instead their access is “managed” by something else, generally a class or a function.

If you haven’t come across this before you are probably wondering why one would want to manage attribute access? One reason might be that you don’t want people to be able to delete the attribute. Another reason may be that you need to ensure that your attribute data is always valid. Or perhaps attribute x is based on attribute y, so every time the value of y changes you want to update the value of x. From these few examples you can see the many possible cases where you might want to control access to certain attributes.

For those of you familiar with other programming languages, this type of access is often referred to as “getters and setters”. In many language, implementing “getters and setters” means using private variables and public functions that get and set the variable’s value. Since Python doesn’t (really) have private variables, the descriptor protocol is basically a built-in and Python-ic way to way to achieve something similar.

This article will introduce you to the descriptor protocol, descriptors, and properties. It will focus on demonstrating how to use them to create managed attributes. Since the descriptor protocol requires new-style classes, all of the examples in this article require Python 2.2 or newer.

A few definitions

Before moving forward, it is important to understand a few related terms. These terms will introduce some basic concepts and help you follow along with the remainder of the article.

descriptor protocol – The following three methods make up the descriptor protocol: __get__, __set__, and __delete__.

descriptor – An “object attribute with binding behavior, one whose attribute access has been overridden by methods in the descriptor protocol.” [1] In other words an “attributes whose usage resembles attribute access, but whose implementation uses method calls.”[2]

data descriptor – A descriptor with the __get__ and __set__ methods of the descriptor protocol defined.

non-data descriptor – A descriptor with only the __get__ method of the descriptor protocol defined. “Python methods (including staticmethod() and classmethod()) are implemented as non-data descriptors.” [3]

property – A built-in type that implements the descriptor protocol and allows you to easily create data descriptors.

Don’t worry if you don’t fully understand these definitions, the remainder of this article will hopefully clarify any confusion you have.

The Descriptor Protocol

Let’s take a closer look at the descriptor protocol and see how we can use it to create a descriptor. As previously mentioned, the descriptor protocol is made up of three methods: __get__, __set__, and __delete__. These methods have specific signatures and they are as follows, where self is the class that owns the methods:

__get__(self, instance, owner)
__set__(self, instance, value)
__delete__(self, instance)

These three methods represent the three basic operations that you perform on attributes in general: querying the value of that attribute; assigning a value to it; and, (very rarely) deleting it. They work as follows:

• The __get__ method is called when the attributes value is being queried. The __get__ method should return the (computed) attribute value or raise an AttributeError exception.” [4] This is where access to the attribute’s value is managed.
• The __set__ method is used in the assignment operation. It is called when we want to set the attribute value. This is where you can control what values, or types of values are being assigned to your attribute.
• Finally, the __delete__ method is called when we want to delete the attribute. Here you can (rarely) decide whether or not to delete the attribute.

There are also three different parameters passed to the three methods (excluding the standard self parameter for methods that belong to a class):

• owner – This “is always the owner class.” [5] This means that it is the actual class, and not an instance of the class. So if the descriptor is in a class called MyClass, owner will be that class.
• instance – An instance of class type owner. It is “the instance that the attribute was accessed through” [6], or None if the attribute is being accessed through the class (owner) instead of an instance.
• value- The value that the attribute is being set to.

This difference between owner and instance might be a bit confusing, so let’s look at a quick example. Let’s say we have a descriptor my_descriptor in the class MyClass, if we were to run the following code:

my_class_instance = MyClass()
print my_class_instance.my_descriptor

The second line queries the descriptors value and results in the __get__ method being called with the instance parameter being my_class_instance. The owner parameter will be the MyClass class.

Note: Notice that we treat the descriptor my_descriptor as though it is a normal attribute. We don’t call print my_class_instance.my_descriptor.__get__(my_class, MyClass). This is what was meant by: “attributes whose usage resembles attribute access, but whose implementation uses method calls.”[7]

If the following code were run:

print MyClass.my_descriptor

The __get__ method will again be called. This time the instance parameter will be None and the owner parameter will be the MyClass class.

Note: Only the __get__ method has the owner parameter. This means that it is the only function in the descriptor protocol that can be accessed through the class. Setting and deleting the descriptor through the class actually changes what the variable is. For example, if we tried to assign the numeric value 2 to a descriptor variable using the class, we would not access the descriptors __set__ method. Instead we would change the type of the variable from a descriptor to an integer with the value of 2.

A Simple Descriptor Example

A simple “transparent” descriptor example can be found in Listing 1. The first thing to notice in the code is that both the SimpleDescriptor and MyClass classes are “new-style” classes because they are derived from object. This is important because, as mentioned above, descriptors only work with “new-style” classes. The second point to notice is that the descriptor has class scope as opposed to instance scope. There are also two extra print statements included in the code. They are there to let us follow the execution a little more easily.

Listing 1

class SimpleDescriptor(object):

    def __get__(self, instance, owner):
        # Check if the value has been set
        if (not hasattr(self, "_value")):
            raise AttributeError
        print "Getting value: %s" % self._value
        return self._value

    def __set__(self, instance, value):
        print "Setting to %s" % value
        self._value = value

    def __delete__(self, instance):
        del(self._value)

class MyClass(object):
    my_value = SimpleDescriptor()

Using Listing 1 to execute the following code:

my_instance = MyClass()
my_instance.my_value = 416
print my_instance.my_value

The output would be the following:

Setting to 416
Getting value: 416
416

As you can see the second line (my_instance.data_descriptor = 416) calls the __set__ method and sets the _value attribute. When we call print my_instance.data_descriptor the __get__, method is called and the _value attribute is returned.

The Problem with the Simple Example

The previous example may look like a perfectly good descriptor, but there is something wrong with it. Take a look at what happens when we runs this new code:

my_instance = MyClass() #Create the first instance
my_instance.my_value = 416 #Set its' value
my_second_instance = MyClass() #Create the second instance
my_second_instance.my_value = 204 #Set its' value
print my_instance.my_value #What was the fist instance's value?

We get the following output:

Setting to 416
Setting to 204
Getting value: 204
204

Notice that when we set the second instance’s my_value descriptor to be 204, we are also setting the first instances. This is because my_value has class scope, so both instances (and the class itself) share the same instance of the SimpleDescriptor class. Since SimpleDescriptor only stores one value, they all actually share the same value. We will get the same results if we check what the classes value of my_value is:

my_instance = MyClass() #Create the first instance
my_instance.my_value = 416 #Set its' value
my_second_instance = MyClass() #Create the second instance
my_second_instance.my_value = 204 #Set its' value
print MyClass.my_value #What's the classes value?

We get the following results:

Setting to 416
Setting to 204
Getting value: 204
204

In the last line of the code (print MyClass.my_value #What's the classes value?) we simply use the class (ignoring both instances) in order to get the value of my_data. This will be an instance where the __get__ function will be called with the instance parameter set to None.

Note: This is not a problem if you want to share the value across all instances.

The Solution to the Problem

In order to get around this issue, you have to remember that if you want values unique to each instance your descriptors must store values that are unique to each instance. This can be in the instance itself, in dictionary in the descriptor, or perhaps in a text file. Just make sure that the value is unique to each instance. Though it seems like a simple solution, in practice a suitable solution for all cases is difficult to implement. As a result you should probably pick a specific implementation for your specific situations.

Using each instance as a key, one can store the value in a dictionary in the descriptor itself. The problem with this solution should be obvious to programmers familiar with Python dictionaries: only immutable types can be used as keys. This is fine if you know what object you are working with, but what about in the future when you want to add a descriptor to your sub-classed list?

Another solution is to store the value in the instance itself. You can do this by easily adding the value to the instance’s __dict__. The limitation is that the descriptor need to be given a suitable key so that there is no collision with anything already in the instance’s __dict__.

Listing 2 shows a solution to the problem from the previous section where the value is stored in the instance’s __dict__. Values are indexed using a key name provided during the creation of the descriptor. Aside from where we store the value, there is little difference between Listing 1 and Listing 2. Notice that the value name is a converted into a string. This is done to ensure that it can be used as a key.

Listing 2

class FixedDescriptor(object):

def __init__(self, value_name):
    self.value_name = str(value_name)

def __get__(self, instance, owner):
    if (instance is None):
        raise AttributeError
    elif (not instance.__dict__.has_key(self.value_name)):
        raise AttributeError
    return instance.__dict__[self.value_name]

    def __set__(self, instance, value):
        print "Setting to %s" % value
        instance.__dict__[self.value_name] = value

    def __delete__(self, instance):
        if (instance.__dict__.has_key(self.value_name)):
            del(instance.__dict__[self.value_name])


class MyClass(object):
    my_value = FixedDescriptor("__value")

The major usage difference between Listing 1 and Listing 2 is the fact that the __get__ method no longer works at class level; it only works at instance level. This should be obvious since this solution stores the value in the instance. If there is no instance, we have nowhere to store the value! If you attempt to access the descriptor at class level (the first if statement) an attribute error will be raised:

def __get__(self, instance, owner):
    if (instance is None):
        raise AttributeError
    elif (not instance.__dict__.has_key(self.value_name)):
        raise AttributeError
    return instance.__dict__[self.value_name]

Easy Data Descriptors with Properties

The descriptor provided in Listing 2 will work for many situations but for my money the easiest way to implement data descriptors is to use the property type. I would recommend using it unless you need the same descriptor across many different classes or attributes (i.e. for type validation). Properties can be thought of as a simple and easy way to create data descriptors. The property type implements the descriptor protocol and gets around the problem of where to store the descriptor value in an easy way: it lets the class that created the descriptor deal with it.

The full signature of the property function is as follows:

property(fget=None, fset=None, fdel=None, doc=None)

Where fget, fset, fdel are methods that will be called when the __get__, __set__, and __del__ members of the descriptor protocol are called. The doc parameter is a string that will be used as the docstring for the descriptor. If the doc parameter is not specified, the docstring of the fget method is used.

The signature of the fget, fset, fdel functions are as follows:

fget(instance)
fset(instance, value)
fdel(instance)

In the three signatures, instance is a reference to the object that owns the property attribute. This is the same instance that gets passed to the descriptor protocol. Since instance is the first parameter of each method, these are, for all intents and purposes, member methods of a class. The value parameter is the same as the value parameter that is passed to the __set__ method. It is what we are setting the attribute to.

The basic way to implement properties can be seen in Listing 3. As you can see, what we do for a property is very similar to what we did for our initial descriptor in Listing 1. We set up the three functions necessary for the descriptor protocol, and then use them to create our property attribute. Creating the property is very simple:

Listing 3

class MyClass(object):
    #Create the fget, fset, and fdel methods
    def __get_value(self):
        print "Getting value: %s" % self.__value
        return self.__value

    def __set_value(self, value):
        print "Setting to %s" % value
        self.__value = value

    def __del_value(self):
        del(self.__value)

    #Create the property
    my_value = property(__get_value
            , __set_value
            , __del_value
            , "This is my property")

    def __init__(self, value=0):
        self.my_value = value

my_value = property(__get_value
        , __set_value
        , __del_value
        , "This is my property")

Make Data Attributes Data-Descriptors

A common reason to use descriptors is to create read-only attributes. This means allowing callers to get (or access) an attribute’s value, but not allowing them to set its’ value. At first glance it seems as though one can accomplish this by using descriptors with only the __get__ method defined, i.e. a “non-data descriptor”.

While this seems to create a read-only descriptor attribute, it actually does nothing of the sort. Instead of using the descriptor’s __set__ function for the assignment operation (since no __set__ function is defined), the default Python assignment operation is used. This means that instead of stopping the assignment operation, a new attribute will be created in the instance (remember that descriptors have class scope) having the same name as the descriptor attribute and taking precedence in future operations.

This may be a bit confusing so let’s look at the example in Listing 4. It’s very much like our previous descriptor examples except it does not specify the __set__ and __del__ functions. Let’s look at what happens when we try to use this as though it were a read-only attribute:

Listing 4

class WonkyDescriptor(object):
    """This Descriptor isn't read only"""
    def __init__(self, value_name):
        self.value_name = str(value_name)

    def __get__(self, instance, owner):
        if (instance is None):
            raise AttributeError
        elif (not instance.__dict__.has_key(self.value_name)):
            raise AttributeError
        print "Getting a value"
        return instance.__dict__[self.value_name]

class MyClass(object):

    my_value = WonkyDescriptor("_value")

    def __init__(self, value):
        #initial value
        self._value = value

my_instance = MyClass(23) #Create the first instance
print my_instance.my_value
my_instance.my_value = "Don't set me!"
print my_instance.my_value

When we run this, we get the following output:

Getting a value
23
Don't set me!

As you can see we did not succeed in creating a read-only attribute, instead something else happened. The “Getting a value” string is printed out when we print out my_value for the first time. This means that we are accessing the value through the descriptor. We then set the value of the descriptor attribute and print out that new value. Since “Getting a value” isn’t printed out a second time, we know that the descriptor’s __get__ method is not accessed when the second print statement is called.

Let’s take a closer look at what is happening using the following code, which prints out the instance’s __dict__:

my_instance = MyClass(23) #Create the first instance
print my_instance.__dict__
my_instance.my_value = "Don't set me!"
print my_instance.__dict__

This results in the following, which explains what is happening:

{'_value': 23}
{'_value': 23, 'my_value': "Don't set me!"}

When we first access the __dict__ we see our value indexed by the _value key that we told the descriptor to use. Now look at the second __dict__. The old value is still there, but so is a new value my_value. When we assigned “Don’t set me!” to my_value, we didn’t overwrite the descriptor attribute at class scope, we created a new instance attribute! And it is not exactly a read-only attribute.

In order to create a read-only attribute, you need to create a data descriptor where the __set__ method raises an AttributeError exception. An easy way to do this is to create a property and only set the fget function. Instead of using the descriptor in Listing 4, we can construct a read-only attribute as follows:

def __get_value(self):
    return self._value
my_value = property(__get_value)

Conclusion

I hope that after reading this article you can see how powerful descriptors (especially data-descriptors) can be. Once you get the hang of them, they are a great way to implement “getters and setters” and read-only attributes. Data-Descriptors are also very useful for performing validation during the assignment operation, i.e. ensuring that a value remains a specific type, or in a specific form.

The more you play with descriptors and use them in your code, the more you’ll see how sophisticated and useful they can be. That being said, it’s important to remember that unless you have a good reason to use descriptors you probably don’t need them. There’s no need to sacrifice the readability of your code or the dynamism of Python just for the sake of using descriptors. But when you have a real reason for managed attributes, you’ll find that descriptors are more than up to the task.

[1] http://docs.python.org/ref/descriptor-invocation.html
[2] http://www.python.org/download/releases/2.2/descrintro/#property
[3] http://docs.python.org/ref/descriptor-invocation.html
[4] http://docs.python.org/ref/descriptors.html
[5] http://docs.python.org/ref/descriptors.html
[6] http://docs.python.org/ref/descriptors.html
[7] http://www.python.org/download/releases/2.2/descrintro/#property