Python in Detail: Customizing Attribute Access

Welcome to the Customizing Attribute Access lesson!

This lesson is shown as static text below. However, it's designed to be used interactively. Click the button below to start!

By default, accessing a missing attribute raises an AttributeError.

class SomeClass:
  pass

obj = SomeClass()
obj.attr_a

Result:

AttributeError: 'SomeClass' object has no attribute 'attr_a'

However, that's only the default behavior. When we access a missing attribute, Python calls the .__getattr__ method. If that method exists, its return value becomes the result of the attribute access, as if an attribute with that name and value actually existed. By customizing .__getattr__, we dictate how missing attributes are accessed.

class AlwaysOne:
  def __getattr__(self, name):
    return 1

always_one = AlwaysOne()
(always_one.attr_a, always_one.attr_b, always_one.attr_c)

Result:

.__getattr__ gets one argument, the attribute name that we accessed.

class AllUppercase:
  def __getattr__(self, name):
    return name.upper()

all_uppercase = AllUppercase()
all_uppercase.hello

Result:

'HELLO'

.__getattr__ is only called when the attribute doesn't exist. Existing attributes bypass .__getattr__, and can be accessed as usual.

class DefaultToOne:
  def __getattr__(self, name):
    return 1

  attr_a = 2

always_one = DefaultToOne()
always_one.attr_b = 3
(always_one.attr_a, always_one.attr_b, always_one.attr_c)

Result:

(2, 3, 1)

What if we want to intercept all attribute accesses, even if the attribute already exists? We can do that by defining .__getattribute__.

class AlwaysOne:
  def __getattribute__(self, name):
    return 1

always_one = AlwaysOne()
always_one.attr_a

Result:

The .__getattribute__ method intercepts every attribute access. That even includes .__getattribute__ itself: when accessed from the outside, it's 1!

class AlwaysOne:
  def __getattribute__(self, name):
    return 1

always_one = AlwaysOne()
always_one.__getattribute__

Result:

It's easy to make mistakes with .__getattribute__. Consider what happens when the body of our .__getattribute__ method itself tries to look up an attribute like self.attr_a. The .__getattribute__ method intercepts its own attribute lookup, which runs .__getattribute__ again. Then that call accesses self.attr_a again, which calls itself again, and so on. This continues until we eventually get a RecursionError.

class AlwaysDefault:
  def __init__(self, default):
    self.default = default

  def __getattribute__(self, name):
    return self.default

always_three = AlwaysDefault(3)
always_three.attr_a

Result:

RecursionError: maximum recursion depth exceeded

We can fix this by using super().__getattribute__. In our case, that calls object.__getattribute__, which implements the normal attribute access behavior that we're familiar with.

class AlwaysDefault:
  def __init__(self, default):
    self.default = default

  def __getattribute__(self, name):
    return super().__getattribute__("default")

always_three = AlwaysDefault(3)
always_three.attr_a

Result:

.__getattribute__ can lead to other unexpected issues with methods. Here's one example. We'll see the code first, then discuss it.

class AlwaysOne:
  def __getattribute__(self, name):
    return 1

  def some_method(self):
    return 2

always_one = AlwaysOne()
always_one.some_method()

Result:

That example tries to call always_one.some_method(). Python methods are just special attributes, so method calls are a two-step process: first get the always_one.some_method attribute, then call it.
Our .__getattribute__ method intercepts the attribute access and returns 1. Then the () in always_one.some_method() try to call the number 1 as a function. That's why the error is "'int' object is not callable": we effectively tried to do 1().
When possible, we recommend using .__getattr__ instead of .__getattribute__, since it's less error-prone. Even debugging tools tend to break when we make mistakes with .__getattribute__.
We've seen that the getattr built-in is an alternative way to get an attribute from an object. It takes .__getattr__ and .__getattribute__ into account.

class AlwaysDefault:
  def __init__(self, default):
    self.default = default

  def __getattribute__(self, name):
    return super().__getattribute__("default")

always_three = AlwaysDefault(3)
(getattr(always_three, "attr_a"), always_three.attr_b)

Result:

(3, 3)

The .__setattr__ method customizes attribute assignment. It's called when we assign or update any attribute, whether it already exists or not. It gets two arguments: the attribute name and the new value.
In the next example, we write a class that remembers which attributes were assigned, but doesn't store the attributes in the normal way.

assignments = []

class RemembersAssignments:
  def __setattr__(self, name, value):
    assignments.append((name, value))

remembers = RemembersAssignments()
remembers.a = 1
setattr(remembers, "b", 2)
remembers.c = 3

assignments

Result:

[('a', 1), ('b', 2), ('c', 3)]

We already saw that accessing an attribute inside of .__getattribute__ can cause infinite recursion. There's a similar risk here: if .__setattr__ assigns an attribute, that can also cause infinite recursion.

class OverridesAttributeName:
  def __setattr__(self, name, value):
    self.overridden = value

overrides = OverridesAttributeName()
overrides.attr_a = 1
overrides.attr_a

Result:

RecursionError: maximum recursion depth exceeded

Let's put these ideas together in a larger example. The example below shows an AttributeLogger class that logs every read and write to every attribute. Getting an attribute adds ("get", attribute), while setting adds ("set", attribute, value).

class AttributeLogger:
  def __init__(self):
    # Create the empty list that we'll use as a log.
    super().__setattr__("log", [])

  def __getattribute__(self, name):
    # We don't log the `.log` attribute itself. That would be confusing when
    # reading the log! This also removes one place where we could have
    # infinite recursion.
    if name == "log":
      return super().__getattribute__("log")

    value = super().__getattribute__(name)
    self.log.append(("get", name))
    return value

  def __setattr__(self, name, value):
    self.log.append(("set", name, value))
    super().__setattr__(name, value)

Now here's a Cat class that inherits from AttributeLogger. As part of that inheritance, it gets the .__getattribute__ and .__setattr__ methods, giving Cat attribute logging behavior.

Note: this code example reuses elements (variables, etc.) defined in earlier examples.

class Cat(AttributeLogger):
  def __init__(self, name, age):
    # It's important that we call our superclass's constructor
    # to create the `.log` list.
    super().__init__()
    self.name = name
    self.age = age

  def is_kitten(self):
    return self.age <= 2

Note: this code example reuses elements (variables, etc.) defined in earlier examples.
>
keanu = Cat("Keanu", 2)
keanu.age += 1
keanu.is_kitten()

keanu.log
Result:
The log shows us everything that happened: from the self.name = name line in Cat's constructor, all the way through the self.age access in the final call to .is_kitten().
The AttributeLogger class gives us a peek at real-world uses for attribute customization. Here are a few real-world use cases:
- Intercept and log all attributes accesses, changes, or both, like we did above.
- Dynamic database access. Each object represents one row. Accessing an object's attributes gives us the value from that column in that row.
- Implement mock objects and other test doubles when testing. Mock objects are objects that are used in place of real objects in a test, often mimicking the real object's behavior.
- Map new attribute names to old names for backwards-compatibility.
Attribute customization seems esoteric at first, but it's widely used!
Here's a code problem:
The Course class exposes course information that we get from an API. The data comes in as a dictionary. When we look up an attribute on Course, like some_course.name, we want to instead check the dictionary. For example, if the dictionary is {"name": "Biology 101"}, then some_course.name should be "Biology 101".

Override Course's attribute access logic so that attribute lookups look inside of self._data. We should also support writing to attributes: some_course.name = "New Name" should update the "name" key in the dictionary.

A few notes:
1. You won't need to change either of the two existing methods, .__init__ and .display_name.
2. Since Course doesn't have any of its own attributes, you'll only be accessing non-existent attributes and should use .__getattr__.
3. The internal dictionary is named ._data to indicate that it's private and shouldn't be accessed outside the class.
class Course:
def __init__(self, data):
# We're going to define `.__setattr__` to intercept attribute writes. But
# we also need a way to set a "normal" attribute to hold the dictionary.
# We can explicitly call our parent class (`object`)'s `.__setattr__` to
# set the attribute.
super().__setattr__("_data", data)

def __getattr__(self, name):
return self._data[name]

def __setattr__(self, name, value):
self._data[name] = value

def display_name(self):
return f"{self.name} (#{self.id})"
intro = Course({
"name": "Intro To Induction",
"id": 101
})
assert intro.name == "Intro To Induction"
assert intro.id == 101

advanced = Course(
{
"name": "Baking For Professionals",
"id": 202,
"course_capacity": 30
}
)
assert advanced.id == 202
advanced.course_capacity = 35
assert advanced.display_name() == "Baking For Professionals (#202)"

# Look at the "private" attribute to be sure that we really updated the
# dictionary.
assert advanced._data["course_capacity"] == 35
Goal:
None
Yours:
None
Show Author's Answer