Python in Detail: Slots

• Normally, Python objects store their attributes in the .__dict__ dictionary.

• >

  class User:
    def __init__(self, name):
      self.name = name
  
  User("Amir").__dict__

  Result:

  {'name': 'Amir'}

• That lets code inspect Python objects' attributes without knowing the attribute names in advance. However, there's also a downside: each instance of the class must have a .__dict__ dictionary, which takes up memory. For example, a drawing app might create hundreds of thousands of Point instances as the user draws. Each point only holds two integers, but it keeps them inside of an entire Python dictionary. The dictionary is much larger than the integers themselves.

• Sometimes that memory usage adds up and becomes a problem. For those situations, Python offers "slots", a way to create objects without a .__dict__.

• We can add a .__slots__ attribute to the class, listing the names of our attributes. The slots tell Python which attributes will exist.

• >

  class Point:
    __slots__ = ["x", "y"]
  
    def __init__(self, x, y):
      self.x = x
      self.y = y

• When a class has a .__slots__ attribute, its instances don't get a .__dict__ at all.

• Note: this code example reuses elements (variables, etc.) defined in earlier examples.

  >

  origin = Point(0, 0)
  hasattr(origin, "__dict__")

  Result:

  False

• A class with slots only allows assignment to those slots. Trying to assign any other attribute is an AttributeError.

• Note: this code example reuses elements (variables, etc.) defined in earlier examples.

  >

  my_point = Point(1, 2)
  my_point.z = 3
  my_point.z

  Result:

  AttributeError: 'Point' object has no attribute 'z'

• The performance of slots and .__dict__ is similar. However, their memory usage is very different.

• Let's quantify the memory benefits of .__slots__. We'll define two equivalent classes, one with regular attributes and one with slots. Then we'll compare the memory usage when instantiating each of them.

• To measure memory use, we'll use the tracemalloc module from the standard library. The details of tracemalloc are out of scope here, so we won't dig into exactly how it works. But it's a great example of how comprehensive the Python standard library is!

• >

  import tracemalloc
  
  def measure_peak_memory_use(f):
    tracemalloc.start()
    f()
    _, peak = tracemalloc.get_traced_memory()
    tracemalloc.stop()
    return peak
• Note: this code example reuses elements (variables, etc.) defined in earlier examples.

  >

  class RegularUser:
    def __init__(self, name):
      self.name = name
  
  class SlotsUser:
    __slots__ = ["name"]
  
    def __init__(self, name):
      self.name = name
  
  # What's the peak memory use when building a regular (non-slots) user?
  first = measure_peak_memory_use(lambda: RegularUser("Amir"))
  
  # What's the peak memory use when building a user with slots?
  second = measure_peak_memory_use(lambda: SlotsUser("Amir"))
  
  (first, second)

  Result:

  Run Code

• The regular object (with a .__dict__) used significantly more memory! Most of that difference comes from the .__dict__ dictionary.

• This might seem like a serious problem: does this mean that Python wastes a lot of memory on .__dict__? Kind of, but there are factors that mitigate the problem for most real-world code. Most Python core types like integers, strings, lists, and dicts have no .__dict__, so they don't have this overhead.

• Occasionally, this memory cost does matter. Imagine that we need to load billions of users into memory at once to do some work on them. With that many users in memory, the .__dict__ overhead may lead to prohibitive memory consumption. If so, we can add .__slots__ to reduce the overhead. (But if we have billions of users, this easily-fixed memory problem will be the least of our technical challenges!)

• One final question: should we define .__slots__ on every object, to make the program more memory-efficient? Only in extreme cases. For example: if the system runs on a very small embedded computer, where memory is highly constrained, then it might make sense. But in that case, Python may not be the best language choice in the first place!

• In general, we don't recommend defining .__slots__ unless there's a specific memory problem. And when you do define .__slots__, we strongly recommend measuring memory use before and after the change, to make sure that it actually helped.