Python in Detail: The Diamond Problem

Welcome to the The Diamond Problem lesson!

This lesson is shown as static text below. However, it's designed to be used interactively. Click the button below to start!

In a previous lesson, we saw the method resolution order (MRO), which is stored in .__mro__. When we access an attribute or method, the MRO tells Python which classes to check for that attribute, and in which order.
Thinking about the MRO can be difficult, especially when we encounter "the diamond problem". This is when a class has multiple parent classes, and the parents share a common grandparent class.

class A: pass
class B(A): pass
class C(A): pass
class D(C, B): pass

We can visualize the inheritance relationships by drawing a diagram where the lower classes inherit from upper classes. The diagram has a diamond shape, which is where the name "diamond problem" comes from.
```
   A
 /   \
B     C
 \   /
   D
```
We'll explore this problem by writing classes that load image files like JPEG, GIF, and PNG. Each file format is implemented by a single mixin class: JpegLoader, GifLoader, etc. (Mixin classes are designed to add functionality to another class without being used on their own.)
Then we'll define a customizable ImageLoader that works with any combination of available formats by inheriting from their loader classes.
In the next example, we define five classes. First is BaseLoader, which all of our mixins inherit from. Then we define mixins for the JPEG, GIF, and PNG formats. Finally, we mix all of those into a single ImageLoader class, which works with all three file formats.

class BaseLoader:
  pass

class JpegLoader(BaseLoader):
  pass

class GifLoader(BaseLoader):
  pass

class PngLoader(BaseLoader):
  pass

class ImageLoader(JpegLoader, GifLoader, PngLoader):
  pass

The inheritance relationships still form a diamond, but it's more complex than our first example. Now there are three branches instead of two. (For brevity, we've omitted the "Loader" suffix from some class names in this diagram.)

         Base
        / | \
      /   |   \
    /     |     \
  JPEG   GIF    PNG
    \     |     /
      \   |   /
        \ | /
      ImageLoader

Our goal is to write a method that lists the image loader's supported image formats. For example, GifLoader.supported_formats() should return ["GIF"], and ImageLoader().supported_formats() should return ["JPEG", "GIF", "PNG"]. We won't actually load any images, though; that would take far too much code.
We could manually list the formats by hard-coding a list inside of an ImageLoader.supported_formats method. But that would duplicate information: we already inherited from the image loaders, so the inheritance relationships themselves already say which formats we support. Duplication is always risky, because we might change the inheritance list without remembering to update the duplicated list of formats.
The solution to this problem is super(). We've already used super() to call methods on a single parent class, but it can do more than that.
When we call super() from inside of a class, it doesn't simply call the corresponding method in a parent class. Instead, it searches for the next class in the MRO. For example, here's ImageLoader.__mro__:
Note: this code example reuses elements (variables, etc.) defined in earlier examples.
>
[cls.__name__ for cls in ImageLoader.__mro__]
Result:
The next class after GifLoader is PngLoader, so calling super() inside of GifLoader gives us PngLoader. Note that those two classes don't have a parent-child relationship, even though they're adjacent in the MRO! That may seem strange, especially since the function name super() seems like it should call the superclass. But this strangeness is unavoidable in any diamond inheritance relationship. Three classes inherit from BaseLoader, but only one class can come directly before BaseLoader in the MRO list.
In short, each class can use super() to delegate to the next class in the MRO, which may or may not be its parent. By using super(), we can do that delegation without directly accessing .__mro__.
Now we can list all the supported formats by calling each loader's .supported_formats methods and combining the results. Read the .supported_formats methods below carefully. Each one builds a list with a single string, then appends it to the list from the next class.
Note that the BaseLoader.supported_formats method returns [], since it doesn't support any formats at all. We'll look at that in more detail later in this lesson.

class BaseLoader:
  def supported_formats(self):
    # The base image loader can't load any formats.
    return []

class JpegLoader(BaseLoader):
  def supported_formats(self):
    return ["JPEG"] + super().supported_formats()

class GifLoader(BaseLoader):
  def supported_formats(self):
    return ["GIF"] + super().supported_formats()

class PngLoader(BaseLoader):
  def supported_formats(self):
    return ["PNG"] + super().supported_formats()

Note: this code example reuses elements (variables, etc.) defined in earlier examples.
>
JpegLoader().supported_formats()
Result:

Note: this code example reuses elements (variables, etc.) defined in earlier examples.

class JpegAndPngLoader(JpegLoader, PngLoader):
  pass

JpegAndPngLoader().supported_formats()

Result:

['JPEG', 'PNG']

Note: this code example reuses elements (variables, etc.) defined in earlier examples.

class ImageLoader(JpegLoader, GifLoader, PngLoader):
  pass

ImageLoader().supported_formats()

Result:

['JPEG', 'GIF', 'PNG']

We can combine our image loader classes in any way simply by inheriting from them. They adapt to the inheritance hierarchy automatically, and always produce a correct list of supported formats.
However, we need to be careful when we use this pattern. If one of the mixins forgets to delegate to super().supported_formats(), then the whole thing falls apart.
In the next example, GifLoader.supported_formats forgets to delegate to super(). Everything else in the example is the same as above. The result is a partial list of supported file formats.

class BaseLoader:
  def supported_formats(self):
    return []

class JpegLoader(BaseLoader):
  def supported_formats(self):
    return ["JPEG"] + super().supported_formats()

class GifLoader(BaseLoader):
  def supported_formats(self):
    return ["GIF"]

class PngLoader(BaseLoader):
  def supported_formats(self):
    return ["PNG"] + super().supported_formats()

class ImageLoader(JpegLoader, GifLoader, PngLoader):
  pass

Note: this code example reuses elements (variables, etc.) defined in earlier examples.
>
# This should be ['JPEG', 'GIF', 'PNG'], but there's a bug.
ImageLoader().supported_formats()
Result:
Finally, let's revisit BaseLoader. Its .supported_formats returns [], which seems pointless. But if we omit it, the system breaks. Eventually, one of the super().supported_formats() calls will call BaseLoader.supported_formats(). If that method doesn't exist, we get an AttributeError.
The code below is the same as the last working version, except now BaseLoader.supported_formats is missing.

class BaseLoader:
  # Note that we don't define the `.supported_formats` method here!
  pass

class JpegLoader(BaseLoader):
  def supported_formats(self):
    return ["JPEG"] + super().supported_formats()

class GifLoader(BaseLoader):
  def supported_formats(self):
    return ["GIF"] + super().supported_formats()

class PngLoader(BaseLoader):
  def supported_formats(self):
    return ["PNG"] + super().supported_formats()

class ImageLoader(JpegLoader, GifLoader, PngLoader):
  pass

Note: this code example reuses elements (variables, etc.) defined in earlier examples.

ImageLoader().supported_formats()

Result:

AttributeError: 'super' object has no attribute 'supported_formats'

Our .supported_formats technique represents a significant trade-off. Its biggest cost is indirection: we can't directly see which methods call which other methods, because the order isn't known until we actually inherit from the individual loaders. This makes the code more difficult to read.
The benefit is flexibility: it allows us to combine the classes in any way that we want, and it will always work as long as they delegate to super() properly. When we write the individual loaders, we might not anticipate needing a JpegAndGifLoader or a GifAndPngLoader. But we can build those combined loaders without changing any existing code.
Is this approach worth the indirection and extra complexity? It depends on what problem we're solving. If we're writing code that loads images in a single application, then we probably know what image types we needs to support. We should probably write a class to do that directly.
However, this technique makes more sense in frameworks and complex libraries. For example, imagine that every image loader class is written by a different person, each published as open source. When building the application, we install only the loaders that we need, then combine them by inheriting from them. None of the individual loaders' authors knew exactly which combination of image formats we'd need to load, but their classes still work together in our application.