TypeScript's excess properties can bite you

Recently, our Cypress tests started hanging silently. This post explains the cause: because of a subtle part of TypeScript's language design, some circular data structures were invisible in the code, which caused a serialization error, which Cypress handled incorrectly. If we don't understand the subtleties of our tools, bugs like this can be inscrutable; but if we learn the subtleties, they can be quick, if annoying, speed bumps.

1. Excess object properties

TypeScript's type system is unusual in many ways. Here's an example highlighting one of them:

type User = {
  name: string,
}

function buildAmir() {
  return {
    name: 'Amir',
    age: 36,
  }
}

const amir: User = buildAmir()

console.log(amir)

In most statically typed languages, this code wouldn't compile, but TypeScript allows it. What comes out when we run it? Amir has the type {name: string}, so it seems like the answer should be {name: 'Amir'}. What actually comes out at runtime is:

{name: 'Amir', age: 36}

This arises from two design decisions in TypeScript. First, objects are allowed to have excess keys and still satisfy an object type that doesn't have those keys. The excess keys are ignored by the type system. For example, the compiler wouldn't let us say amir.age, even though the age property does exist at runtime!

Second, the TypeScript compiler emits code by stripping the types away. What's left is JavaScript. For our example, the compiler emits JavaScript roughly like this:

function buildAmir() {
  return {
    name: 'Amir',
    age: 36,
  }
}

const amir = buildAmir()

console.log(amir)

When we look at it that way... of course the output is { name: 'Amir', age: 36 }. That's what the code says!

At first glance, this all seems fine. So how did it make our tests silently fail to run?

2. Large objects

Internally, Execute Program has an Example type representing a single interactive code example in a lesson. Its type looks like this, heavily simplified and tweaked to make sense in isolation.

type Example = {
  kind: "example"
  id: string
  code: string
  expectedResult: string
  isInteractive: boolean
}

For example, we might have an example where code is '1 + 1' and expectedResult is '2'. In that case, Execute Program would show the user 1 + 1 and wait for them to type 2.

In the past, the lessons referenced their examples, but the examples didn't know about the lessons that contain them. (There's no lesson property in the object above.) That was done because backreferences would create a reference cycle: the lesson references the example, which references the lesson, which references the example, and so on forever. Reference cycles make garbage collectors work harder, as well as causing confusing bugs like the one that we're working toward here.

(We'll use "reference cycle" and "circular data" interchangeably in this post.)

It's inconvenient for examples not to know about their lessons, so we recently added some lesson data to the example type. But not a full lesson, just a little bit of data: the containing lesson's ID. This made a particular new feature much easier to implement. Now the Example type becomes:

type Example = {
  // This stuff is all the same as before.
  kind: "example"
  id: string
  code: string
  expectedResult: string
  isInteractive: boolean

  // This is new.
  lesson: {
    id: string
  }
}

No reference cycles! That little lesson object only has an id, not a list of examples that would cause a cycle.

But unfortunately, we naively created the example object like this:

const example: Example = {
  kind: "example",
  id,
  code,
  expectedResult,
  isInteractive,
  lesson,
}

The lesson there is a full lesson object. From TypeScript's perspective, example.lesson's type is {id: string}. But at runtime it will be the entire lesson object, which is (1) huge, and (2) contains references to all of the lesson's examples, which contain references back to the lesson, which contains references to all of the examples, and so on. The static types show no reference cycle, but one still exists.

3. Test data generation

Our Cypress tests need to load up our actual lessons for testing. Unfortunately, there's not a great way to do that in Cypress. What we do today is:

• Use a custom Cypress plugin (they're easy to write, fortunately) that runs before the Cypress test browser is launched.
• The plugin looks at the fully loaded curriculum object, which is loaded from our lesson files in the same way that it's loaded in the normal app.
• The plugin dumps that curriculum object (including courses, which contain lessons, which contain examples) to a JSON file.
• The tests can then import that JSON file to access the lesson data.

This seems clunky because it is clunky. But Cypress comes with a lot of constraints, and this was how we satisfied them.

When we added lesson to the Example object, the clunks came home to roost. Trying to call JSON.stringify on any circular data structure will cause an exception in Node. Here's a shell command demonstrating:

$ node -e 'const x = {}; x.x = x; JSON.stringify(x)'
TypeError: Converting circular structure to JSON

The bug

The JSON.stringify call on our curriculum object definitely threw an exception inside our Cypress plugin code. But the way that manifested in Cypress was... nothing. Cypress launched the browser process to run the tests and it sat there, blank.

Having seen this movie before, I took a guess and opened Activity Monitor. As expected, the Cypress process was using 100% of the CPU. Eventually, CPU usage dropped to zero, but Cypress still showed nothing. I knew that errors in the plugin loading stage sometimes (maybe always?) pass silently, and I knew that we only had one plugin, and I knew that it was serializing a lot of data.

A quick check and... yes, every Example in the system contains a circular reference to its Lesson. But then why did this cause 100% CPU usage for several seconds? Shouldn't Node have errored immediately after seeing the circular reference? What probably happened is: the curriculum object was so large that it took multiple seconds to get to the point where it could even check for cycles! (As I write this, our course list contains 4,024 code examples, plus all of the accompanying text, all stored in fine-grained data structures with a lot of metadata.)

The fix

The bug is weird, but the fix is very simple. Here's the buggy code that creates a circular reference:

const example: Example = {
  kind: "example",
  id,
  code,
  expectedResult,
  isInteractive,
  lesson,
}

We need to avoid putting the entire lesson object inside the example. Instead, we pull out only the property we care about, the ID:

const example: Example = {
  kind: "example",
  id,
  code,
  expectedResult,
  isInteractive,
  lesson: {
    id: lesson.id
  },
}

Now there's no cycle and everything works again. A trivial solution to a complex problem implicating several subtle topics. To summarize:

• TypeScript allows excess properties on objects, which will exist at runtime even though they're not shown in the static types.
• TypeScript's generated JavaScript code doesn't strip those excess properties, so they'll still exist at runtime.
• This allows us to write code where the types have no cycles, but the objects at runtime do have cycles.
• Which will cause an exception when trying to serialize a circular object into JSON.
• Which can cause silent failures if your third party tools, like test runners, aren't careful about handling errors.

Where do we place blame here? TypeScript's design decisions, including this one, allow it to be flexible enough to interact with existing code on the web. It's hard to argue against that; it's a major reason that we use TypeScript in the first place. And JSON.stringify is correct to error on cyclic data because that data is unrepresentable in JSON.

Cypress' silent error handling, on the other hand, was a genuine bug. But suppose that it had surfaced the error, "TypeError: Converting circular structure to JSON". We'd still have to understand circular data structures, and we'd still have to understand the TypeScript design quirk that lets them show up even when the types don't allow them.

That leaves us with this plan for debugging problems like this:

1. Know the subtleties of your tools, especially your programming language, so you can make reasoned guesses when you see surprising behavior.
2. Optional: only use perfect tools that will never have bugs, ever. This will let you skip some steps, like using Activity Monitor to guess at what Cypress is doing, but you still have to do most of (1).