The Code Is the To-Do List

Managing our focus during development can be a challenge. Here's a common scenario: we're in the middle of working on user.ts when we find ourselves thinking: "I really need to go change this other thing in login.ts". But it's rarely a good idea to interrupt ourselves during a complex change. We have a lot of context in our heads that we don't want to lose.

There are two common ways to deal with this: we can write down a note that reminds us to do it later, or we can write a quick failing (or pending) test that forces us to do it later.

Both approaches work just fine. However, there's a third way that's superior to both in some situations: we can use the code itself as a to-do list. Specifically, we can use the linter to enforce our to-dos.

In Execute Program, we use ESLint with the "no-warning-comments" lint rule enabled:

module.exports = {
  "rules": {
    "no-warning-comments": ["error", {terms: ["xxx"], location: "anywhere"}],
    ...
  },
  ...
}

This configuration makes any comment containing "XXX" an error. Lint errors cause our CI builds to fail, which means our XXX comments block deployment to production. As a result, we can now use XXX comments to mark critical problems that should never reach production.

(We could also use TODO comments instead of XXX by changing the ESLint configuration above. But our convention is that TODO comments are for longer-term deferred maintenance like "remove this old API endpoint" and XXX means "do not let this go to production". The particular string "XXX" isn't important here.)

This approach is often better than separate to-do lists and failing tests because:

1. Blocking deployment makes XXX comments more reliable than separate notes. We might accidentally deploy code without addressing notes in a separate system, but the ESLint rule above prevents deploys that contain "XXX" comments.
2. Not all changes are easily expressed as tests.
3. Leaving a comment in the code is usually faster than writing a test or switching to a note-taking app.
4. Everyone working in the code can see the XXX comments, with no need to agree on a shared note-taking system.

In the rest of this post, we'll see how XXX comments have helped our development process. We'll review four real examples from the Execute Program codebase, ordered from simplest to most complex. These examples come from a feature branch that's diverged from our main branch by about 3,500 lines.

$ git diff main --stat | tail -1
 40 files changed, 2448 insertions(+), 1155 deletions(-)

There are a lot of moving parts in this release, so it's crucial to keep track of all the changes we need to make as we develop!

1. Simplifying code

// XXX: Remove this module; it's trivial now.

We have a module that used to export several TypeScript types. On this branch, all but one of those types are now gone. We don't need this whole module to export one small type. The type can live in the subsystem's top-level module instead.

This change feels quick, so it's tempting to make it immediately. But it will be just as fast if we do it tomorrow. We don't have any special context in our heads that makes this change faster today.

On the other hand, we probably have a lot of context in our heads about the bigger change that we were making when we noticed this. Programming is highly dependent on short-term memory, and any distraction is a chance for us to forget details that may be important. Leaving an XXX comment only takes a few seconds, keeps our attention on the current task, and ensures that we'll remember to remove this module later.

2. Deferring a data format change

// XXX: This is inserting "Start Case" property names. They should be "camelCase".

While working on this branch, we needed to change a data format. Unlike the type definition change above, updating the rest of the code to match the new format will take a substantial amount of work. This change will also impact some other parts of the system, so we'd rather isolate it in a separate commit.

Like the module removal example, we don't want to distract ourselves. And the need for a separate commit makes deferring this change an even easier decision.

Sometimes, the fine-grained version control details also matter. You might think: Why not make the change now, then tease the two changes apart after the fact with git add -p or an interactive rebase? In this case, many of the affected lines are very close to lines that we've already changed. Individual modifications from both of our changes would end up in the same diff hunks, which makes teasing them apart much more difficult. The version control wrangling could end up taking longer than the change itself.

3. Managing temporary environment changes

// XXX: Changed the number here for faster testing. Change it back.

Our code has an event-triggering threshold: when event A happens enough times, it triggers event B. On this branch, we sometimes want to manually trigger event B after only a few occurrences of A, so we've temporarily lowered this threshold. But we don't want to accidentally push that change out, so we added an XXX comment.

Because we're working on a very stable part of the system, it doesn't make sense to lower the threshold permanently for our development environment. Changing the threshold based on the environment adds application complexity. Worse, it's yet another way that dev diverges from production, which is never desirable. We decided that it was better for this branch to diverge temporarily than for the dev environment to diverge permanently.

An XXX comment allows us to keep using the lower threshold, and gives us peace of mind that we won't accidentally push this dangerous change to production.

4. Documenting our work

The application-specific details are removed from this example. But its structure is as we wrote it in the real code.

/* XXX: Write summary comment explaining this module
 *   - Mention that the foo() function is responsible for frobbing.
 *   - Mention that bar() hits Stripe, so avoid multiple calls to it. But none
 *     of the other functions is allowed to hit Stripe, so call them at will.
 */

This branch introduces a major new module to the system. The module will need a summary comment, so it's tempting to write it right away. But writing module-level comments early on can be risky.

First, we may end up reverting our current changes, and this module may not even exist when the branch lands. If we write a perfect, nicely-formed comment, that effort might be wasted.

Second, this module or the other modules around it might change during the lifetime of our branch. We'll have a better sense for how the module fits into the system once the branch is stabilized, so that's a better time to finalize a high-level comment.

Finally, even if neither of those apply, the usual reasoning does still apply: we don't want to interrupt the change that we're making right now.

Even if we don't write the big comment up front, we can still make notes about what to include in it. Using a bulleted list lets us capture notes in the moment, without needing to take the time to write a full, nicely-formatted comment. This is a great example of how using the code as a to-do list balances tracking future work vs. interrupting our current task.

ESLint in action

Once the comments are in place, we can see them in our ESLint output. Our dev script automatically runs ESLint on every source file change, so we'll see this continually. This is also the output that will show up in CI, blocking deploys to production.

(The filenames are anonymized here because the exact details of the new subsystem aren't important.)

[eslint:server] src/server/old-subsystem/old-file.ts
[eslint:server]   63:13  warning  Unexpected 'xxx' comment: 'XXX: This is inserting "Start Case"...'  no-warning-comments
[eslint:server]
[eslint:server] src/server/new-subsystem/types.d.ts
[eslint:server]   5:1  warning  Unexpected 'xxx' comment: 'XXX: Remove this module; it's trivial...'  no-warning-comments
[eslint:server]
[eslint:server] src/server/new-subsystem/index.ts
[eslint:server]    7:1   warning  Unexpected 'xxx' comment: 'XXX: Write summary comment explaining...'    no-warning-comments
[eslint:server]   94:11  warning  Unexpected 'xxx' comment: 'XXX: Changed the number here for faster...'  no-warning-comments
[eslint:server]
[eslint:server] ✖ 4 problems (0 errors, 4 warnings)
[eslint:server]
[eslint:server] ESLint found too many warnings (maximum: 0).

Development flow

In large changes, it's common for us to have dozens of these XXX lint errors live at once. Usually, the arc of a large change is:

• Begin with the most risky parts of the change. It's usually best to start with the riskiest part. If the risks turn into serious problems, we may reprioritize or even cancel this project. We want that to happen at the beginning, rather than after we've spent a lot of effort on it.
• As we do the most risky work, we defer other work that we encounter along the way, adding XXX comments.
• At some point, we reach "peak XXX", which is often dozens of XXX comments at once.
• Then we begin resolving them until there are none left.

By the time we're resolving XXX comments, some will be outdated due to other changes made after we added the comment. That's fine; it's just a comment, and we can delete it. (But if we'd made the change at the time, that work would've been wasted!)

Even when the comments are still current, they're often quick changes that were just a bit too big to distract ourselves with at the time. We estimate that the downward slope from "peak XXX" to "0 XXX" is usually around 10% of the total development time for a branch. All of that work was going to happen one way or another; we just shifted it in time to aid our concentration and to allow better risk prioritization.

Using the linter to block XXXs from going to production is a great alternative to keeping a separate to-do list, or writing a test for every change we think of. Writing XXX comments isn't just faster; it protects our focus and prioritizes our attention. Give this approach a try on your next project!