Let’s say you’re making A Thing For Humans To Use. It has Many Features, and that Sometimes, Bugs Are Discovered.

In many organizations, bugs are organized by priority:

• P1 (“Showstopper”) - a critical issue that must be fixed before any P2’s. Often a terrible crasher or something that prevents beta-testing.
• P2 (“Must-Have”) - an issue that must be fixed before the next release.
• P3 (“Below-The-Line”) - an issue that, while important, shouldn’t hold up a release. These are often edge-case issues.
• P4 (“Nice-To-Have”) - usually nit-fixes and pixel-nudges - aren’t prioritized over other bugs, and often aren’t prioritized over new features.

This prioritization is important - but as with any system, what gets measured is what matters, and sometimes, those P3’s and P4’s really add up. While individual bugs matter, your users experience the product as a cohesive whole, and many will abandon it after one or two bugs mar their experience.

It’s important that bugs represent individual units of work for developers and testers - so bugs often get broken down into small, specific components, and teams fix those components.

However, this breaking-down of bugs often means that design bugs are filed as P4 issues - they aren’t crashing the app, and each individual design issue only affects one part of one screen.

How Do Small Bugs Add Up?

Let’s say bug1 appears, and it affects 10% of the users, so we decide it’s an “edge case”. This’d be a “P3” (or “Priority 3”) bug - not something worth delaying a release. Here’s what the odds of hitting bug1 look like for a user:

What happens if bug2 appears, and it also affects 10% of your users? Here’s our graph, now with two dimensions - your odds are now 19% that you’ll hit a bug:

What if bug3 shows up, and it also affects 10% of your users? At this point, we’d have to start drawing 3D diagrams, so Let’s Use Math Instead™:

oddsOfHittingABug = 1 - (bugProbability ^ bugCount)

What are the odds you’ll hit a bug?

(Of course, this is an oversimplification - bugs are often not independently distributed. FWIW, I got the idea for this math from page 100 of The Beginning of Infinity.)

An Infestation of Small bugs

When I was a designer, I’d often find many design bugs filed as P4 - things that were too small to bother fixing with urgency. This was really frustrating! We’d spend time making pixel-perfect designs, and then wind up shipping a product with glaring typography, layout, animation, and copywriting bugs.

Putting our Product Manager Hats on, these bugs are often goofy little giblets. “Look”, you say, “We built an app that talks to the server, gets a result back, and renders it - that is so much! Why complain about these little nits?”

At some point, all these little tiny bugs form this Nasty Hive that really mucks up the experience:

• A mound of small bugs makes it harder for users to trust you with their data, their money, and their time.
• Designer morale drops - why bother making pixel-perfect comps, style guides, animations, or filing bugs if they’re never going to get fixed?
• Engineers, QA, and Product Managers want to build something beautiful, too! When your culture doesn’t value design bugs, it means that engineering slacks on implementing the design, and QA stops looking for them. After all, if design bugs aren’t important, why waste time on them?

Calling In An Exterminator

So, what can we do about this hive-of-a-thousand-P4-bugs?

• Create a meta-bug, called “We Have Lots Of P4s”, give it a P2 or P3 priority, and let a few team members go off and tackle the bugs for a fixed amount of time. You’ll be surprised how much this can polish things up and boost morale.
• Have an occasional “nit-fix” week (say, once a quarter?), where you let team members fix whatever bugs are bothering them.
• Use tools that help avoid P4s in the first place, like a color / font palette in your code (more on this in a future post).
• Involve the team in the bug-prioritization, so they understand and accept the rationale.
• What’s your Design GPA? How important is each screen? What is each screen’s grade (A, B, C, etc)? Combining these two bits of information gives you a weighted “Design GPA” - a different measurement of your product that provides a different measurement of the design bugs in your app. Maybe those 5 P4 bugs on the checkout screen really are a collective problem!

The P1-P4 prioritization system works great for many reasons, but it disincentivizes teams from polishing up the design of a product. What Gets Measured Gets Done - take time to measure things differently every once in awhile.