In the fast paced world of application development we deal with many things. Software limitations, lack of standardization in the industry, lack of time, resources, and/or quality. These things (and more) lead to bugged, erroneous or just plain bad applications…
As developers, we hate poor quality and we hate it when our products aren’t up to snuff. Take pride in your craft, do it well and do it right.
But sometimes it can be a bit overwhelming. When someone creates something of great value and many people start using it, many bugs can be discovered in a short period of time. This can lead to walking into your office in the morning and noticing you have 200+ e-mails in your inbox.
Seasoned Developers have seen these come and go. Under time constraints one has to decide: “What are the real problems?” and “What comes first?”.
“Triage is a process of determining the priority of patients’ treatments based on the severity of their condition.”
When you get hundreds of e-mails with people panicking and screaming…there’s a certain amount of pain there. There’s a severity. So how do we determine severity in application development?
There was a well written essay on the Lost Garden: Improving Bug Triage with User Pain in which they talk about this subject.
“The traditional bug triage process is miserably inefficient. Over my decade in this industry, I’ve spent months of my life sitting in windowless offices manually reviewing (and re-reviewing) thousands of bugs. Often times, there are three or four folks on the triage team, typically the most skilled people on the team, sitting about and bickering for hours over the finer points of obscure bugs. Politics, boredom and arbitrary decisions are unfortunately common. The result is wasted time and poorly managed bugs.”
“So we came up with a better way.”
Problems with current bug triage
Traditional bug triaging is a time consuming and tedious process. Bugs come into a bug database with little prioritization, the team leads sort and rate each problem and then assign them to the appropriate members of the team. This process tends to run into several issues:
- Lack of shared criteria: Different people often value different aspects of a bug, which leads to unhealthy disagreement. A designer might think a usability issue is a critical fix, while a programmer might be concerned about a crash. With no common criteria, it is hard to build consensus quickly.
- Wasted time: Often the highest skilled team members are required to triage bugs. They spend hundreds of hours poring over mundane issues again and again. This is time that could be better spent improving the product.
- Bottlenecks: Bugs are often required to go through a review process so that precious developer time isn’t spent on bugs that would have otherwise been punted. The loop from the submitter to the triage team to the developer can cause delays for critical bugs.
- Big undifferentiated bins of bugs: Since the incoming rate of bugs is often higher than the fix rate, large piles of bugs will accumulate for each developer. If a developer has 50 bugs on their plate, they will fix them in a semi-random order or rely on micromanagement by the triage team. The first tactic means critical bugs are sometimes left to be fixed until the end. The second means more time is wasted on reviewing bugs.
- Triage burn out: After reviewing thousands of bugs, many triage teams stop caring or become fixated on a few bugs at the cost of reviewing others. The result is that some bugs are poorly triaged and the quality of bug ranking in the database is low.
The system we use for the teams that I am apart of, revolve around the following system.
Use anchored scales
Now that we have our factors, each one needs a rating scale. At this point, we do something slightly tricky. Each point on the three scales is anchored to an objective description. Here are the anchored scales I prefer:
Type (What type of bug is this?)
- 7: Crash: Bug causes crash or data loss. Asserts in the Debug release.
- 6: Major usability: Impairs usability in key scenarios.
- 5: Minor usability: Impairs usability in secondary scenarios.
- 4: Balancing: Enables degenerate usage strategies that harm the experience.
- 3: Visual and Sound Polish: Aesthetic issues
- 2: Localization:
- 1: Documentation: A documentation issue
Priority (How will those affected feel about the bug?)
- 5: Blocking further progress on the daily build.
- 4: A User would return the product. Cannot RTM. The Team would hold the release for this bug
- 3: A User would likely not purchase the product. Will show up in review. Clearly a noticeable issue.
- 2: A Pain – users won’t like this once they notice it. A moderate number of users won’t buy.
- 1: Nuisance – not a big deal but noticeable. Extremely unlikely to affect sales.
Likelihood (Who will be affected by this bug)
- 5: Will affect all users.
- 4: Will affect most users.
- 3: Will affect average number of users.
- 2: Will only affect a few users.
- 1: Will affect almost no one.
If Billy Bob sends me an e-mail and says “After the last update, none of the sites on my server can connect to the database! HELP!!!!11!!!”, I can reasonably start making severity calculations:
Type:  Crash
Priority:  A User would return the product.
Likelihood:  Will affect most users.
Total: 15. Naturally I know this is probably the most important thing at the moment and I should focus my attention on it. But if on the other hand I received this from Jane Doe: “After installing Skype, all my phone numbers in Internet Explorer are being restyled!!!! Help!111!11!!!”. I can do the same thing:
Type:  Documentation
Priority:  Nuisance
Likelihood:  Will only affect a few users.
Total: 4. If I have more important things to do, I am going to put this one on hold. I will tell the client I am going to look into the issue and get back with them when I can. But now I know for myself, it’s appropriately prioritized.
One last thing to note. The User Pain system is not a system that can be set in stone. You cannot prioritize something once and it to keep that value. As time goes on, things tend to hurt more. If you get a cut on your leg and it doesn’t hurt too bad, a doctor may move on to a more severe patient. But if he never comes back, that cut can get infected and you could end up losing your leg. Or in this case, the client.
So always factor time, how often the issue occurs and keep in mind that these are reported by people. All people have different pain thresholds.
Implement User Pain as a way of ordering your priorities. But do not let it be your religious text on the subject. Learn and adapt as you must when you deal with these things.