Writing a good error report

Published by Michael Banzon on Fri Mar 9, 2018

Good error reports lead to easier and more complete error fixing! In fact bad error reports can lead to worse software!

With that in mind let’s look into what makes a good error report.

What causes an error report?

Error reports - or simply just feedback from users - is caused by one of two things: Either the program crashes and automatically reports the error, or the user actively puts in some effort to report an error.

These two types of feedback is caused by fundamentally different types of errors - but the overall goal is basically the same. What is the purpose of an error report?

The goal of an error report is always the same - wether the user submits it actively or inactively. The program is in a state that makes the program useless or less usable that desired - and the users wish is to change this state by having the developer(s) apply changes to the software that will make the program (more) useable.

This is the very broad definition - it should fit all scenarios - and it gives us something to work with when we define good error reports and the content of such. What should be in an error report?

Working with the definition above we can quite easily deduct what must be in a good error report to make it useful - useful means the developer can help the user get the software into the desired state.

First of all the error report should contain a detailed description of what was experienced and why this was wrong - this description would normally be a description of observed vs. expected behaviour.

Depending on the type of error in question the observed vs. expected might be pretty obvious - there is a great gray area (naturally) where the observed vs. expected will be a topic for discussion - after all we are dealing with the expectations of real people. Although in many cases the software should produce very well defined results - there might even exist some documentation and/or definition of the expectation.

Secondly the error report should contain a detailed step guide that can be used to bring the software from a known good state (a state that is expected and acceptable) to the error state. This list of steps might range from a few steps to many steps - and the steps needs to be explained extensively - the work needed to produce and re-produce these steps along with describing them might seem like very tedious work being done for no good reason - after all, the error should be obvious to everyone and it should just be fixed - right?!

Not quite - computers and software are essentially state machines - and they perform actions that produce states/results. Given the same actions the same results should be achieved - with that in mind it should also be noted that any other set of actions would be likely to produce a different state. If the set of actions needed to get to the error state is missing or incomplete it might not be possible to reproduce and/or observe the error.

The reproducibility is what makes the developer able to fix the error/state achieved.

As a general rule of thumb - if a developer can successfully reproduce an error, she can successfully prevent if from happening again.