Here’s a pattern I see fairly often at the end of bug reports:
Expected: “Total” field should update and display correct result.
Actual: “Total” field updates and displays incorrect result.
Come on. When you write a report like that, can you blame people for thinking you’re a little slow? Or that you’re a bureaucrat, and that testing work is mindless paperwork and form-filling? Or perhaps that you’re being condescending?
It is absolutely important that you describe a problem in your bug report, and how to observe that problem. In the end, a bug is an inconsistency between a desired state and an observed state; between what we want and what we’ve got. It’s very important to identify the nature of that inconsistency; oracles are our means of recognizing and describing problems. But in the relationship between your observation and the desired state, the expectation is the middleman. Your expectation is grounded in a principle based on some desirable consistency. If you need to make that principle explicit, leave out the expectation, and go directly for a good oracle instead.
[…] Blog: Very Short Blog Posts (24): You Are Not a Bureaucrat – Michael Bolton – http://www.developsense.com/blog/2015/02/very-short-blog-posts-24-you-are-not-a-bureaucrat/ […]
Thanks for an interesting post! I’ve been thinking about the nature of reporting lately.
It’s thanks to blogs like this one that I question formats like the one you provided, which I think helps me to know that more background on strategy, coverage, oracles, impact and risk might just be required:
“Expected”… by whom? Why do they expect it? Is it important that they do?
Desired state… by whom? Why? When?
“between what we want”… who’s “we”? Why does it matter what “we” want?
“what we’ve got”… according to whom? How do they know, and why should I trust them?
And so on.
A very interesting blog post. For me the expected behavior is a very important part of a good bug report. When I write the expected behavior, I think one extra time. Have I understood the function right? Do I have the correct configuration? I my message in a clear manner? The expected behavior becomes a lifeline for me as testers to improve the quality of my own bug reports.
Additional benefits of the expected behavior is that I think it will be easier to write a descriptive title for my report. It also becomes easier to avoid judgmental words like shall or should. My advice is to create the title last, when you have finished typing the expected behavior.
Michael replies: It’s possible for you to ask “Have I understood the function right? Do I have the correct configuration? I my message in a clear manner?” without mentioning the expected behaviour, or even thinking about it. But in addition, I worry that you haven’t read the blog post that I linked to here. Where does your expectation come from? It comes from some principle or model. Why not cite the model directly?