Delivering the News (Test Reporting Part 3)

In the last post in this series, I noted some potentially useful structual similarities between bug reports (whether oral or written) and newspaper reports. This time, I’ll delve into that a little more. To our clients, investigative problem reports are usually the most important part of the product story. The most respected newspapers don’t earn their reputations by reprinting press releases; they earn their reputations through investigative journalism. As testers … Read more

Braiding The Stories (Test Reporting Part 2)

We were in the middle of a testing exercise at the Amplifying Your Effectiveness conference in 2005. I was assisting James Bach in a workshop that he was leading on testing. He presented the group with a mysterious application written by James Lyndsay—an early version of one of the Black Box Test Machines. “How many test cases would you need to test this application?” he asked. Just then Jerry Weinberg … Read more

Why Pass vs. Fail Rates Are Unethical (Test Reporting Part 1)

Calculating a ratio of passing tests to failing tests is a relatively easy task. If it is used as a means of estimating the state of a development project, though, the ratio is invalid, irrelevant, and misleading. At best, if everyone ignores it entirely, it’s simply playing with numbers. Otherwise, producing a pass/fail ratio is irresponsible, unethical, and unprofessional. A passing test is no guarantee that the product is working … Read more

Do Not Close This Window (Or Click The Back Button)

Here’s a classic case of poor design and user experience. Most of us have seen something like it. It happened to my wife yesterday. It will happen to you again soon, probably. You’re making an online payment for some product or service. You press a button that says something like “Submit Payment”. A web page appears that says something like “Your payment is being submitted. Please do not close this … Read more