A tester asks… How do we handle production bugs? When management asks “Did you test this?”. how do I respond? When management asks “Why didn’t you find that bug?”, the first step is to accept in your own mind responsibility for looking for bugs, but not a commitment to finding every bug. The latter is a great aspiration, but an unreasonable commitment, and management shouldn’t be holding you to it. … Read more
This post serves two purposes. It is yet another installation in The Series That Ate My Blog; and it’s a kind of personal exploration of work in progress on the Rapid Software Testing Guide to Test Reporting. Your feedback and questions on this post will help to inform the second project, so I welcome your comments. As a tester, your mission is to evaluate the product and report on its … Read more
Throughout this series, we’ve been looking at an an alternative to artifact-based approaches to performing and accounting for testing: an activity-based approach. Frieda, my coaching client, and I had been discussing how to manage testing without dependence on formalized, scripted, procedural test cases. Part of any approach to making work accountable is communication between a manager or test lead and the person who had done the work. In session-based test … Read more
Throughout this series, we’ve been looking at an an alternative to artifact-based approaches to testing: an activity-based approach. In the previous post, we looked at a kind of scenario testing, using a one-page sheet to guide a tester through a session of testing. The one-pager replaces explicit, formal, procedure test cases with a theme and a set of test ideas, a set of guidelines, or a checklist. The charter helps … Read more
As a tester, I try to represent the interests of users. Saying the user, in the singular, feels like a trap to me. There are usually lots of users, and they tend to have diverse and sometimes competing interests. I’d like to represent and highlight the interests of users that might have been forgotten or overlooked. There’s another trap, though. As Cem Kaner has pointed out, it’s worth remembering that … Read more
I hear a lot from testers who discover problems late in development, and who get grief for bringing them up. On one level, the complaints are baseless, like holding an investigate journalist responsible for a corrupt government. On the other hand, there’s a way for testers to anticipate bad news and reduce the surprises. Try producing a product coverage outline and a risk list. A product coverage outline is an … Read more
Here’s a question I get from testers quite a lot: “What do I do when the developers give me something to test with insufficient information and time to test it?” Here’s my quick answer: test it. Here’s my practical answer: test it with whatever time and information you have available. (Testing is evaluating a product by learning about it through exploration and experimentation.) When your time is up, provide a … Read more
Last week on Twitter, I posted this: “The testing is going well.” Does this mean the product is in good shape, or that we’re obtaining good coverage, or finding lots of bugs? “The testing is going badly.” The product is in good shape? Testing is blocked? We’re noting lots of bugs erroneously? — Michael Bolton (@michaelbolton) January 31, 2018 “The testing is going well.” Does this mean the product is … Read more
Here’s an easy-to-remember little substitution that you can perform when someone suggests “automating the testing”: “Automate the evaluation and learning and exploration and experimentation and modeling and studying of the specs and observation of the product and inference-drawing and questioning and risk assessment and prioritization and coverage analysis and pattern recognition and decision making and design of the test lab and preparation of the test lab and sensemaking and test … Read more
A junior tester relates some of the issues she’s encountering in describing her work. To the people who thinks she “just breaks stuff all day”, here’s what I might reply: It’s not that I don’t just break stuff; I don’t break stuff at all. The stuff that I’ve given to test is what it is; if it’s broken, it was broken when I got it. If I break anything, consider … Read more