In a Webinar that he did for uTest on December 10, 2009, James Whittaker mused aloud about what a great idea it would be to structure exploratory testing and capture ideas about it in a repository for sharing with others. It seems to me that one ideal version of that would take the form of a bibliography in a book about exploratory testing, but apparently that’s not available. Yet I … Read more
And now for the immodest part of the EuroSTAR 2009 Test Lab report: I won the Best Bug award, although it’s not clear to me which bug got the nod, since I reported several fairly major problems. I tested OpenEMR. For me, one candidate for the most serious problem would have been a consistent pattern of inconsistency in input handling and error checking. I observed over a dozen instances of … Read more
Yesterday I set up a thought experiment in which we divided our day of testing into three 90-minute sessions. I also made a simplifying assumption that bursts of testing activity representing some equivalent amount of test coverage (I called it a micro-session, or just a “test”) take two minutes. Investigating and reporting a bug that we find costs an additional eight minutes, so a test on its own would take … Read more
If you’re a tester, you’ve probably been asked, “Why is testing taking so long?” Maybe you’ve had a ready answer; maybe you haven’t. Here’s a model that might help you deal with the kind of manager who asks such questions. Let’s suppose that we divide our day of testing into three sessions, each session being, on average, 90 minutes of chartered, uninterrupted testing time. That’s four and a half hours … Read more
At the EuroSTAR 2008 conference, I gave a talk entitled, “Two Futures of Software Testing” which was rated as the highest-scoring track session at the conference. Conference attendees also chose the talk as the winning entry for the CapGemini Award for Innovation. Here I provide a number of answers to questions that people have asked since the presentation. Q: How can we predict the future of software testing?A: Well, we … Read more
The distinction between testing vs. checking got a big boost recently from James Bach at the Øredev conference in Malmö, Sweden. But a recent tweet by Brian Marick, and a recent conversation with a colleague have highlighted an issue that I should probably address. My colleague suggested that somehow I may have underplayed the significance or importance or the worth of checking. Brian’s tweet said, “I think the trendy distinction … Read more
How is it useful to make the distinction between testing and checking? One colleague (let’s call him Andrew) recently found it very useful indeed. I’ve been asked not to reveal his real name or his company, but he has very generously permitted me to tell this story. He works for a large, globally distributed company, which produces goods and services in a sector not always known for its nimbleness. He’s … Read more
An article from a while back on StickyMinds entitled How To Choose Between Scripted and Exploratory Testing refers to a bunch of factors in making choices between scripted testing and exploratory testing. The problems start early: “As a test manager or engineer, you may be considering whether or not to use exploratory testing on your next project.” If you’re not planning on investigating any problems that you find, I suppose … Read more
Not too long ago, I posted a list of stopping heuristics for testing. As usual, such lists are always subjective, subject to refinement and revision, and under scrutiny from colleagues and other readers. As usual, James Bach is a harsh critic (and that’s a compliment, not a complaint). We’re still transpecting over some of the points; eventually we’ll come out with something on which we agree. Joe Harter, in his … Read more
After a presentation on exploratory approaches and on testing vs. checking yesterday, a correspondent and old friend writes: Although the presentation made good arguments for exploratory testing, I am not sure a small QA department can spare the resources unless a majority of regression checking can be moved to automation. Particularly in situations with short QA cycles. (Notice that he and I are using “testing” and “checking” in this specific … Read more