Today, someone on Twitter pointed to an interesting blog post by Alan Page of Microsoft. He says: “How do testers determine if a bug is a bug anyone would care about vs. a bug that directly impacts quality (or the customers perception of quality)? (or something in between?) Of course, testers should report anything that may annoy a user, but learning to differentiate between an ‘it could be better’ bug … Read more
Let’s imagine, for a second, that you had a political problem at work. Your CEO has promised his wife that their feckless son Ambrose, having flunked his university entrance exams, will be given a job at your firm this fall. Company policy is strict: in order to prevent charges of nepotism, anyone holding a job must be qualified for it. You know, from having met him at last year’s Christmas … Read more
The post “Exploratory Testing and Review” continues to prompt comments whose responses, I think, are worthy of their own posts. Thank you to Parthi, who provides some thoughtful comments and questions. I always wondered and in attempted to see the difference between the Exploratory testing that you are talking about and the testing that I am doing. Unlike the rest of the commenter’s, this post made this question all the … Read more
In a comment on the previous post, Rahul asks, One doubt which is lingering in my mind for quite sometime now, “Can exploratory testing be automated?” There are (at least) two ways to interpret and answer that question. Let’s look first at answering the literal version of the question, by looking at Cem Kaner’s definition of exploratory testing: Exploratory software testing is a style of software testing that emphasizes the … Read more
In a recent blog post, Rahul Verma suggests that all testing is confirmatory. First, I applaud his writing of an exploratory essay. I also welcome and appreciate critique of the testing vs. checking idea. I don’t agree with his conclusions, but maybe in the long run we can work something out. In mythology, there was a fellow called Procrustes, an ironmonger. He had a iron bed which he claimed fit … Read more
On April 19, 2010, I was interviewed by Gil Broza. In preparation for that interview, we solicited questions from the listeners, and I promised to answer them either in the interview or in my blog. Here’s the second one. Unit testing is automated. When functional, integration, and system test cannot be automated, how to handle regression testing without exploding the manual test with each iteration? This question provides a great … Read more
The distinction between testing vs. checking got a big boost recently from James Bach at the Øredev conference in Malmö, Sweden. But a recent tweet by Brian Marick, and a recent conversation with a colleague have highlighted an issue that I should probably address. My colleague suggested that somehow I may have underplayed the significance or importance or the worth of checking. Brian’s tweet said, “I think the trendy distinction … Read more
How is it useful to make the distinction between testing and checking? One colleague (let’s call him Andrew) recently found it very useful indeed. I’ve been asked not to reveal his real name or his company, but he has very generously permitted me to tell this story. He works for a large, globally distributed company, which produces goods and services in a sector not always known for its nimbleness. He’s … Read more
After a presentation on exploratory approaches and on testing vs. checking yesterday, a correspondent and old friend writes: Although the presentation made good arguments for exploratory testing, I am not sure a small QA department can spare the resources unless a majority of regression checking can be moved to automation. Particularly in situations with short QA cycles. (Notice that he and I are using “testing” and “checking” in this specific … Read more
In the first post in this series, I proposed “that those things that we usually call ‘unit tests‘ be called ‘unit checks‘.” I stand by the proposal, but I should clarify something important about it. See, it’s all a matter of timing. And, of course, sapience. After James Bach‘s blog post titled “Sapience and Blowing Peoples’ Minds“, Joe Rainsberger commented: Sadly, the distinction between testing and checking makes describing test-driven … Read more