Testing is not about making sure that test cases pass. It’s about using any means to find problems that harm or annoy people. Testing involves far more than checking to see that the program returns a functionally correct result from a calculation. Testing means putting something to the test, investigating and learning about it through experimentation, interaction, and challenge. Yes, tools may help in important ways, but the point is … Read more
A member of Linked In asks if “a test case can have multiple scenarios”. The question and the comments (now unreachable via the original link) reinforce, for me, just how unhelpful the notion of the “test case” is. Since I was a tiny kid, I’ve watched trains go by—waiting at level crossings, dashing to the window of my Grade Three classroom, or being dragged by my mother’s grandchildren to the … Read more
In response to my post from a couple of days ago, Gus kindly provides a comment, and I think a discussion of it is worth a blog post on its own. Michael, I appreciate what you are trying to say but the simile doesn’t really work 100% for me, let me try to explain. The simile has prompted you to think and to question, so in that sense, it works … Read more
Imagine that you are working for me, and that I want your help in qualifying and hiring new staff. I start by giving you my idea of how to interview a candidate for a job. “Prepare a set of questions with specific, predetermined answers. Asking questions is expensive, so make sure to come up with as few questions as you can. Ask the candidate those questions, and only those questions. … Read more
Having trouble understanding why James Bach and I think it’s important to distinguish between checking and testing? Consider this: a pile of leaves is not a tree. Leaves are important parts of trees, but there’s a lot more to a tree than just its leaves. The leaves owe their existence to being part of a larger system of the tree. Nature makes sure that leaves drop off and are replaced … Read more
Dale Emery, a colleague for whom we have great respect, submitted a comment on my last blog post, which in turn referred to Testing and Checking Refined on James Bach‘s blog. Dale says: I don’t see the link between your goals and your solution. Your solution seems to be (a) distinguishing what you call checking from what you call testing, (b) using the terms “checking” and “testing” to express the … Read more
If you’ve been in testing for long enough, you’ll eventually report or demonstrate a problem, and you’ll hear this: “No user would ever do that.” Translated into English, that means “No user that I’ve thought of, and that I like, would do that on purpose, or in a way that I’ve imagined.” So here are a few ideas that might help to spur imagination. The user made a simple mistake, … Read more
In which the conversation about heuristics and oracles continues… “So what’s the difference,” I asked my tester friend Tony, “between an oracle and a heuristic?” “Hmm. Well, I’ve read the Rapid Testing stuff, and you and James keep saying an oracle is a principle or mechanism by which we recognize a problem.“ “Yes,” I said. “That’s what we call an oracle. What’s the difference between that and a heuristic?” “An … Read more
A heuristic is a fallible method for solving a problem or making a decision. “Heuristic” as an adjective means “something that helps us to learn”. In testing, an oracle is a heuristic principle or mechanism by which we recognize a problem. Some years ago, during a lunch break from the Rapid Software Testing class, a tester remarked that he was having a good time, but that he wanted to know … Read more
I often do an exercise in the Rapid Software Testing class in which I ask people to catalog things that, for them, make testing harder or slower. Their lists fit a pattern I hear over and over from testers (you can see an example of the pattern in this recent question on Stack Exchange). Typical points include: I’m a tester working alone with several programmers (or one of a handful … Read more