Some may say that the purpose of testing is to validate assumptions made by business analysts, designers, or developers. To me, that is at best a potential side effect of testing—but not the goal. If you want testing to reveal important problems in the product, do not focus on validating assumptions (to do so would be more like checking; testing may include some checking). To foster discovery, excellent testing—like excellent … Read more
People often suggest that “understanding the requirements” is an essential step before you can begin testing. This may be true for checking or formal testing—examining a product in a specific way, or to check specific facts. But understanding the requirements is not a necessary precursor to testing, which is learning about a product through experimentation (a larger activity which might include checking) and creating the conditions to make that activity … Read more
In response to my post from a couple of days ago, Gus kindly provides a comment, and I think a discussion of it is worth a blog post on its own. Michael, I appreciate what you are trying to say but the simile doesn’t really work 100% for me, let me try to explain. The simile has prompted you to think and to question, so in that sense, it works … Read more
Imagine that you are working for me, and that I want your help in qualifying and hiring new staff. I start by giving you my idea of how to interview a candidate for a job. “Prepare a set of questions with specific, predetermined answers. Asking questions is expensive, so make sure to come up with as few questions as you can. Ask the candidate those questions, and only those questions. … Read more
Having trouble understanding why James Bach and I think it’s important to distinguish between checking and testing? Consider this: a pile of leaves is not a tree. Leaves are important parts of trees, but there’s a lot more to a tree than just its leaves. The leaves owe their existence to being part of a larger system of the tree. Nature makes sure that leaves drop off and are replaced … Read more
Some testers have got into the habit of saying that “we break the software”. That leads to psychological and political problems: “The product was fine until the testers broke it.” The software is what it is, either broken or not, when we get it. So, try saying “We look for problems that could threaten the value of the software.” As James Bach says, the only things we break are illusions.
It is not the job of testing to build confidence in the product. Confidence is a relationship between the product and some stakeholder. It is much more the job of testing to identify problems in the product—and in people’s perceptions of the product—that are based on or that would lead to unwarranted confidence.
When someone says “We don’t have time for testing”, try translating that into “We don’t have time to think critically about the product, to experiment with it, and to learn about ways in which it might fail.” Then ask if people feel okay about that.
Apropos of my post from the other day and the inconsistency with image oracle heuristic, have a look here. I actually encountered this dialog myself several years ago. As one person commenting on the blog post reports, this security message from Windows has a typo in it (“form”, where it should be “from”), which has the ironic effect of making a security warning look even more like malware than it … Read more