I hear a lot from testers who discover problems late in development, and who get grief for bringing them up. On one level, the complaints are baseless, like holding an investigate journalist responsible for a corrupt government. On the other hand, there’s a way for testers to anticipate bad news and reduce the surprises. Try producing a product coverage outline and a risk list. A product coverage outline is an … Read more
In response to yesterday’s post on The Happy Path colleague and friend Albert Gareev raises an important issue: Until we sufficiently learned about the users, the product, and the environment, we have no idea what usage pattern is a “happy path” and what would be the “edge cases”. I agree with Albert. (See more of what he has to say here.) This points to a kind of paradox in testing … Read more
This post is adapted from a recent blast of tweets. You may find answers to some of your questions in the links; as usual, questions and comments are welcome. Update, 2017-01-07: In response to a couple of people asking, here’s how I’m thinking of “test case” for the purposes of this post: Test cases are formally structured, specific, proceduralized, explicit, documented, and largely confirmatory test ideas. And, often, excessively so. … Read more
In response to my recent blog post, 100% Coverage is Possible, reader Hema Khurana asked: “Also some measure is required otherwise we wouldn’t know about the depth of coverage. Any straight measures available?” I replied, “I don’t know what you mean by a ‘straight’ measure. Can you explain what you mean by that?” Hema responded: “I meant a metric some X/Y.” In all honesty, it’s sometimes hard to remain patient … Read more
In testing, what does “100% coverage” mean? 100% of what, specifically? Some people might say that “100% coverage” could refer to lines of code, or branches within the code, or the conditions associated with the branches. That’s fine, but saying “100% of the lines (or branches, or conditions) in the program were executed” doesn’t tell us anything about whether those lines were good or bad, useful or useless. “100% code … Read more
Instead of studying and learning from every bug, you can save a lot of time by counting and aggregating bug reports. That’s a good thing in its way, because if you don’t study and learn from every bug, you’ll need all the time you can get to deal with problems that seem to keep happening over and over again.
You do not need acceptance criteria to test. Reporters do not need acceptance criteria to investigate and report stories; scientists do not need acceptance criteria to study and learn about things; and you do not need acceptance criteria to explore something, to experiment with it, to learn about it, or to provide a description of it. You could use explicit acceptance criteria as a focusing heuristic, to help direct your … Read more
Like software products, models change as we test them, gain experience with them, find bugs in them, realize that features are missing. We see opportunities for improving them, and revise them. A product coverage outline, in Rapid Testing parlance, is an artifact (a map, or list, or table…) that identifies the dimensions or elements of a product. It’s a kind of inventory of aspects of the product that could be … Read more
A member of Linked In asks if “a test case can have multiple scenarios”. The question and the comments (now unreachable via the original link) reinforce, for me, just how unhelpful the notion of the “test case” is. Since I was a tiny kid, I’ve watched trains go by—waiting at level crossings, dashing to the window of my Grade Three classroom, or being dragged by my mother’s grandchildren to the … Read more
When someone says “We don’t have time for testing”, try translating that into “We don’t have time to think critically about the product, to experiment with it, and to learn about ways in which it might fail.” Then ask if people feel okay about that.