People often suggest that “understanding the requirements” is an essential step before you can begin testing. This may be true for checking or formal testing—examining a product in a specific way, or to check specific facts. But understanding the requirements is not a necessary precursor to testing, which is learning about a product through experimentation (a larger activity which might include checking) and creating the conditions to make that activity possible. Indeed, you may need to test in order to develop an understanding of the requirements, which in turn triggers more and better testing, yielding even better understanding of the requirements—and so on.
More generally, when thinking about testing, think more about loops, and less about lines.
I keep thinking I need to write an article about all the parallels between testing and empirical science. Testing is experimentation. Saying you need to understand the requirements before you can begin testing is like saying you need to understand relativity before you can begin dropping weights off the Leaning Tower of Pisa. Saying you can/should define all test cases up front is like saying Newton should have been designing Large Hadron Collider experiments. Principles of experimental design like isolating variables also apply to testing. I think there are even historical parallels – there was something in Neal Stephenson’s Baroque Cycle about the early history of empirical science that reminded me of the early days of software testing.
Michael replies: Thanks, Scott. That kind of article is in the works for me, too. Meanwhile, you might enjoy Simon Schaffer’s and Sajay Samuel’s segments on CBC’s How to Think About Science.
Hi Michael,
How else do you understand what problem your solution is trying to solve if not with requirements.
Understanding the problem space is ONE key part of the way I approach testing.
Michael replies: Yes, I expect that’s true. And testing is one key part of the way you come to understand the problem space, right?
Some explicit requirements might be available as we first engage with the product and the project. Other requirements will initially be tacit, not yet explicit; still others will never be made explicit, and will remain tacit forever. This kind of tacit knowledge develops through socialization in the project and through experiece with the product and the business domain. You know something about the product space so that you can test; you know more about it later because you tested. As I said in another post: we’re not working with lines here, but with loops.