“Missing Requirements”

This article was inspired by a thread on LinkedIn a while back. Thank you to Rahul Parwal for starting the thread off.

People sometimes suggest that requirements are unavailable, when what they really mean is that requirement documents aren’t available. That distinction is significant. (Plus, if requirement documents aren’t available… what are the programmers working from?)

There might not be great requirements documents, but there are always requirements. There’s always someone who requires or needs something from the product. That person, or others, also has (have) desires for the product that may or not be requirements, strictly speaking. They’re not what people need, but what they want, or what they’d like, or what they’d prefer. My friend George Dinwiddie refers to these whimsically — and accurately — as “desirements”.

Requirements and desires might be explicit (spoken, written, sketched, drawn, expressed in a table, rendered as automated checks, etc.) or tacit (not stated; not told). Tacit requirements and desires may be conscious or sub-conscious; that is, people may be aware of specific requirements now, or may not be consciously aware of them until later.

Those requirements might have been decreed by a single person, or they may be been developed as part of collaborative process, over time. Explicit requirements are expressions of someone’s intentions and notions about some of the needs and desires for the product. Some needs and desires; not all of them; and intentions for the product, not the reality of the product.

Testers often are guided to some degree by explicit requirements. A good tester looks for inconsistencies between the product and people’s stated claims about what it should look like and how it should behave. That’s fine, but that’s only one class of oracles. Good testing includes much more than demonstrating and confirming that the product can appear to do what people say it should do.

A good tester is skilled in drawing inferences about what people might require or desire from a product, and in performing testing that brings the tacit and sub-conscious requirements to light. Legitimate requirements might not be documented.

A good tester reports on the actual behaviour of the product, and assesses whether behaviour might represent a problem. Many potential problems are unanticipated and undocumented. Requirements documents rarely account for possible trouble.

A good tester also notices when explicit requirements might represent misunderstandings, errors, irrelevancies, or outdated ideas. Requirements documents can be wrong.

Inferences and insights don’t all come immediately, from inside the tester’s head. They also come from developing an understanding how people might use the product; the ways in which they might get value from it; and the ways in which problems about the product might threaten its value.

One highly practical and efficient way to develop that understanding is to perform testing, getting actual experience with the actual product. Artifacts or speech provide some ideas about how the product might be used. Interacting with the real product and experimenting with it presents many other ideas and possibilities.

As we explore, perform experiments, and obtain experience with the product, good testers keep notes, create diagrams, collect data, and marshall evidence to support the testing story. Mapping and describing the product are essential parts of testing. Developing those maps — whether they are laid out explicitly, or whether they remain tacit, as mental models — includes identifying what’s there, what’s missing, and what parts of the product have not been explored or investigated yet. Empty areas in our maps may represent areas of potential problems and risk. Detail on the maps might inspire ideas about going deeper still — or help us to decide that we’re satisfied that we’ve covered the product sufficiently. Silent omissions may highlight areas where everyone’s understanding is incomplete.

Sharing and discussing those maps and other work products, developed while testing the product, can help to shine further light on what people might need or desire. The results of those discussions can be fed back into requirements documents and other artificacts — but even if that doesn’t happen, the conversations help to refine the collective understanding of the product.

In other words: at the beginning of the any burst of development, we know something about what the requirements might be, but we don’t know them deeply until after we’ve tested the product. And we don’t know how to test the product until we’ve tried to test the product. In engineering work, the final product often looks quite different from people’s original conceptions of it. Requirements aren’t simply collected; they’re emergent, developing as the product develops. As Karl Wiegers is fond of saying, “If you’re talking about requirements gathering, you’re doing it wrong.”

Requirements documents are typically useful, but we often don’t learn very deeply from them. We learn more deeply about what we want from a product, and about how to test it, by testing it. If we’re paying attention and thinking critically about problems and risks, experience with the product doesn’t just confirm our preconceived needs and desires. Testing a product recontextualizes our intentions, helps to refine them; and helps us to recognize unanticipated problems, risks, benefits, and value.

Missing requirement documents can be a problem for testing. Missing (or insufficient) requirements documentation is also a problem that testing can help to solve.