You’ve “Built Quality In”. Are You Sure About That?

It’s common these days to hear people say that they don’t want to focus on finding bugs; they want to focus on preventing bugs. They want to focus on “building quality in”.

Let’s face it: building quality in is a pretty great idea, and preventing bugs from reaching customers is a really good thing. On this, reasonable people agree.

To prevent bugs from reaching customers, you’ll have to become a designer, a developer, or a manager; someone who has the capability, authority, role, and responsibility for building the product. If someone discovers a problem along the way, a designer has the responsibility to address the design; the developer has the responsibility to address the code; and the manager has the authority to require the problem to be addressed. You didn’t prevent that problem, but in the role of a designer, developer, or manager, you can take steps to prevent that problem from going any further.

On the other hand, as a designer, developer, or manager, your natural inclination is to focus your vision of what a successful, finished product will look like. To some degree, attending to that single conclusion requires you to push aside a virtually infinite number of possible ways in which things can go wrong. Considering all those possibilities can be paralyzing. Even shifting your mind temporarily from envisioning success to anticipating failure is time-consuming, effortful, and somewhat emotionally fraught. How many developers have you met who say “I love testing!”

In other words: when you’re focused on success, failure is—at best—in your peripheral vision.

Moreover, whether designer, developer, or manager, your perspective is inevitably a maker’s perspective; an insider to the process of creation. You develop mental models of the product as it’s being made, in a community of other makers.

In Rapid Software Testing, we don’t insist that there be any person on a project with the title “tester”. But we do believe that taking on the role of tester—someone somewhat estranged from the process of building—is a powerful heuristic to avoid over-focusing on success. The testing role, focused on revealing trouble, affords the opportunity to notice problems about the design, the code, and the relationship of the product to outsiders, the non-builders. That’s why review by developers who haven’t written the code exposes problems that the writers haven’t noticed. That’s a good thing.

You might think that reviewing the product from the builder’s perspective might be good enough—and you might be right about that. If you’re committed to building quality in, chances are that you’ll have anticipated and squashed many important problems before you decide that you’re ready to ship your product or deploy your service.

At that point, you could release the product and hope that the customers alert you to problems in it. For this to be successful, you’ll need to make some assumptions and do some work to support the strategy.

• You’ll need to demonstrate to customers that you’re eager to hear about problems and to resolve them. Let’s face it: people have become accustomed to being ignored by most software companies. Therefore…
• You’ll need friendly, helpful support staff to respond to the customers and allow them to feel heard. (A bot’s reply to an online report won’t do this very well.)
• You’ll need the development team to be able to act to those problems and get them addressed quickly in a way that demonstrates that you’re not only listening, but also responding.
• Even given all that, you’ll need faith that the customers will actually report on problems that matter to them and to be forgiving and patient while you fix them.
• Most importantly, you’ll have to be confident that problems that you missed aren’t so bad as to destroy the value of the product; to lose customer data; to threaten health or safety or money; to mess up your own systems; or to overwhelm the support department. You’ll have to be confident that those problems won’t damage the reputation of the product or your organization.

On the other hand, if you’re really committed to building quality in, you’ll be aware the risk of over-focus on success. You’ll realize the even the most skillful, capable people, as individuals and as groups, have blind spots and make mistakes. If you’re seriously committed to building quality in, you’ll be skeptical about how successful you’ve been at achieving it.

Checking functional output and the existence of elements on Web pages might be important, but that output and those elements are only a means to an end. The real purpose of a product is to help customers to solve problems, to get things done, to be entertained. Whether that has been achieved without introducing new problems requires more than confirmatory output checks, demonstrating that the product can work.

Quality is not a property of a product; it’s a set of many-to-many-to-many relationships between elements of the product, a variety of customers, and their different needs, desires, and preferences. Deciding that we have a well-checked product doesn’t mean that we’ve got a problem-free product, and doesn’t mark the end of testing. A well-checked product does provide a foundation for faster, more efficient, deeper testing that can happen in parallel with ongoing development.

To find hidden, subtle, intermittent, emergent problems in a product, you’ll want help from people who are estranged to some degree from builders’ focus. Finding the deep problems takes determination, time, effort, preparation, and a degree of disruption to the builders’ mindset.

To find problems without disrupting the developers’ focus, you’ll want someone attending full-time to trouble, problems and risk; someone who interacts with the product and gets experience with it before you inflict problems on your customers. You’ll want someone committed to learning and studying many things: the technology in and around the product; the problems that the product is intended to solve; the worlds of the users of the product who are outside the process of building it.

You’ll want testers. Or at least, a tester

A skilled tester doesn’t just test the product. A skilled tester tests our beliefs about the product. When you’re building quality in, and you think you’re done, you may believe that you’ve prevented problems about your product. You won’t know until you’ve tested deeply and systematically. Even then, you can’t know for sure. But with skilled testing, you’ll know better.