Oracles from the Inside Out, Part 1: Introduction

In Rapid Testing (and, we believe, in all worthwhile testing) learning about the product and finding problems that matter are central to the mission. In the role of a tester, I support the business and development team by finding and describing things I believe to be important problems that might threaten the value of the product.

A skilled tester should be able do that at any stage—as soon as there’s something to test—with any kind of product, whether it’s a compiled program, a component, a function, source code, a specification, a requirements document, a feature idea, a model, a diagram… Any of these products may have observable, describable problems associated with them.

To be effective, I must be able to provide a credible description of why I believe those problems to be problems that matter. The goal is to give people information that helps them to decide whether the product they’ve got is the product they want. Anyone who is a consumer of that information is a client of the service that I offer.

As a tester, I’m neither the developer nor the product owner. I don’t have authority over the product, the project, or the business. It is not part of the testing role to fix the problem (that rests with the developers); it is not a tester’s responsibility to decide that the problem shall be fixed, nor when, nor how (that rests with the developers and the managers); nor is it a tester’s role to make decisions about the scope of the product (that too rests with others).

I don’t even have authority to declare definitively that something is a problem. For some problems, designers and developers make that determination. A higher authority rests within a business role—the person responsible for decisions about developing and releasing the product, like a project manager or product manager.

Some testers are uncomfortable with the reality of these limitations. To me, that would be like an investigative reporter being upset that he can’t stamp out malfeasance, prosecute corrupt politicians, and restructure local government. Investigative reporters investigate to reveal problems and issues. That’s an honourable and important job on its own—and it requires a specialized set of skills.

When an investigative reporter yearns to change things to be the way he wants them, he becomes a politician. Politicians need skill sets, mindsets, and priorities that are different from those held by reporters. Politicians have certain kinds of authority that investigative reporters don’t have. Politicians can—and perhaps should—take heed of the information that investigative reporters provide, but being a politician means that you’re entitled to ignore that information.

Similarly, being a tester is an honourable and important job, requiring a specialized mindset and a specialized set of skills. Building a quality product is a goal that everyone shares, but the priorities of developers and managers may be different from yours. If you want to be the one making decisions about the product, become a designer, a programmer, or a project manager.

When I encounter something that looks like a problem, I want my clients to understand the nature of the problem as I perceive it. In Rapid Testing, we say that an oracle is a means by which we recognize a problem that we encounter during testing. Yet a couple of years ago, we began to notice another role that oracles play in the social dynamics of software testing. Oracles help us to describe and articulate the problems that we see; and oracles calibrate our feelings about those problems with the rest of the team.

A few years ago, James and I developed this chart to help testers to illustrate the relationships between things as we process our recognition of a problem.

Recognizing a problem tends to start with a feeling, and tends to end with an understanding of the problem and feelings about it that are shared between the tester and the other members of the project community. How does that play out in practice? That’s going to be the subject of some posts over the next little while. Let’s go to the next one.