First Aid for the Mission Statement

A while back, a tester brought a patient in for treatment. It wasn’t a human patient; it was a sentence about building and testing in an organization. The tester asked me for help.

“Could you provide me with a first aid kit for this statement that came from my management?”

“We have to move on to DevOps to be able to release code more often but we also have to increase testautomation in any way we can and minimize manual time consuming testing.”

This is the sort of statement that needs more than first aid; it needs emergency room treatment. We’ll start with handling some critical problems right away to get the patient stabilized. Then we need to prepare the way for longer-term recovery, so that the patient can be restored to good health and become an asset to society.

I’ll suggest a number of quick treatments. As I do that, I’ll identify why I believe the patient needs them.

• Replace “have to” with “choose to” in each case.

Unless someone is about to run afoul laws of nature, of government, or of ethics, no one has to do anything. People and organizations choose to do things. It’s important to preserve your agency. When you have to do things, you don’t have control over them. When you choose to do things, you remain in charge.

• Replace “move on to DevOps” with “apply DevOps principles and practices”.

DevOps is not something to do; it’s a set of ideas and approaches for getting important things done. The central principle of DevOps—that development and operations people work together to support the needs of the business—is what matters most. If that principle is absent in the organization, there are practically infinite ways in which things will go wrong.

There will be more to say about DevOps-related practices and principles as we proceed.

• Replace “to be able to” with “to be better able to”.

Not being a thing in and of itself, DevOps is not necessary to be able to do anything in particular. There were plenty of organizations building and releasing software successfully before DevOps, and there will be plenty of successful organizations long after DevOps is forgotten. Nonetheless, many of the ideas currently associated with DevOps could be very helpful.

DevOps doesn’t guarantee success, but applying its core principle might improve the odds.

• Replace with “release code” with “release valuable products”.

The product is not the code, and the code is not the product. The product is the sum of code, software platforms, machinery, data, documentation, and customer support. The product is all of that together, delivering whatever experience, value, and problems that the customer encounters, good and bad. The code is part of the product, and the code enables the product. The code controls the mechanical parts of the product. That’s not to diminish the importance of the code. The code makes the product possible. If it’s a software product and there’s no code, there’s no product. If there is code, and it’s bad code, it’s less likely that we have a good product.

It’s a good thing to release valuable products. Therefore it’s a good thing to understand the code that we’re releasing—but not just the code, and not just its behaviour. That’s not trivial, but it’s the easy part of testing. The harder part of testing is understanding the relationships between the code, its behaviour and the people who will be using the product or otherwise interacting with it—including developers, operations people, and testers.

• Replace “more often” with “at an efficient and sustainable pace”.

Delivering working software frequently is one of the principles behind the Manifesto for Agile Software Development. DevOps principles and practices are intended to support agility. Building efficiently and frequently—certainly a DevOps practice, but not a practice exclusive to DevOps—affords the opportunity to discover whether there are problems that threaten value before we inflict them on customers.”More often” isn’t the point, though, because “more often” might be good, bad, or irrelevant.

Consider the extremes.

When we build a product very rarely, problems get buried under layers of increasing complexity in the product and our inexperience with it. Testers become overwhelmed with the volume of learning and investigation (and, very probably, bug reporting) to be done.

When we build a product frenetically, we also get infrequent experience with it, because each new build comes along before we can gain experience with last one. In this case, testers get overwhelmed by the pace of the builds, and shallow testing, at best, is all that’s possible.

There is probably business value in being able to deploy software promptly, but that’s not the same as deploying software constantly. There might be contexts in which frequent deployment might provide real business value. In other contexts, a firehose of deployment can disrupt your customers, such that they don’t care about the deployment, but they do care about the disruption. The key is to recognize what context you’re in, and to minimize costly disruption.

One reasonable compromise in most cases is to set up systems to build the product easily and therefore frequently. In the build and in the processes leading up to it, include a smattering of automated checks to provide a quick alert to problems close to the surface. From the stream of builds, choose one periodically, and spend some time testing each one deeply to find rare, hidden, or subtle problems that can escape even a disciplined development process.

• Replace “but we also” with “Thus”.

The first clause in the statement sets up the second clause. The first clause doesn’t undermine the second clause; the first clause puts legs under the second.

In our emergency treatment, let’s use “thus” to reattach the legs to the body. Also, we’ll add a sentence break, giving the patient a little more room to breathe.

• Replace “we have to increase” with “we choose to apply”.

We’ve already covered the “we have to” part of this replacement.

Increasing something is not necessarily a good thing. In engineering work, everything is a tradeoff between desirable factors. Every activity that we might consider valuable comes with some degree of opportunity cost, reducing our capacity to do other things that we might also consider valuable. When we choose to apply something, we can choose to apply it more, or less, or just as we’re currently doing, to obtain the greatest overall benefit.

• Replace “testautomation” with “powerful tools”.

I’m sincerely hoping that testautomation is a typo, rather than a new term. We’ll put a space in there.

There are many wonderful ways to apply tools in testing. Automated checking is one of them; only one of them.

Tools can help us to build the product, to prepare for testing, and to reconfigure our systems efficiently for better coverage. Tools can help us to probe the internals of the system to see things that would otherwise be invisible. Tools can help us to collect and represent data for analysis; to see patterns in output. Tools can help us to record and review our work.

Replacing “test automation” with “powerful tools” could help reduce the risk that “test automation” will be interpreted only as “output checking”.

• Replace “in any way we can” with “in ways that help us”.

There are some things that we can do that we probably don’t want to do. We can serve steak with turpentine sauce, but no one should eat it and it’s a waste of good turpentine.

Tools can definitely help us with building the product quickly, and with identifying specific functional problems that might threaten its value. Tools can also be overapplied, reducing our engagement and human interaction with the product.

Fixation on tooling to exercise functions in the product can be a real problem if we forget that people use software. Even if our product is a software service, its API is used by people directly—programmers putting the API to work—and indirectly—end users who interact with the product through an interface designed for non-programmers. Functional correctness is important; so are parafunctional elements of the product: usability, performance, supportability, testability…

Tools can help us to focus our attention on important observations. Tools can also dazzle or distract us, diverting our focus from other important observations. We choose to apply tools not in any we can, but in ways that help—and our choices can include changing or dropping tools when they’re not helping.

Consequently, it might be a good idea to remember what we’re using the tools for. So, in addition to the replacement, let’s add “to build the product, to understand it, and to identify problems efficiently”.

• Replace “and minimize manual time consuming testing” with… something else.

This particular wound has become infected, and there’s a lot of debris in it. It requires a fair amount of cleanup, emergency surgery, and some stitches.

One principle of DevOps is the idea that teams use “practices to automate processes that historically have been manual and slow”. That’s a good idea for tasks that can be mechanized, and that benefit from being mechanized. It’s not such a great idea to forget that many tasks—and parts of tasks—are non-routine, rely on expertise and tacit knowledge, and can’t be made explicit or mechanized.

Programming involves strategizing, interpreting, designing, speculating, reflecting, analysing… Testing involves all of those things too, and more. Organizations reasonably want programmers work quickly, but no one suggests “minimizing manual time-consuming programming”. This is because no one considers programming a manual process. Programming is an intellectual process; a cognitive process; a social process. So is testing.

Programmers type; that’s the manual part of programming. Just as for programming, the central work of testing is not the typing. No one refers to the typing part as “manual programming”; and when a programmer sets a build process in motion or takes advantage of plugins in an integrated development environment, no one refers to “automated programming”.

Just as there is no manual programming, and no automated programming, there is no manual testing, and there is no automated testing. There is testing.

https://www.developsense.com/blog/2013/02/manual-and-automated-testing/
https://www.developsense.com/blog/2017/11/the-end-of-manual-testing/

Anything worth doing requires some time and effort, and we usually want to apply our limited time and effort to things that are worth doing. It does make sense to minimize or eliminate the amount of time that we spend on unimportant things. It also makes sense to apply appropriate effort to things that are worth doing; to maintain their value; and to increase that value where possible.

Just as some tasks in programming can be carried out by time-saving tools like compilers, some of the tasks in testing can be carried out by time-saving tools like automated checks.

Compiling, though, is not the central task of programming. The central task of programming is modeling and expressing things in the human world in a way that machinery can deal with them. We can then use that programmed machinery to extend, enhance, accelerate, intensify, or enable human capabilities. All that requires a significant degree of preparation, technical savvy and social judgement—and time for cycles of design, experimentation, learning, and refinement. No one should complain about this taking the time it needs.

Checking is not the central task of testing. The central task of testing is the search for problems that matter—ways in which the software fails to meet the needs or desires of its users, or introduces new problems of its own. That requires not only checks for problems that we can anticipate, but a search for problems that we didn’t anticipate.  Like development work, testing work also requires a signficant degree of preparation, technical savvy and social judgement—and time for experiencing, exploring, discovery, and investigation. No one should complain about this taking the time it needs.

At least one class of testing tasks is even faster than running automated checks: the tasks that we choose not to do, because cost, value, and risk don’t warrant them. It’s worth the investment to pause every now and then to assess the relative value of unattended automated checks; instrumented tool-supported testing; and direct, unmediated experience with the product.

The trick here is to set ourselves up to do the fastest, least expensive testing that fulfills the mission of finding problems that matter before it’s too late. One way to get there is to apply fast, easy, non-interruptive checks that don’t slow down development. Then, periodically, do deep testing to find rare, subtle, hidden, intermittent, emergent bugs that might elude even highly capable and well-disciplined programmers.

So, replace “and minimize manual time consuming testing” with with “We want to minimize distracting, unhelpful, or unnecessary work. We want to and maximize our ability to evaluate and learn about the product both efficiently AND sufficiently deeply.”

With all those replacements, the text might be longer, but it’s more accurate and more precise; bigger and stronger.

So:

We choose to move to DevOps to be better able to release valuable products at an efficient and sustainable pace; thus we choose to apply powerful tools in ways that help us to build the product, to understand it, and to identify problems efficiently. We want to minimize distracting, unhelpful or unnecessary work. We want to maximize our ability to evaluate and learn about the product both efficiently AND sufficiently deeply.

Every recovering patient can use motivation and support, so we’ll set our patient up with a motivating and supporting statement, and send them on their way together.

As developers, testers, and operations people working togther, our goal is to enable and support the business by delivering, testing, building, and deploying valuable, problem-free products. As testers, our special focus is to help people to become aware of any important problems that would threaten the value of the product to people that matter; to help our clients determine whether the product that they’ve got is the product they want.