On Scripting

A script, in the general sense, is something that constrains our actions in some way.

In common talk about testing, there’s one fairly specific and narrow sense of the word “script”—a formal sequence of steps that are intended to specify behaviour on the part of some agent—the tester, a program, or a tool. Let’s call that “formal scripting”.

In Rapid Software Testing, we also talk about scripts and scripting in a more general way. It’s the same kind of scripting that some psychologists might talk about when they refer to “behavioural scripts”: things that direct, constrain, or program our behaviour in some way. Scripts of that nature might be formal or informal; explicit or tacit; structured by procedure, by data, or by the application we’re testing. We might follow these scripts consciously or unconsciously. Scripts shape the ways in which people behave, influencing what we might expect people to do in a scenario as the action plays out.

As James Bach says in the comments to our blog post Exploratory Testing 3.0,

By “script” we are speaking of any control system or factor that influences your testing and lies outside of your realm of choice (even temporarily). This does not refer only to specific instructions you are given and that you must follow. Your biases script you. Your ignorance scripts you. Your organization’s culture scripts you. The choices you make and never revisit script you. (my emphasis, there)

Imagine that you’re driving to a party out in the country. The list of directions that you got from the host scripts you. Many other things script you too. The starting time of the part—combined with cultural norms that establish whether you should be very prompt or fashionably late—script you, prompting you to leave home at a certain time. The traffic laws and the local driving culture script you, conditioning your behaviour and your interactions with other people on the road. The marked detour along the route scripts you to take a different route from the route you intended. The weather and the driving conditions script you. Your temperament and your current emotional state script you too. And if you don’t drive, the actions of the bus driver or the taxi driver script you.

In this more general sense of “scripting”, any activity can become heavily scripted, even if it isn’t written down in a formal way.

Scripts are not universally bad things, of course. They often provide compelling advantages. Scripts can save cognitive effort; the more my behaviour is scripted, the less I have to think, do research, make choices, or get confused. In the driving example, a certain degree of scripted behaviour (following the instructions) helps you to get where you’re going; another kind (using the turn signals reflexively) helps you to get along with other drivers; and yet another kind (staying within the lines) helps you to avoid collisions.

Yet if you followed any of those scripts rigidly, without applying your agency, you’d quickly get in trouble. Learning to drive involves learning how to follow certain scripts when they’re helpful, and to drop those scripts when they’re not helpful.

If you want to get to the party without harm to yourself or other people, you must bring your own agency to the task. You must stay vigilant, present, and attentive, making conscious and intentional choices. Scripts might influence your choices, and may even help you to make better choices, but they should not control you. As a human social agent—neither a zombie nor a robot—you must remain in control. Following a script within some part of an action means giving up engagement and responsibility for that part of the action—which is either a bug or a feature.

From time to time, testing might include formal testing. We define formal testing as testing that must be done in a specific way, or to check specific facts. On those occasions, formal scripting—especially the kind of formal script followed by a machine—might be a reasonable approach to enabling certain kinds of tasks and managing them successfully. A highly scripted approach could be helpful for rote activities like operating the product following explicitly declared steps and then checking for specific outputs. A entirely scripted approach—a set of instructions written in code to be performed by a machine—might also enable or extend certain kinds of variation—randomizing data, for example.

There are many other activities in testing that cannot be formally scripted in a helpful way. These include learning about the product, designing a test strategy, interviewing a domain expert, recognizing a new risk, investigating a bug—and dealing with problems that we encounter in formally scripted activities. In those cases, variability and adaptation are essential, and an overly formal approach is likely to be damaging, time-consuming, or outright impossible.

Here is something else that is almost never formally scripted: the actions and experiences of normal people using software.

Notice on the one hand that formal testing is, by its nature and by our definition, scripted. Most of the time, scripting constrains or even prevents exploration by constraining variation. On the other hand, if you want to make really good decisions about what to test formally, how to test formally, why to test formally, it helps enormously to learn about the product in unscripted and informal ways: conversation, experimentation, investigation…

Excellent scripted testing and excellent checking are rooted in exploratory work. They begin with exploratory work and depend on exploratory work. To use language as Harry Collins might, scripted testing is parasitic on exploration.

We say that any testing worthy of the name is fundamentally exploratory. We say that to test a product means to evaluate it by learning about it through experiencing, experimenting and exploring.

To explore a product means to investigate it, to examine it, to create and travel over maps and models of it. Testing includes studying the product, modeling it, questioning it, making inferences about it, operating it, observing it. Testing includes reporting, which itself includes choosing what to report and how to contextualize it. We believe these activities cannot be encoded in explicit procedural scripting in the narrow sense that I mentioned earlier, even though they are all scripted to some degree in the more general sense.

Excellent testing—excellent learning—requires us to think and to make choices, which includes thinking about what might be scripting us, and deciding whether to control those scripts or to be controlled by them. We must remain aware of the factors that are scripting us so that we can manage them, taking advantage of them when they help and resisting them when they interfere with our mission.