This is a lightly-edited excerpt from a longer blog post that you’ll find here. I recommend you read it too, but if you’re short on time, here’s the core of it.
If you care about understanding the status of your product, you’ll probably care about testing it. You’ll want testing to find out if the product you’ve got is the product you want. If you care about that, you need to understand some important things about testing.
If you want to understand important things about testing, you’ll want to consider some things that commonly get swept a carpet with the words “manual testing” repeatedly printed on it. Considering those things might require naming some aspects of testing that you haven’t named before.
Experiential Testing
Did someone just say “manual testing”? Maybe they were thinking about experiential testing, in which the tester’s encounter with the product, and the actions that the tester performs, are indistinguishable from those of the contemplated user. After all, a product is not just its code, and not just virtual objects on a screen. A software product is the experience that we provide for people, as those people try to accomplish a task, fulfill a desire, enjoy a game, make money, converse with people, obtain a mortgage, learn new things, get out of prison…
Contrast experiential testing with instrumented testing. Instrumented testing is testing wherein some medium (some tool, technology, or mechanism) gets in between the tester and the naturalistic encounter with and experience of the product. Instrumentation alters, or accelerates, or reframes, or distorts; in some ways helpfully, in other ways less so. We must remain aware of the effects, both desirable and undesirable, that instrumention brings to our testing.
Interactive Testing
Are you saying “manual testing”? You might be referring to the interactive, attended or engaged aspects of testing. In that mode, the tester is directly and immediately observing and analyzing aspects of the product and its behaviour in the moment that the behaviour happens. The tester may be controlling the product, either directly through the user interface, or indirectly through some kind of tool.
Having noticed that there are times when you observe and interact directly with the product in the moment, you might want to contrast that with testing work that doesn’t require your immediate presence or control. Some tasks can be handled by the algorithmic, mechanistic things that machines can do unattended. Some of these are things that some people label “automated testing”—except that testing cannot be automated. Input can be provided, functions triggered, and output checked automatically. That’s good, but at best it’s only part of a test. To make something a test requires the design before the automated behaviour, and the interpretation afterwards. Those parts of the test, which depend upon human social competence to make a judgement, cannot be automated.
Exploratory Testing
Did you say “manual”? You might be referring to exploratory work, which is interestingly distinct from experiential work as described above. Exploratory—in the Rapid Software Testing namespace at least—refers to agency; who or what is in charge of making choices about the testing, from moment to moment. There’s much more to read about that. Contrast that with scripted testing, in which something from the past controls or constrains the tester’s choices.
Wait… how are experiential and exploratory testing not the same?
It’s a good idea to test like a user. That’s only part of testing like a tester. You could be exploring—making unscripted choices—in an interactive way that is entirely unlike the user’s normal encounter with the product. For instance: you could be generating mounds of data and interacting with the product to stress it out; or you could be exploring while attempting to starve the product of resources. You could be performing an action and then analyzing the data produced by the product to find problems, at each moment remaining in charge of your choices, without control by a formal, procedural script.
That is, you could be exploring while encountering the product to learn about it, to probe it, to perform research on it, to investigate a bug that you’ve discovered. Those are all great things, but they’re all encountering the product like a tester, rather than like a user. It might be a really good idea to be aware of the differences between those two encounters, and take advantage of them, and not mix them up.
You could be doing experiential testing in a highly scripted, much-less-exploratory kind of way. For instance, you could be following a user-targeted tutorial and walking through each of its steps to observe inconsistencies between the tutorial and the product’s behaviour. To an outsider, your encounter would look pretty much like a user’s encounter; the outsider would see you interacting with the product in a naturalistic way, for the most part—except for the moments where you’re recording observations, bugs, issues, risks, and test ideas. But most observers outside of testing’s form of life won’t notice those moments, which makes it easier to dismiss them.
Of course, there’s overlap between those two kinds of encounters. A key difference is that the tester, upon encountering a problem, will investigate and report it. A user is much less likely to do so.
Again, please consider reading the original post for a more complete explication of the background and motivations for these distinctions — and a couple more besides.