Touring is one way of structuring exploratory testing, but exploratory testing is not necessarily touring, and touring is not necessarily exploratory.
At one extreme, a tourist might parachute into a territory for which there is no detailed knowledge of the landscape, flora and fauna, or human culture, with the goal of identifying what’s there to be learned. Except in such cases, we wouldn’t call her a tourist; we’d call her an anthropologist, or a field botanist, or a field geologist, or an archaeologist. The activity is in this case is interactive with the territory. At the other extreme, a tourist might visit a travel agent, get on a plane to Orlando, meet a chartered bus at the airport, and sit through the rides at Disney World. The touring activity there is largely passive. and we would call someone engaged in it a “tourist”—although “engagement” is something of an exaggeration here. For this kind of person, serious explorers and locals would probably use the word “tourist” in a somewhat deprecating kind of way.
Touring a program can be done in a more scripted or more exploratory way, just as touring a city can be done in a more scripted or more exploratory way. A tourist has many options. Before going on a trip, a tourist might study what is already known about a particular destination. To prepare, she might supply herself with maps and travel guides, and some ideas about destinations of interest. Upon arrival, she might choose a set of walking tours from a guidebook and follow the routes closely, eating only at the restaurants identified in the guidebook, noting buildings and artifacts and other objects of interest by matching them with the descriptions and illustrations. At a given site, she might listen to a prepared audio guide that directs her observations very specifically. She might spend all of her time in the presence of a tour guide who tells her what to observe and how to interpret it. She might choose to accept everything the tour guide told her as the complete story, and refrain from asking questions. Even though the experience would be new to her, and she might learn something from it, she would not likely add much to what is already known. We call that activity touring, but it isn’t very exploratory, and a report on such a tour would largely recapitulate the guidebook. Is your testing like that?
On the other hand, rather than touring like a tourist, she might cover a territory as a historian, or a social scientist, or a travel writer. In that kind of role, she would have a research goal based on the idea of obtaining new knowledge. Learning something new and imparting it to other people requires a more open agenda than sitting on the bus while someone or something else directs your attention. Our researcher might make her way directly to particular destinations or landmarks and begin her research there, or she might amble through neighbourhoods or historical sites to discover new things about them. She could choose to focus on specific aspects of what’s there to observe, or she could choose to let the observations come to her—and, of course, she might do both. She might work with descriptions that she had been given with the intention of adding to them, or she might work from a set of questions that haven’t been asked before. Depending on her mission, she might choose to look for specific patterns or problems, or she might seek deeper understanding that would help her to identify or refine what kind of patterns or problems to look for. Even though the mission to discover new information might come from someone else, she remains in control of the specifics of the itinerary and of each activity from one moment to the next. Is your testing more like that?
One of the hallmarks of exploratory activity is the extent to which it is guided and structured by the person performing that activity. Another hallmark is the extent to which new knowledge feeds into choice of which action to perform next. Touring is not equivalent to exploration; touring can be done is a scripted way or an exploratory way.
This series continues with What Exploratory Testing Is Not (Part 2): After-Everything-Else Testing.
And, of course, in the face of all the forthcoming instances of what exploratory testing is not, you might want to know our current take on what exploratory testing is.
Correction: Disneyland is in L.A., Disney World is in Orlando. =)
Michael replies: Doh! I initially thought of Disneyland, but the rides at EPCOT are even more rigidly guided and less interactive. Fixed. Thanks for spotting it.
In your last paragaph you say touring isn’t equivalent to exploration, don’t you mean to say it isn’t “always” equivalent? It can be but as you stated above its just one way of structing exploration (or exploratory testing).
No, I meant what I said. A way of structuring something is not the same as the thing itself. In music, twelve-bar blues is a pattern or structure for writing or playing or performing music, but it’s not music itself. A map of a city shows some structures of the city, but it is not the city. Be careful of reification error.
Thanks for the comment and the question.
Interesting take on it Michael. I guess everyone can do good or bad testing depending upon how they make use any technique.
Michael replies: Yes. But there are some approaches to testing that can inadvertently make things worse, even though the intention is to make things better. As James Bach says, here, “You can’t overcome lack of basic skills with written instructions.”
I once read a whole book about tours, which was supposed to be about exploratory testing. I couldn’t finish it though, but I didn’t learn a lot about London Tour Guides.
What’s your take on using checklists of test ideas?
Michael replies: I like the idea of checklists as a means of helping to make sure that important testing gets done, while at the same time affording freedom and responsibility for the tester to explore and investigate. Checklists, when well done, can add structure without overly structuring the work. Cem Kaner covers the distinction between scripted and checklisted approaches very thoroughly in his presentation “The Value of Checklists and the Danger of Scripts: What Legal Training Suggests for Testers.”
interesting analogy, thanks.
where does adhoc fit in? subset of exploratory? nothing more than a monkey on a keyboard?
Michael replies: I don’t know what you mean by “ad hoc”, so I can’t answer those questions in a helpful way. If you can tell me what you mean by “ad hoc”, I can compare and contrast your description with my notion of exploratory testing. That said, some thoughts on the subject appear about eight paragraphs down here.
[…] Exploratory Testing Is Not (Part 1): […]
Suppose a tourist goes without tour guide and explores the city. Sometimes, there is a chance missing important or special tourist spots. Is that my prdiction rite. If so, is it possible to happen in exploratory testing?
Michael replies: Of course it’s possible. It’s possible no matter what.
In an exploratory approach, your strategies and tactics may guide you generally what to observe, and yet (because of distraction or inattention, for example) you may fail to notice it anyway. Your strategies and tactics might fail to identify important or special things. You may apply an invalid oracle, causing you to believe that something isn’t a problem when it is, or is a problem when it isn’t. You may be required to investigate and report bugs that you’ve found, leading you to reduce your coverage given your time constraints.
If you’re using a scripted approach, the script may tell you quite specifically what to observe, and yet (because of distraction or inattention, for example) you may fail to notice it anyway. The script itself may fail to identify important or special things. The script might be based on an invalid oracle, causing you to believe that something isn’t a problem when it is, or is a problem when it isn’t. You may be required to investigate and report bugs that you’ve found, leading you to reduce your coverage given your time constraints.
Since problems can appear in both cases, what do you do to make your testing robust to these problems? You manage it. If there are specific things to look for, note them. If you’re concerned that you might miss something, prepare carefully. Diversify your team and your tactics. Diversify your oracles. Dynamically manage your focus; alternate between focusing and defocusing heuristics. Sharpen your reporting skills so as to provide the most economical report you can while still providing the information that the programmers need to address the problems that you’ve discovered.
One key difference between an exploratory vs. a scripted approach: in the exploratory approach, the tester is mandated to exercise the freedom and responsibility to optimize the quality of the work (that’s a central clause in Cem Kaner’s definition of exploratory testing). The feedback loop inherent in an exploratory approach is an important contributor to resolving the problems that you allude to above.
>suppose, once exploratory testing done for a product & successfully launched. After a while, same product has been enhanced & released for testing. At the same time, previuosly tested tester cant test the enhanced product for any reason. Is that new tester able to test enhanced product with available exploratory testing documents? If so, what kind of documentation should prepare for exploratory testing?
Michael replies. I’m not sure I understand the questions. To the degree that I believe I understand them, I’ll answer that exploratory testing can be informed by any document, model, sketch, idea, prototype, product, code fragment, spreadsheet, story, model, law, standard, or conversation that the tester evaluates as being relevant to the mission that the tester has been given. This is investigative work, guided by the mission and by critical thinking. Anything that’s useful is in scope.
[…] What Exploratory Testing Is Not by Michael Bolton – a fantastic five part blog series which touches on the major fallacies about ET. It is not touring, it is not after-everything-else-testing, it is not tool-free testing, it is not quick testing and it is (most definitely) not undocumented testing. […]
[…] What Exploratory Testing is Not (Part 1): Touring […]
[…] Part 1: Touring […]
[…] http://www.developsense.com/blog/2011/12/what-exploratory-testing-is-not-part-1-touring […]