Until today, my reading of Mary Reilly’s Play As Exploratory Learning had been limited to occasional stolen glances into Cem Kaner’s library, but a copy of this out-of-print book arrived today. Browsing (that is, a little exploratory reading) yielded this (from Chapter 3, “An Explanation of Play”, page 117):
“The pursuit of the rumored goodness and usefulness that play might have for man is plagued by the difficulties inherent in the processes of exploration. One of the first problems is the very obviousness of play. It is a behaviour endlessly in plain sight, and because it is a behaviour there in plain sight it lacks the intrigue that the unknown has for scientists. Intellectuals go to any lengths to avoid the obvious and theorists in particular disdain it. The real difficulty that confronts any broadened explanation of play is that it is an obvious commonplace behaviour breeding contempt dampening investigatory interest.”
Well, ain’t that just like exploratory approaches to testing? (For “play” read “exploratory testing”; for “scientists” and “theorists”, read “many programmers”; and for “intellectuals” read “traditional testing theorists”.) Explaining exploratory testing is hard because we appear to be simply “trying the program to see if it works”. What’s not obvious to the untrained observer is the stuff that’s going on behind the eyeballs: modeling the test space; determining coverage, oracles, and procedures; and making decisions about how to configure, operate, observe, and evaluate the product. It would be nice if other people could see all that stuff, but we can’t. So, as a community, those of us who recognize the value of exploratory approaches—that is, the fact that they’re central to excellent testing—are going to have to keep talking about them.
What’s interesting is I often use the word “play” to quickly communicate what I’m doing. For instance, today, I was trying to communicate that my team would explore a certain area to see how it stands up — my quick response was, “We’ll play with it and see what comes up.” I use this term a lot, as I think it both communicates what we’re doing, and that we like our job. 🙂
Are things like Session Based Test Management a step in the direction of making the process more visible?
I like exploratory testing, but it lacks a lot of the artifacts that have become expected of a testing group as promoted by more established test methods.
For project managers who like test plans and lots of documentation, exploratory testing would seem to be a poor fit without some education. And even then it would be a tough road.
One thing I see lacking in the testing field is that there is a lot of talk about great methods, but most of the time it assumes an environment where there is strong support for the testing function. How do you test in an environment where the rest of the engineering functions are hostile to your methods?
Luckily, I’m not in such a situation now, but I left my last job because of a profound, and previously hidden, difference in testing philosophy between myself as the test lead and the project manager. The project manager felt that if bugs got through it was the testers fault, never mind the developer who wrote the bug in the first place. In his case he took the lack of traditional test documentation as a sign of a poor testing process. Unfortunate.
Tim said: What’s interesting is I often use the word “play” to quickly communicate what I’m doing…I think it both communicates what we’re doing, and that we like our job. 🙂
Fair enough, but I’d be careful to consider the audience. As Mary Reilly suggests, play often isn’t taken seriously, even though it ought to be.
I have a three-year-old daughter, and watching her play is watching her learning. Testing is learning, too.
Anonymous said: Are things like Session Based Test Management a step in the direction of making the process more visible? I like exploratory testing, but it lacks a lot of the artifacts that have become expected of a testing group as promoted by more established test methods.
Anon, thanks for this comment. I’m having trouble answering it–not because I don’t have an answer, but because I have so many. It’s worth a blog post of its own. Stay tuned.