I regularly converse with people who say they want to introduce exploratory testing in their organization. They say that up until now, they’ve only used a scripted approach.
I reply that exploratory testing is already going on all the time at your organization. It’s just that no one notices, perhaps because they call it
- “review”, or
- “designing scripts”, or
- “getting ready to test”, or
- “investigating a bug”, or
- “working around a problem in the script”, or
- “retesting around the bug fix”, or
- “going off the script, just for a moment”, or
- “realizing the significance of what a programmer said in the hallway, and trying it out on the system”, or
- “pausing for a second to look something up”, or
- “test-driven development”, or
- “Hey, watch this!”, or
- “I’m learning how to use the product”, or
- “I’m shaking out it a bit”, or
- “Wait, let’s do this test first instead of that test”, or
- “Hey, I wonder what would happen if…”, or
- “Is that really the right phone number?”, or
- “Bag it, let’s just play around for a while”, or
- “How come what the script says and what the programmer says and what the spec says are all different from each other?”, or
- “Geez, this feature is too broken to make further testing worthwhile; I’m going to go to talk to the programmer”, or
- “I’m training that new tester in how to use this product”, or
- “You know, we could automate that; let’s try to write a quickie Perl script right now”, or
- “Sure, I can test that…just gimme a sec”, or
- “Wow… that looks like it could be a problem; I think I’ll write a quick note about that to remind me to talk to my test lead”, or
- “Jimmy, I’m confused… could you help me interpret what’s going on on this screen?”, or
- “Why are we always using ‘tester’ as the login account? Let’s try ‘tester2’ today”, or
- “Hey, I could cancel this dialog and bring it up again and cancel it again and bring it up again”, or
- “Cool! The return value for each call in this library is the round-trip transaction time—and look at these four transactions that took thirty times longer than average!”, or
- “Holy frijoles! It blew up! I wonder if I can make it blow up even worse!”, or
- “Let’s install this and see how it works”, or
- “Weird… that’s not what the Help file says”, or
- “That could be a cool tool; I’m going to try it when I get home”, or
- “I’m sitting with a new tester, helping her to learn the product”, or (and this is the big one)
- “I’m preparing a test script.”
Now it’s possible that none of that stuff ever happens in your organization. Or maybe people aren’t paying attention or don’t know how to observe testing. Or both.
Then, just before I posted this blog entry, James Bach offered me two more sure-fire clues that people are doing exploratory testing: they say, “I am in no way doing exploratory testing”, or “we’re doing only highly rigorous formal testing”. In both cases, the emphatic nature of the claim guarantees that the claimant is not sufficiently observant about testing to realize that exploratory testing is happening all around them.
Update, October 12, 2015: In fact, in the Rapid Software Testing namespace, we now maintain it’s redundant to say “exploratory testing”, in the same way it’s redundant to say “carbon-based human” or “vegetarian potato”. It is formal scripting—not exploration‐that is the interloper on testing’s territory. We explain that here.
Good point.
My first project was like that. I, as well as the other testers, used an exploratory approach in my work. But the test lead refused to acknowledge it. I attended a couple of seminars and lectures about ET with the test team and the response afterwards was always the same: “Well yes, this is probably a great approach but not in our business”.
Michael replies: Some people really never, ever learn, do they? Perhaps they never learn because they never observe what’s going on right in front of them. To people who say “Well yes, this is probably a great approach but not in our business”, I would ask, how do the managers work in your business? I think, if they observed more closely, they might see some parallels between testing and management.
As usual, James Bach would be a little more blunt.
Isn’t it dangerous to call pretty much everything exploratory testing? Won’t the term itself loose value if it can be reduced down to “doing software development”?
Michael replies: I think the risk of losing value is far greater if we reduce exploratory testing down to “test execution by banging on a keyboard in undocumented, unstructured, strictly manual, and unprofessional manner”. The ISTQB and the IEEE do a similar thing when they reduce “test design” to “documentation” or “error guessing”.
I’m not calling pretty much everything exploratory testing. For example, I’m famously adamant on the difference between testing and checking, and while programming involves exploratory activities, some of those activities aren’t what we’d call testing. I am, though, pointing out that many of the things that people do in the process of developing and testing software are highly exploratory and unscripted activities. There’s a point to this: that if we want to understand and improve on the nature of those activities, we need to engage in study of exploration and learning.
When they say they introduce exploratory testing I think they are talking about the approach, where a common (unhealthy) approach is the scripted one. With that you have the perception on testing, test organisation, mindset of the testers and so on.
Cem Kaner makes a good comparison in A Tutorial in Exploratory Testing (http://www.kaner.com/pdfs/QAIExploring.pdf).
I agree that everyone do Exploratory testing, but not everyone sees testing the same way. Four schools of testing is another way of looking at it. Perhaps it is that transition they actually describe?
Have we limited ourselves by putting too many things under the roof of ET?
I’ve seen this question before. I find the response from exploratory testers puzzling (Tester: I have a problem, I need your help. Etester: YOU DO NOT HAVE A PROBLEM!!)
This post has some good ideas about how we do exploratory testing whether we consciously try to do it or not. However, I don’t think that is the question being asked everytime.
I think sometimes this is the question being asked (I am speaking on behalf of others):
“Currently we script testcases. Progress is measured by number of testcases complete. When we deviate from the test steps, the PM and developers are puzzled why we do that. When test is lagging we add additional testers to catch up. In some cases the additional testers are developers. They read the testcases and *execute* them with *blinders on*.
Is there a problem? Not for anyone who chooses not to ask. The support department handles customer issues. We don’t have visiblity on what happens there.
Do any of the testers see a problem? Not if they choose not to ask.
We have people who have been doing this for 20 years. Some of them are executives.
I think exploratory testing can drastically improve the impact of the test team. However, I think it would be disruptive in our environment. How can you I introduce such drastic change in my environment. (btw, the *change* I am referring to is not how we test. Even if I agree that we are doing exploratory testing (I don’t), the *change* I am referring to is in the *management* and the environment/organization.”
—–
As an aside, I *think* that the environment I have described is quite common. It is perfectly in line with some of the common test process frameworks.
just a different point of view…..
For me, Exploratory Testing is the most natural way of going forward with testing.
[…] Exploratory Testing is All Around You […]
[…] uncomfortable. What is really important is to point out that they do exploratory testing as well [6], but perhaps not framing [7] or structuring it well. I’ve heard arguments that exploratory […]