In response to my post, Testers: Get Out of the Quality Assurance Business, my colleague Adam White writes,
I want ask for your experience when you’ve your first 3 points for managers:
- Provide them with the information they need to make informed decisions, and then let them make the decisions.
- Remain fully aware that they’re making business decisions, not just technical ones.
- Know that the product doesn’t necessarily have to hew to your standard of quality.
In my experience I took this to an extreme. I got to the point where I wouldn’t give my opinion on whether or not we should ship the product because I clearly didn’t have all the information to make a business decision like this.
I don’t think that’s taking things to extreme. I think that’s professional and responsible behaviour for a tester.
Back in the 90s, I was a program manager for several important mass-market software products. As I said in the original post, I believe that I was pretty good at it. Several times, I tried to quit, and the company kept asking me to take on the role again. As a program manager, I would not have asked you for your opinion on a shipping decision. If you had merely told me, “You have to ship this product!” or “You can’t ship this product!” I would have thanked you for your opinion, and probed into why you thought that way. I would have also counselled you to frame your concerns in terms of threats to the value of the product, rather than telling me what I should or should not do. I likely would have explained the business reasons for my decisions; I liked to keep people on the team informed. But had you continued in telling me what decision I “should” be making, and had you done it stridently enough, I would have stopped thanking you, and I would have insisted to you (and, if you persisted, your manager) to stop telling me how to do my job.
On the other hand, I would have asked you constantly for solid technical information about the product. I would have expected it as your primary deliverable, I would have paid very close attention to it, and would have weighed it very seriously in my decisions.
This issue is related to what Cem Kaner calls “bug advocacy“. “Bug advocacy” is an easy label to misinterpret. The idea is not that you’re advocating in favour of bugs, of course, but I don’t believe that you’re advocating management to do something specific with respect to a particular bug, either. Instead, you’re “selling” bugs. You’re an advocate in the sense that you’re trying to show each bug as the most important bug it can be, presenting it in terms of its clearest manifestation, its richest story, its greatest possible impact, its maximum threat to the value of the product. Like a good salesman, you’re trying to sell the customer on all of the dimensions and features of the bug and trying to overcome all of the possible objections to the sale. A sale, in this case, results in a decision by management to address the bug. But (and this is a key point) like a good salesman, you’re not there to tell the customer that he must buy what you’re selling; you’re there to provide your your customers with information about the product that allows them to make the best choice for them. And (corresponding key point) a responsible customer doesn’t let a salesman make his decisions for him.
I started to get pushback from my boss and others in development that it’s my job to give this opinion.
“Please, Tester! I’m scared to commit! Will you please do my job for me? At least part of it?”
What do you think? Should testers give their opinion on whether or not to ship the product or should they take the approach of presenting “just the facts ma’am”?
I remember being in something like your position when I was at Quarterdeck in the late 1990s. The decision to ship a certain product lay with the product manager, which at the time was a marketing function. She was young, and ambitious, but (justifiably) nervous about whether to ship, so she asked the rest of us what she should do. I insisted that it was her decision; that, since she was the owner of the product, it was up to her to make the decision. The testers, also being asked to provide their opinion, also declined. Then she wanted to put it to a vote of the development team. We declined to vote; businesses aren’t democracies. Anticipating the way James Bach would answer the question (a couple of years before I met him), I answered more or less with “I’m in the technical domain. Making this kind of business decision is not a service that I offer”.
But it’s not that we were obstacles to the product, and it’s not that we were unhelpful. Here are the services that we did offer:
- We told her about what we considered the most serious problems in the product.
- We made back-of-the-envelope calculations of technical support burdens for those problems (and were specific about the uncertainty of those calculations).
- We contextualized those problems in terms of the corresponding benefits of the product.
- We answered any questions for which we had reliable information.
- We provided information about known features and limitations of competitive products.
- We offered rapid help in answering questions about the known unknowns.
Yet we insisted, respectfully, that the decision was hers to make. And in the end, she made it.
There are other potentially appropriate answers to the question, “Do you think we should ship this product?”
- “I don’t know,” is a very good one, since it’s inarguable; you don’t know, and they can’t tell you that you do.
- “Would anything I said change your decision?” is another, since if they’re sure of their decision, their answer would be the appropriate one: No. If the answer is Yes, then return to “I don’t know.”
- “What if I said Yes?” immediately followed by “What if I said No?” might kick-start some thinking about business business based alternatives.
- “Would I get a product owner’s position and salary if I gave you an answer?”, said with a smile and a wink might work in some rare, low-pressure cases although, typically, the product owner will have lost his sense of humour by the time you’re being asked for your ship/no-ship opinion.
As I said in the previous post, managers (and in particular, the product owners) are the brains of the project. Testers are sense organs, eyes for the project team. In dieting, car purchases, or romance, we know what happens when we let our eyes, instead of our brains, make decisions for us.