Managers are responsible for hiring testers, for training them, and for removing any obstacles that make testing harder or slower. Managers are also responsible for hiring developers and designers, and providing appropriate training when it’s needed. If there are problems in development, managers are responsible for helping the developers to address them.
Managers are also responsible for the scope of the product, the budget, the staffing, and the schedule. As such, they’re responsible for maintaining awareness of the product, of product development, and anything that threatens the value of either of these. Finally, managers are responsible for the release decision: is this product ready for deployment or release into the market?
Misbegotten metrics like “Defect Detection Percentage” (I won’t dignify references to them with a link) continue to plague the software development world, and are sometimes used to evaluate “testing effectiveness”. But since it’s management’s job to understand the product and to decide when the product ships, a too-low defect detection percentage suggests the possibility of development or testing problems, unaware management, or a rash shipping decision. Testers don’t decide whether or when to ship the product; that’s management’s responsibility. In other words: Defect Detection Percentage—to the degree that it has any validity at all—measures management effectiveness.
On the point of the metrics misuse, I completely agree. Quantifying the qualifiable is a dead end, lacking a noble goal each and every time.
Let’s chat about ownership of releases though. I know it’s somewhat a side note, but perhaps I need you to clarify what you mean about managers having release ownership.
Traditionally, yes it’s been management’s job to say when a product or release occurs; however I think (and hope) that’s evolving across the industry. At my company, it’s the team (none of which are managers) that decides when to push the big red button. Epic-level deadlines are shied away from, in favor of a ‘release when ready’ model. I’m a fan of teams owning the release, with testers being the go/no-go lynch pin that ultimately makes that call. Sometimes this has to be done in conjunction with the product owner to determine showstopper priority when the tester is lacking subject matter expertise (e.g. If a defect will require a rollback or not within the release window).
Michael replies: So when, exactly, did the testers become responsible for what is manifestly a management and business decision? When, exactly, did the testers start to weigh all of the cost, value, and risk factors (all of them, not just the technical ones) that contribute to a business decision? When, exactly did the product owner cede ownership of the product to people who are not product owners? What, precisely, does he/she get paid for? If you, the tester, are making management decisions, why aren’t you getting paid like a manager? Why don’t you have the other responsibilities of the manager?
In short, management should know more about how to coach, mentor and give autonomy than about the actual product readiness itself.
I believe that management should know how to coach, mentor, and give autonomy; sure. But that doesn’t mean giving technical service providers (which testers and developers are) the reins of the business, unless we decide all of a sudden that “management” means something dramatically different from managing the business.
While I believe they should be aware of that, it is ultimately the team’s responsibility or own the go button right? I suppose management can be heavy handed and say ‘Prove it to me’ every time, but ideally we’d hire smart professionals to do a job we can trust them to do.
What does it mean to hire someone to do a job or perform a service for you? Imagine you’re having renovations done to your house. Do you leave it to the contractor to decide when to down tools, to determine whether you’re satisfied with the service that they’ve provided? “You’re done whenever you feel like you’re done; finish without putting down the floorboards, or stay a few weeks extra and gold-plate the faucets as you see fit.” “Listen, you’re the waiter, and he’s the cook; you guys tell me when I’ve had enough to eat—or when I’ve had too much.”
No; we are providing a service. Smart professionals offer their clients something that the smart professionals might believe is a finished product, but smart clients decide whether to accept that offer; whether that product is appropriate to fulfill their needs.
Don’t get me wrong; a manager who ignores the information provided to him or her by the team is foolish. But that information is only part of what management must evaluate in order to run the business. Testers, in particular, are extensions of the managers’ sensory systems. We help managers see things that are very small or far away when those things are important. We help managers hear and smell things that they might not hear or smell on their own. But we are sense organs for the project; we are not the brain. And having had some life experience, I presume you know the mischief that follows when your eyes make your decisions for you.
Perhaps we’re on the same page already and this blog was simply too short to elaborate, but I’d love to hear your thoughts either way.
My thoughts are made pretty clear here. Moreover, testers should stop pretending that they’re in the management business.
[…] Defective Detection Effectiveness Written by: Michael Bolton […]
I have read that post before, and will put my thoughts there, as my comments more relate to that post’s main theme.
[…] Blog: Very Short Blog Posts (29): Defective Detection Effectiveness – Michael Bolton – http://www.developsense.com/blog/2015/07/very-short-blog-posts-29-defective-detection-effectiveness/ […]
The response from Connor raises many questions; even beyond those that Michael already answered.
So, questions to Connor:
1. What do /you/ mean by managers? Management responsibility (and accountability, if there is any) changes as per context.
2. You said,”I’m a fan of teams owning the release, with testers being the go/no-go lynch pin that ultimately makes that call. Sometimes this has to be done in conjunction with the product owner to determine showstopper priority when the tester is lacking subject matter expertise.”
Just adding to the questions that Michael has already asked:
If teams own the release and it screw up, whose accountability would that be; product owner’s or team’s?
If testers become the lynch pin, what happens if they miss testing certain requirements (missing or hidden or ambiguous or incomplete or..) and still make a ‘go’ decision?
Who determines whether a tester is lacking subject matter expertise and how would they determine whether an area is missing or whether a tester is lacking in expertise? If they know about tester’s expertise (or lack of), would the tester still be the lynch pin (in this case, a faulty one)? Who makes the decision in that case?
You said,” I suppose management can be heavy handed and say ‘Prove it to me’ every time, but..”
My current CIO often says,’Prove it to me’, Which I find helpful. If you can’t prove it to the management, how would you prove it to the customer? Anyway, who are technical teams to make decisions on behalf of business who actually own the accountability? Technical teams often lack the holistic view that business owners have. Our focus mostly is on technical areas as we are experts in /that/ and not in business decision making.
Michael – I have to say I’ve really enjoyed these short, and to-the-point very short blog posts, which have provided a lot of food for thought.
Today – for very personal reasons – this really helped me rethink what’s going on in a current engagement.
Michael replies: Delighted to hear it. Knowing that these posts are helpful is the best reward I can think of.
@Rajesh says: If teams own the release and it screw up, whose accountability would that be; product owner’s or team’s?
> It depends. What was screwed up? What was the context of the mistake? If a test was run but the result was wrong, if there are faults that were known and it was released, if the product does not work in a way it wasn’t anticipated to be used, whose fault is those? Even then it depends on the context. A mature organization would figure that out – not a blame analysis, but a cause analysis – to learn & improve.
@Michael: I do not observe defect detection percentage. It tells me nothing. I look at risk, test to reduce it, then explain where we are at for remaining risk (the known unknowns, as Rumsfeld would call it).
It’s good that you state your position so clearly Michael. I don’t think we need to dig into the details much; it has been handled well in the comments already. After a reasonably long time in the software business, I do see models evolving. One, that you seem to suggest above, believes that management is “responsible” and that management needs to deal with special cause variation – that is, deal with exceptions.
There are other ways of thinking about management.
Another popular view of management is that they are more like the dean of science at a large university. These folks deal with administrative matters; their job is to enable the technical staff to get things done, not to give final sign-off on the curricula for a graduate course in biology when their background is slightly out of date chemistry. In this environment, advanced technical staff, say the endowed chair recipient doing real research, can often get greater accolades, and get paid more, than the “management” “above” them.
Or, to use your analogy, sometimes the tech team is the cook, and they definitely do decide whether the food is done cooking and ready to send out on a plate. And that is okay.
Michael replies: And the management of the restaurant has ultimate responsibility for every plate that goes out. If someone is sufficiently displeased, and the waiters and cooks don’t handle the problem appropriately, that person will go to the management. If someone gets sick, the authorities will hold the management responsible, even if the problem derived from a rogue cook who refused to wash his hands; it’s management that’s responsible for making sure he does that.
For what it’s worth, I do get paid like a manager, and started getting paid that way when I started working with a team that distributed many of the responsibilities you are putting into the management bucket.
I respect what you are saying as one way to run a business; I just don’t see it as the only way.
Even businesses that are run as co-operatives have a board. Churches have wardens. Towns have mayors. And in a university, there are deans and boards of regents precisely so that someone has final authority to decide what will and will not be on the curriculum. That is orthogonal to the issues of accolades or salary. Don’t confuse prestige or pay with responsibility. When a professor says or does something outrageous, the quotes come off the “management” “above” them; the professor will be held responsible by people with a higher level of responsibility.
Thank you for getting the conversation going!