Several years ago, I worked for a few weeks as a tester on a big retail project. The project was spectacularly mismanaged, already a year behind schedule by the time I arrived. Just before I left, the oft-revised target date slipped by another three months. Three months later, the project was deployed, then pulled out of production for another six months to be fixed. Project managers and a CIO, among many others, lost their jobs. The company pinned an eight-figure loss on the project.
The software infrastructure was supplied by a big database company, and the software to glue everything together was supplied by development organization in another country. That software was an embarrassment—bloated, incoherent, hard to use, and buggy. Fixes were rarely complete and often introduced new bugs. At one point during my short tenure, all effective worked stopped for five days because the development organization’s servers crashed and no backups were available. All this despite the fact that the software development company claimed CMMI Level 5.
This morning, I was greeted by a Tweet that said
“Deloittes show how a level 5 CMMi company has bad test process at #TMMi conf in Korea! So CMMi needs TMMi – good.”
The TMMi is the Testing Maturity Model Integration. Here’s what the TMMi Foundation says about it:
“The Test Maturity Model Integration has been developed to complement the existing CMMI framework. It provides a structured presentation of maturity levels, allowing for standard TMMi assessments and certification, enabling a consistent deployment of the standards and the collection of industry metrics.”
Here’s what the SEI—the CMMi’s co-ordinator and sponsor—says about it:
“CMMI (Capability Maturity Model Integration) is a process improvement approach that provides organizations with the essential elements of effective processes, which will improve their performance. CMMI-based process improvement includes identifying your organization’s process strengths and weaknesses and making process changes to turn weaknesses into strengths.”
What conclusions could we draw from these three statements?
If a company has achieved CMMI Level 5, yet has a bad test process, then there’s a logical problem here. Either testing isn’t an essential element of effective processes (in which case the TMMI should be unnecessary) or it is (in which case the SEI’s claim of providing the essential processes is unsupportable).
One clear solution to the problem would be to adjudicate all this by way of a Maturity Model Maturity Model (Integrated), the MMMMI, whereby your organization can determine (in a mature fashion, of course) what essential processes are in the first place. Mind you, that could be flawed too. You’d need a set of essential processes to determine how to determine essential processes, so you’ll also need a Maturity Model Maturity Model Maturity Model (Integrated), an MMMMMMI. And in fairly short order, your organization will disappear up its own ass.
Jerry Weinberg points in a different direction, using very strong language. This is from Quality Software Management, Volume 1: Systems Thinking, p. 21:
…cultural patterns are not more or less mature, they are just more or less fitting. Of course, some people have an emotional need for perfection, and they will impose this emotional need on everything they do. Their comparisons have nothing to do with the organization’s problems, but with their own.
“The quest for unjustified perfection is not mature, but infantile.
“Hitler was quite clear on who was the ‘master race’. His definition of Aryan race was supposed to represent the mature end product of all human history, and that allowed Hitler and the Nazis to justify atrocities on “less mature” cultures such as Gypsies, Catholics, Jews, Poles, Czechs, and anyone else who got in their way. Many would-be reformers of software engineering require their ‘targets’ to confess to their previous inferiority. These little Hitlers have not been very successful.
“Very few healthy people will make such a confession voluntarily, and even concentration camps didn’t cause many people to change their minds. This is not ‘just a matter of words’. Words are essential to any change project because they give us models of the world as it was and as we hope it to be. So if your goal is changing an organization, start by dropping the comparisons such as those implied in the loaded term ‘maturity.'”
It’s time for us, the worldwide testing community, to urge Deloitte, the SEI, the TMMI, and the unfortunate testers in Korea who are presently being exposed to the nonsense to recognize what many of us have known for years: maturity models have it backwards.
Nice post, but I think you are still standing in the Antithesis of Software Project Management (Thesis: Predictive Project Management based on processes; Antithesis: Agile Project Management totally opposed to the thesis).
Michael’s reply: There’s an enormous difference between predicting and anticipating. There’s an equally enormous difference between planning and being prepared. There’s an enormous difference between valuing one thing over another, versus completing ignoring half of the picture. Predictive Project Management based on processes, if it means what you appear to be suggesting, says that we should make a prediction about the future (rather than anticipating a range of possible futures); that we should plan for that predicted future (rather than being prepared for a range of possible futures); that we should base management upon following a process (rather than recognizing that managed processes must serve and be aware of human values and human needs).
I would like to hear from you the Synthesis of this discussion, meaning that which are the tools, techniques and processes from both theories that are useful and beneficial to projects. Because, there is no such a thing like the perfect theory or recipe that resolves all our problems.
Yes. You know me, right? You know my work, right? You know that I’m this big anti-best-practice dude, right? This context-driven guy, right?
It’s important to know this: a context-driven tester doesn’t have to approve of or agree with all contexts and all situations, though it’s worthwhile to be adaptable and learn how to navigate through as many of them as possible. The basis of context-driven thinking is to consider context first. After we’ve considered context and people and missions and givens, it’s perfectly acceptable to deplore management styles that we find dehumanizing.
Meanwhile, I think in your description of where I’m “standing”, you’re mischaracterizing both my work and that of the serious Agilists out there. The synthesis is already in those approaches. I don’t identify myself as large-A Agile, but two kinds of values are noted in the two columns of the Agile Manifesto, and the priorities for each pair of values fit my world view. It’s got people in it, you see. The xMMi twins, it seems to me, pay attention to the right-hand column only. I’ve scoured the xMMi literature for the human element, and I can’t find any serious attention paid to the issue. My world view has two sides; the xMMi does not. A “synthesis” of the two approaches, it seems to me, would balance matters by mostly ignoring people, or partially noticing people.
I’m not opposed to the idea that processes might be important. I’m opposed to worship of them, or placing them first, or leaving them in the centre of the discussion. The Manifesto says a similar thing: “That is, while there is value in the items on the right, we value the items on the left more.” I’m also tired in the extreme of the view of software development and testing processes as linear, rather than organic; an assembly line, rather than a lab or a studio. The least the xMMi proponents could do is watch how people actually work, invent, create, and learn, rather than promoting neo-Platonist models—one-dimensional maps of rich, complex, human territories.
[…] xMMwhy […]
Yesterday I attened a testing conference orgznized by CSTQB. A korea consultant proudly announced he has finished 35 TMMi assessment in Korea.
Although we know the drawback of standards and assessment, still, a lot of people put these things their main focus to work on. Still many people are eager to making more standards, more processes, more assessment…
What’s worst, many general people don’t have the ability to diffirenciate what is essential in testing from what is not.
One would hope that a standard would do that well. There’s a big difference between improved testing and increased documentation.
“And in fairly short order, your organization will disappear up its own ass.”
I laughed out loud mate, and therefore my colleagues are now aware of this post. Ha
I’ve never understood the xMMi twins… maybe because I can’t bring myself to read the 180 page explaination of each of them! If it truly is about continuous improvement then how can it stop at 5… surely it must just go on, and on, and on…