Measurement, Metrics, and Magic Numbers
Tutorial at the 2023 EuroSTAR Software Testing Conference June 13-16, 2023
For more information and to register
Measurement, Metrics, and Magic Numbers
- Bugs that could have been found in requirements gathering cost 1000 times more when found in production.
- Testing costs 40% of the total development effort.
- When the fix rate trendline crosses the failure rate trendline, it’s time to release the product.
- The ROI of testing can be calculated by a straightforward formula.
- Risk = Probability x Impact
- Defect Removal Efficiency is a good way to evaluate the quality of testing.
- You can’t manage what you can’t measure.
If you’re a tester or test manager, you’ve probably seen someone pull numbers out of a hat. As part of the trick, skepticism often vanishes into thin air.
Numbers can have magical powers. In the right hands, numbers can reveal remarkable things that might otherwise be invisible. Yet numbers may also have enormous power to dazzle, distract, and deceive. How do you know that what you seem to be seeing is real, and not just an illusion?
Since software development is real, and not magical, we must apply critical thinking – thinking about thinking to avoid being fooled. This half day tutorial, presented by Michael Bolton, is designed to help you learn strategies and skills – including questioning, modeling, analysis, and systems thinking, to help you think critically, confidently and thoughtfully about numbers.
In the tutorial, we’ll learn about reliability and validity in measurement and metrics. To get started, we’ll examine some claims from tool vendors and measurement magicians. We’ll analyze some common testing metrics, and learn how to recognize when they’re dangerous, based on smoke and mirrors. We’ll consider circumstances in which we might apply numbers appropriately. Finally, we’ll develop ways to provide reasonable, helpful, answers when people ask for metrics, KPIs, or estimates.
**NB** Participants are encouraged to bring a Windows-based laptop computer to the workshop.
Michael will be attending the rest of the conference and loves to talk about testing. Contact Michael to meetup.