There is a testing techique that I often apply. I have recently decided to name it the First Hurdle Heuristic. The basic idea: get the product out of the starter’s blocks, and see how it performs given a relatively easy challenge. This heuristic can useful when you want to identify problems and risks immediately, or to determine whether a product might not be ready for use or for deeper testing. … Read more
This article was inspired by a thread on LinkedIn a while back. Thank you to Rahul Parwal for starting the thread off. People sometimes suggest that requirements are unavailable, when what they really mean is that requirement documents aren’t available. That distinction is significant. (Plus, if requirement documents aren’t available… what are the programmers working from?) There might not be great requirements documents, but there are always requirements. There’s always … Read more
I very occasionally visit Xitter (pronounciation tip: it goes like the name of the President of the People’s Republic of China). The other day, Jason Huggins said Just in case you’re using a screen reader, that’s “I occasionally use the Tesseract OCR library for text recognition. I think that means I’m a senior machine learning engineer now, I guess.” I felt a little impish, but I also felt quite lazy. … Read more
With the current mania for AI-based systems, we’re finally starting to hear murmurs of moderation and the potential for risk. How do we test systems that incorporate an LLM? You already know how something about how to test LLM systems if you know how to test. Testing starts with doubt, and with a desire to look at things critically. The other day on LinkedIn, Paramjit Singh Aujla presented a problem … Read more
There’s a significant mistake that people might make when using LLMs to summarize a requirements document, or to produce a test report. LLMs aren’t all that great at summarizing. That’s definintely a problem, and it would be a mistake to trust an LLM’s summary without reviewing the original document. The bigger mistake is in believing that the output, the artifact, is the important thing. We might choose to share a … Read more
Are you interested in learning about something on the Web? How about learning about my approach and my services, Rapid Software Testing? You could use a search engine to look it up. But there’s a problem; it’s ominous; and it goes way beyond Rapid Software Testing. It has consequences for the entire world of online search. The problem came home to me on April 6, 2024. I had created a … Read more
To all managers and executives: despite how it’s in fashion these days, it’s not a good time to be laying off testers, or to be leaving them unprepared and untrained. Software can be wonderful. It can help us with all kinds of stuff, unimaginably quickly and at enormous scale. This sounds very appealing. Skilled testers, at least, have always known that we must treat output from machinery with appropriate skepticism … Read more
This happened to me again today in Quicken. It’s happened before. Worse, it’s an example of an extremely common phenomenon. My task here is to send an invoice to a particular person at a particular company. I fill in a part of a form — an address field — that appears right under the word “Invoice”. It’s not the topmost input element on the dialog, but it’s definitely the most naturalistic … Read more
An impressive article by Kara Swisher in the Washington Post includes this quote from French philosopher Paul Virilio: “When you invent the ship, you also invent the shipwreck; when you invent the plane, you also invent the plane crash; and when you invent electricity, you invent electrocution. Every technology carries its own negativity, which is invented at the same time as technical progress.” Paul Virilio, Philippe Petit, Sylvère Lotringer (1999). … Read more
Our principal job as testers is to discover the actual status of the product. Remember: everyone else on the project is focused on Success, Preventing Problems, Building Quality In, Adding Value, and all that. Those are all good things, and there’s nothing wrong with them. But there is a catch. The optimistic focus required to build the product can direct people’s attention away from problems that threaten its quality and … Read more