There’s a significant mistake that people might make when using LLMs to summarize a requirements document, or to produce a test report. LLMs aren’t all that great at summarizing. That’s definintely a problem, and it would be a mistake to trust an LLM’s summary without reviewing the original document. The bigger mistake is in believing that the output, the artifact, is the important thing. We might choose to share a … Read more
Are you interested in learning about something on the Web? How about learning about my approach and my services, Rapid Software Testing? You could use a search engine to look it up. But there’s a problem; it’s ominous; and it goes way beyond Rapid Software Testing. It has consequences for the entire world of online search. The problem came home to me on April 6, 2024. I had created a … Read more
A prototype of this post originally appeared on LinkedIn. Today I was using Microsoft Word, and for the first time I took a look at a feature that’s probably been there for a long while. Also today, there’s at least one more LinkedIn poll with an interview question — apparently aimed at testers — on a fairly trivial aspect of Java programming. Questions of that nature might reasonable if the … Read more
To testers (and to designers, developers, and managers, and others who might do testing), this is your periodic reminder to do things out of order. By “do things out of order”, I mean: perform sequences of actions that do not follow the designer or the developer’s imagination of how things should happen; how things normally happen; what the customers might want; or what’s easiest to process. Here’s an example from … Read more
In the Guardian, October 22 2023, John Naughton writes an article on the complications associated with moving semiconductor manufacturing from Taiwan to the United States. The article and the problems that it describes are pretty interesting. At the centre of the product is the difficulty of transferring tacit knowledge about chip fabrication. It turns out that documentation on its own doesn’t work very well. What matters most is having people … Read more
ChatGPT may produce inaccurate information about people, places, or facts. https://chat.openai.com/ Testing work comes with a problem: the more we test, the more we learn. The more we learn, the more we recognize other things to learn. When we investigate a problem, there’s a non-zero probability that we’ll encounter other problems — which in turn leads to the discovery of more problems. In the Rapid Software Testing namespace, we’ve come … Read more
Yet another post plucked and adapted from the walled garden of LinkedIn “What the large language models are good at is saying what an answer should sound like, which is different from what an answer should be.” —Rodney Brooks, https://spectrum.ieee.org/gpt-4-calm-down Note for testers and their clients: the problem that Rodney Brooks identifies with large language models applies to lots of test procedures and test results as well. People often have … Read more
“A test that is defined in terms of one expected result is undefined against the other types of results available from that test.” —Cem Kaner, 2004 (https://lnkd.in/gjFZYNGs) Almost 20 years on, that message is still lost on many testers, developers, and managers. Yet, as of today, we have another chance to acknowledge it and to spread the word! Excellent testing is not really about obtaining the expected result. Excellent testing … Read more
For a long time, I’ve suggested that testing should focus on product problems that pose risk to the business. That remains true, but lately I’m thinking there’s another consideration. For instance: yesterday, I accepted an invitation for an online meeting from a potential client. The invitation contained a link to a Microsoft Teams meeting. (If you know where this is going, and find it too painful, just skip to the … Read more
After 20 years of working together to develop the Rapid Software Testing approach, James Bach and I have decided that — improbable as it may seem — it’s time to wrap it all up. Perhaps this will be a surprise to our followers in the community, but we now must confront what we previously thought was unimaginable: recent developments in technology have, for all intents and purposes, made testing obsolete. … Read more