Michael Bolton Since June 2023, James Bach and I have been collecting a set of “syndromes” associated with certain forms of AI — chatbots based on Large Language Models (LLMs) and Generative Pre-trained Transformers (GPTs). The most prominent of these, at this writing is OpenAI’s ChatGPT. Today we added a new syndrome: Voldemort Syndrome. Today LinkedIn (and much of the rest of the internet) lit up over the “The Man Who Shall … Read more
Today I visited Facebook to post a notice of my upcoming trip to New Zealand. There will be three stops on the tour: Auckland (for Testers and Automation, Avoiding the Traps, February 17-19), Wellington (Testers and Automation, Avoiding the Traps, February 24-26), and Christchurch (Rapid Software Testing Explored, March 10-12). Facebook’s AI Layer (I’ll just call that FAIL) offered to turn it into an event. I accepted the offer, and … Read more
At the Pacific Northwest Software Quality Conference in October 2024, I gave a keynote presentation titled “What Are We Thinking in the Age of AI?“ There’s a lot to think about, and for testers, there’s a lot to do. For one, we need to understand the basis for the “AI” claim. Any kind of software can be marketed as “AI”, since it’s doing something that (presumably) a human could do, … Read more
Every day, in some discussion about testing, someone talks about the need for “balance between automated testing and manual testing”. This seems to me to be a supremely unhelpful way to think about testing work. First, and once again, testing is neither manual nor automated. No one in any other cogntive, intellectual, investigative domain talks about their work that way; and no one in any such domain allows other people … Read more
Please remember: there’s a big difference between quality and testing; and so there’s a big difference between a quality strategy and a testing strategy. Understand the Nature of Quality The essence of quality is value to people. A quality strategy is a set of guiding ideas for building a product or service, in order to achieve the goal(s) of providing value to people. To develop a successful product, the people … Read more
(Yet another post that started on LinkedIn…) There’s lots of advice out there on how to use Large Language Models to generate test ideas, test data, or test cases. Everything I’ve done and seen myself suggests that the test ideas from LLMs are pretty generic, banal, and uninspired. Considering how LLMs work, this is unsurprising. The majority of LLMs are trained on testing material from the Web, where the overwhelming … Read more
This is a lightly-edited version of a post I made on LinkedIn, which in turn was a followup to the previous post. “We have to run a full regression test suite on every build!” First: you don’t have to do anything. There is no law of nature, nor any human regulation, that says you must repeat any particular test. You choose to do things. (You don’t have to automate. You … Read more
This post picks up on a small LinkedIn essay from a few months back. There’s a fair amount of preamble here before I talk about regression testing as such. Be careful; you might have heard about testing and checking from people who don’t talk about it the ways we do in Rapid Software Testing (RST). If you’re familiar with RST, maybe you’re fine jumping here. If you’re not so familiar … Read more
This is a lightly-edited excerpt from a longer blog post that you’ll find here. I recommend you read it too, but if you’re short on time, here’s the core of it. If you care about understanding the status of your product, you’ll probably care about testing it. You’ll want testing to find out if the product you’ve got is the product you want. If you care about that, you need … Read more
In any testing situation, when you’re using a tool, you must understand its working principles. You must know what it can and cannot do. You must know how to configure it, and how to calibrate it, how to observe it in action, and how to adjust or repair it when it’s not working properly. To do THAT effectively, you must be able to recognize when your tool is not working. … Read more