DevelopsenseLogo

Artificial Intelligence

Bug of the Day: Facebook’s AI Layer Mangles Two Posts

Today I visited Facebook to post a notice of my upcoming trip to New Zealand. There will be three stops on the tour: Auckland (for Testers and Automation, Avoiding the Traps, February 17-19), Wellington (Testers and Automation, Avoiding the Traps, February 24-26), and Christchurch (Rapid Software Testing Explored, March 10-12). Facebook’s AI Layer (I’ll just call that FAIL) offered to turn it into an event. I accepted the offer, and … Read more

What Are We Thinking in the Age of AI?

At the Pacific Northwest Software Quality Conference in October 2024, I gave a keynote presentation titled “What Are We Thinking in the Age of AI?“ There’s a lot to think about, and for testers, there’s a lot to do. For one, we need to understand the basis for the “AI” claim. Any kind of software can be marketed as “AI”, since it’s doing something that (presumably) a human could do, … Read more

Testing AI : five obstacles and seven workarounds

‍Recorded for the 3rd Zebrunner Expert Series Webinar we discuss the noise and help you understand what AI truly is and what it means for testing practices. There’s an incredible amount of noise around artificial intelligence these days, but very little reliable signal. AI will bring doom and destruction, or a world where cheerful robots feed us peeled grapes while we lie on the couch. Some say AI is already … Read more

Test Tools Need Testing

In any testing situation, when you’re using a tool, you must understand its working principles. You must know what it can and cannot do. You must know how to configure it, and how to calibrate it, how to observe it in action, and how to adjust or repair it when it’s not working properly. To do THAT effectively, you must be able to recognize when your tool is not working. … Read more

Language Models

“Language models” is typically interpreted as a compound noun, something that models language. A model is an idea in your mind, an activity, or an object (such as a diagram, a list of words, a spreadsheet, a role play, a person, a toy, an equation, a demonstration, or a program…) that represents (literally! re-presents!) another idea, activity, or object (such as something complex that you need to work with or … Read more

The First Hurdle Heuristic

There is a testing techique that I often apply. I have recently decided to name it the First Hurdle Heuristic. The basic idea: get the product out of the starter’s blocks, and see how it performs given a relatively easy challenge. This heuristic can useful when you want to identify problems and risks immediately, or to determine whether a product might not be ready for use or for deeper testing. … Read more

Yes, We Still Need To Look. Carefully.

I very occasionally visit Xitter (pronounciation tip: it goes like the name of the President of the People’s Republic of China). The other day, Jason Huggins said Just in case you’re using a screen reader, that’s “I occasionally use the Tesseract OCR library for text recognition. I think that means I’m a senior machine learning engineer now, I guess.” I felt a little impish, but I also felt quite lazy. … Read more

Testing ChatGPT’s Programming “Skills”

With the current mania for AI-based systems, we’re finally starting to hear murmurs of moderation and the potential for risk. How do we test systems that incorporate an LLM? You already know how something about how to test LLM systems if you know how to test. Testing starts with doubt, and with a desire to look at things critically. The other day on LinkedIn, Paramjit Singh Aujla presented a problem … Read more

It’s Not About the Artifact

There’s a significant mistake that people might make when using LLMs to summarize a requirements document, or to produce a test report. LLMs aren’t all that great at summarizing. That’s definintely a problem, and it would be a mistake to trust an LLM’s summary without reviewing the original document. The bigger mistake is in believing that the output, the artifact, is the important thing. We might choose to share a … Read more

Testing, Now More Than Ever

To all managers and executives: despite how it’s in fashion these days, it’s not a good time to be laying off testers, or to be leaving them unprepared and untrained. Software can be wonderful. It can help us with all kinds of stuff, unimaginably quickly and at enormous scale. This sounds very appealing. Skilled testers, at least, have always known that we must treat output from machinery with appropriate skepticism … Read more