DevelopsenseLogo

Lessons Learned in Finding Bugs

This story is put together from several parallel experiences over the last while that I’ve merged into one narrative. The pattern of experiences and epiphanies is the same, but as they used to say on TV, names and details have been changed to protect the innocent. I was recently involved in testing an online shopping application. In the app, there’s a feature that sends notifications via email. On the administration … Read more

Experience Report: Katalon Studio

Warning: this is another long post. But hey, it’s worth it, right? Introduction This is an experience report of attempting to perform a session of sympathetic survey and sanity testing on a “test automation” tool. The work was performed in September 2021, with follow-up work November 3-4, 2021. Last time, the application under test was mabl. This time, the application is Katalon Studio. My self-assigned charter was to explore and … Read more

Experience Report: mabl Trainer and runner, and related features

Screen shot from Mattermost while testing mabl

Warning: this is a long post. Introduction This is an experience report of attempting to perform a session of sympathetic survey and sanity testing, done in September 2021, with follow-up work October 13-15, 2021. The product being tested is mabl. My self-assigned charter was to perform survey testing of mabl, based on a performing a basic task with the product. The task was to automate a simple set of steps, … Read more

Bug of the Day: AI Sees Bits, Not Things

An article that I was reading this morning was accompanied by a stock photo with an intriguing building in the background. I wanted to know where the building was, and what it was. I thought that maybe Chrome’s “Search Google for image” feature could help to locate an instance of the photo where the building was identified. That didn’t happen, but I got something else instead. Google Images provided me … Read more

Lessons Learned from a Little Bug

Almost 10 years ago, I wrote a series of blog posts on project estimation and black swans. And, almost 10 years after that, Chris NeJame reported an observation about the following passage towards the end of Part 4 of the series: As Jerry (Weinberg) has frequently pointed out, plenty of organizations fall victim to back luck, but much of the time, it’s not the bad luck that does them in; … Read more

Testers Don’t Prevent Problems

Testers don’t prevent errors, and errors aren’t necessarily waste. Testing, in and of itself, does not prevent bugs. Platform testing that reveals a compatibility bug provides a developer with information. That information prompts him to correct an error in the product, which prevents that already-existing error from reaching and bugging a customer. Stress testing that reveals a bug in a function provides a developer with information. That information helps her … Read more

As Expected

This morning, I started a local backup. Moments later, I started an online backup. I was greeted with this dialog: Looks a little sparse. Unhelpful. But there is that “More details” drop-down to click on. Let’s do that. Ah. Well, that’s more information. But it’s confusing and unhelpful, but I suppose it holds the promise of something more helpful to come. I notice that there’s a URL, but that it’s … Read more

A Bad Couple of Days

I’m home in Toronto for a day after several weeks of helping people learn to test software, and as far as I can see, the whole Web is screwed up. Here are some of the things that have happened in the last 48 hours or so. A fellow on Twitter told me about an interesting Skype bug: send the string “http://:” (no quotes), and Skype hangs. Fpr me, it did … Read more

Taking Severity Seriously

There’s a flaw in the way most organizations classify the severity of a bug. Here’s an example from the Elementool Web site (as of 14 January, 2015); I’m sure you’ve seen something like it: Critical: The bug causes a failure of the complete software system, subsystem or a program within the system. High: The bug does not cause a failure, but causes the system to produce incorrect, incomplete, inconsistent results … Read more

I’ve Had It With Defects

The longer I stay in the testing business and reflect on the matter, the more I believe the concept of “defects” to be unclear and unhelpful. A program may have a coding error that is clearly inconsistent with the program’s specification, whereupon I might claim that I’ve found a defect. The other day, an automatic product update failed in the middle of the process, rendering the product unusable. Apparently a … Read more