Selenium! Playwright! Cypress! Postman! There’s constant hubbub around these tools and others primarily focused on checking output—and now the din is getting louder about low-code or no-code tools for checking output. (Last year I reviewed a couple of those, here and here. Did these companies use their own tools to test their own tools? Either a Yes or No answer would be disturbing.) Want some heuristics for choosing tools of … Read more
I sit down with a coffee, retrieve my mail, and open it up. I’ve received a spammy kind of message. It provides a link to unsubscribe. The link returns {“code”:1020,”message”:”fail”}. I’ve received another spammy kind of message, this time from Yellow Pages (yp.ca). In the text, it is addressed to “[{Recipient.FirstName}]”. The message asks “How do you show up online?” Better than YellowPages shows up in my inbox, I hope. … Read more
Deep testing — testing that maximizes the change of finding every elusive bug that matters — is the kind of testing work that engages and motivates serious testers. Developers tend to prefer shallow testing that doesn’t disrupt the developer’s mindset and the developer’s focus on getting the danged product built. (Before anyone gets too upset, note that the shallowness of shallow testing is a feature, not a bug. There’s lots more … Read more
A few years ago, a friend introduced me to a recording of the mind-blowing musical Hamilton (if you haven’t seen or heard it, do yourself a favour and check it out). Our whole family was, for a time, obsessed with it. A few months after that, I was preparing for a talk at Xebia in Amsterdam. I was wondering how I could spice up the talk a little by doing … Read more
A while back I wrote a post on coverage. In fact, I’ve written a few posts about coverage, but I’m talking about this one. A question came up recently on LinkedIn that helped me to realize I had left out something important. In that post, referring to coverage as “the proportion of the product that has been tested”, I said A software product is not a static, tangible thing; it’s a … Read more
A tester asks… How do we handle production bugs? When management asks “Did you test this?”. how do I respond? When management asks “Why didn’t you find that bug?”, the first step is to accept in your own mind responsibility for looking for bugs, but not a commitment to finding every bug. The latter is a great aspiration, but an unreasonable commitment, and management shouldn’t be holding you to it. … Read more
Frequently, both online and in face-to-fact conversations, testers express reservations to me about making a clear distinction between testing and checking when talking to others. It’s true: “test” is an overloaded word. In some contexts, it refers to a heuristic process: evaluating a product by learning about it through experiencing, exploring and experimenting; that’s what testers refer to when they’re talking about testing, and that’s how we describe it in … Read more
In several of our Rapid Software Testing classes, for the last four years at least, these three slides have part of our materials on risk: And then what do you know?! This happened. (“Taylor Swift Crashes Ticketmaster as Fans Scoop Up Presale Tickets”; here’s the link.) And this happened too… (“Justice Dept. is Said to Investigate Ticketmaster’s Parent Company”) Here’s the link to that. Don’t say we didn’t warn you.
“The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming.” -Donald Knuth, The Art of Computer Programming Most of the time testing shouldn’t be too formal — unless you want to miss lots of bugs. To test something formally means … Read more
After almost a year of the blog lying fallow, it’s time to continue the series on Testing Deep and Shallow that begins here. (Shallow testing (not an insult!) is testing that has a chance of finding every easy bug. Deep testing maximizes the chance of finding every elusive bug that matters.) Premise 6 of Rapid Software Testing is about cost and value. “We commit to performing credible, cost-effective testing, and … Read more