DevelopsenseLogo

How Models Change

Like software products, models change as we test them, gain experience with them, find bugs in them, realize that features are missing. We see opportunities for improving them, and revise them.

A product coverage outline, in Rapid Testing parlance, is an artifact (a map, or list, or table…) that identifies the dimensions or elements of a product. It’s a kind of inventory of aspects of the product that could be tested. Many years ago, my colleague and co-author James Bach wrote an article on product elements, identifying Structure, Function, Data, Platform, and Operations (SFDPO; think “San Francisco DePOt”, he suggested) as a set of heuristic guidewords for creating or structuring or reviewing the highest levels of a coverage outline.

A few years later, I was working as a tester. While I was on that assignment, I missed a few test ideas and almost missed a few bugs that I might have noticed earlier had I thought of “Time” as another guideword for modeling the product. After some discussion, I persuaded James that Time was a worthy addition to the Product Elements list. I wrote my own article on that, Time for New Test Ideas).

Over the years, it seemed that people were excited by the idea of using SFDPOT as the starting point for a general coverage outline. Many people reported getting a lot of value out of it, so in my classes, I’ve placed more and more emphasis on using and practicing the application of that part of the Heuristic Test Strategy Model. One of the exercises involves creating a mind map for a real software product. I typically offer that one way to get started on creating a coverage outline is to walk through the user interface and enumerate each element of the UI in the mind map.

(Sometimes people ask, “Why bother? Don’t the specifications or the documentation or the Help file provide maps of the UI? What’s the point of making another one?” One answer is that the journey, rather than the map, is the point. We learn one set of things by reading about a product; we learn different things—and we typically learn more deeply—by touring the product, interacting with it, gaining experience with it, and organizing descriptions of what we’ve found. Moreover, at each moment, we may notice, infer, or wonder about things that the documentation doesn’t address. When we recognize something new, we can add it to our coverage model, our risk list, or our test ideas—plus we might recognize and note some bugs or issues along the way. Another answer is that we should treat anything that any documentation says about a product as a rumour until we’ve engaged with the product.)

One issue kept coming up in class: on the product coverage outline, where should the map of the user interface go? Under Functions (what the product does)? Or Operations (how people use the product)? Or Structure (the bits and pieces of the product)? My answer was that it doesn’t matter much where you put things on your coverage outline, as long as it fits for you and the people with whom you might be sharing the map. The idea is to identify things that could be tested, and not to miss important stuff.

After one class, I was on the phone with James, and I happened to mention that day’s discussion. “I prefer to put the UI under Structure,” I noted.

“What? That’s crazy talk! The UI goes under Functions!”

“What?” I replied. “That’s crazy talk. The UI isn’t Functions. Sure, it triggers functions. But it doesn’t perform those functions.”

“So what?” asked James. “If it’s how the user gets at functions, it fits under Functions just fine. What makes you think the UI goes under Structure?”

“Well, the UI has a structure. It’s… structural.”

“Everything has a structure,” said James. “The UI goes under Functions.”

And so we argued on. Then one of us—and I honestly don’t remember who—suggested that maybe the UI was important enough to be its own top-level product element. I do remember James pointing out that if when we think of interfaces, plural, there might be several of them—not just the graphical user interface, but maybe a command-line interface. An application programming interface.

“Hmmm…,” I said. This reminded me of the four-user model mentioned in How to Break Software (human user, API user, operating system user, file system user). “Interfaces,” I said. “Operating system interface, file system interface, network interface, printer interface, debugging interface, other devices…”

“Right,” said James. “Plus there are those other interface-y things—importing and exporting stuff, for instance.”

“Aren’t those covered under ‘Functions’?”

“Sure. Or they might be, depending on how you think about it. But the point of this kind of model isn’t to be a template, or a form you fill out. It’s to help us reduce the chances that we might miss something important. Our models are leaky abstractions; overlaps are okay,” said James. Which, of course, was exactly the same argument I had used on him several years earlier when we had added Time to the model. Then he paused. “Ah! But we don’t want to break the mnemonic, do we? San Francisco DePOT.”

“We can deal with that. Just misspell ‘depot’ San Francisco DIPOT. SFDIPOT.”

And so we updated the model.

I wonder what it will look like five years from now.

Subscribe to the Newsletter

17 replies to “How Models Change”

  1. Michael, I couldn’t agree more – models need to adapt as we learn. I do like your adaptation SFDePOT to calling out the interfaces – I often start with a high level context diagram to understand what/who the application / system talks (or will talk) to, and often that reduces a lot of misunderstandings before coding even starts.

    Reply

  4. Today i feel that the testing model has got changed from a Developer based testing to a Tester based testing. The Tester provides all the details to be developed and test the application or product using all types of testing required.

    Michael replies: I don’t buy this model. In the moment that someone provides “all the details to be developed”, that person may be a product owner or a designer or a development manager, but not a tester. A testers neither designs the product and nor runs the project.

    Reply

  5. This is like that model toy the kids get in their Happy meal. My child begs to let me put the stickers on, and when they do I notice they don’t match the picture on the Happy meal box. Yet the kid is still happy because he ‘used’ all of the stickers on it.

    This raises the question of, does exact order or location of things always matter? Sometimes it probably does, but always? Maybe not.

    Michael replies: The exact order or local things might matter. But as far as I know, there’s no reasonable requirement and no test for “always”.

    Reply

  6. For me, listing the different types of interfaces under each of S,F and O forces me to think about the different interfaces and the different perspectives surrounding them.

    Michael replies: To me, this kind of model is a fractal.

    Having 1 single “Interfaces” node on my map feels like it might limit my thinking subconsciously somehow… but I guess the emphasis is on the word being plural.

    Yes, it is. But in our view of the world, you get to use whatever model you like, in whatever way you like, if your model helps you get to better thinking and better testing. So, rock on.

    Anyway, thanks for another great blog Michael! It has inspired me to revisit my own models I use and investigate how I might be able to enhance them. 🙂

    Reply

  8. I would offer “Environment” as a suitable “E” for your DEPOT. I have found that testing an application (especially performance testing a web application) will benefit when considering the environment that the application is loaded in. Network latency, cache engines, firewall rules, and other applications competing for resources all fall into a realm of consideration which would sometimes be overlooked if the environment of the production application is not considered and compared against the test environment. If there are errors there that were not caught in testing, the question will still be asked, “Why were these not found before we released?”

    Michael replies: In the Rapid Testing namespace, we would think of Environment as part of Platform; anything upon which the product depends (and, from another perspective, anything that depends on it). Nonetheless, Environment is a terrific idea, and I agree with your reasons for proposing it. I’ll bring it up the next time that James and I revisit our models—something that we do constantly. We’ll may reject it because we need the space in the mnemonic for Interfaces, and because Platform already covers what we’ve got in mind. For us.

    And yet. You’re under no obligation to adopt our model, so make Environment part of yours. We believe that every tester should invent testing for himself or herself, so if you want to make Environment an element in your model, make it so (IF POSTED!) Change other words, too. Commit it to memory and use it effortlessly. Recite it. Publicize it. Talk about it proudly. Models for testing shouldn’t be locked down, winners of a competition or a one-size-fits-all deal; they should be like bins in candy store with all kinds of delicious treats available.

    Thank you for the thoughtful offering.

    Reply
  17. Pingback: Memorizing Everything: How Mnemonics Aid in Testing - Trailhead Technology Partners

Leave a Comment