Testing Needs Variation

This happened to me again today in Quicken. It’s happened before. Worse, it’s an example of an extremely common phenomenon.

My task here is to send an invoice to a particular person at a particular company. I fill in a part of a form — an address field — that appears right under the word “Invoice”. It’s not the topmost input element on the dialog, but it’s definitely the most naturalistic place to go, relative to the task at hand.

Upon completing the form, I opt to save it. The function stops and complains that I haven’t made an entry in the field in the top left of the screen: a combo box (that’s the kind of input element where you can type text input or choose an item from a drop-down) that identifies the addressee.

If I choose the addressee from a drop-down, the full address is filled in automatically, which is nice. If the addressee is not already in the list of existing addressees, the name that I type in the combo box becomes the first line of the address. That’s nice too. So far so good.

However, if the address field is already completed and I type something into the combo box, the text I typed becomes the first line of the address field, and all the rest of the data in address field is wiped out, so I have to start over.

This is a violation of The First Commandment of Reliability and Usability: Thou shalt not ever, ever, EVER lose the customer’s data.

It’s 2024, people; this lesson should have been learned ages ago, yet the problem persists all over the place.

Testers, we can’t stop designers and developers from making ridiculous assumptions about users faithfully following their models of how people will work, but we can sure test for these problems. Trouble is, when we simply follow instructions and do what we’re told — or when we program an automated process to do it — we miss bugs like this.

So, a heuristic: whenever you’re testing something that happens in a workflow, challenge the assumption that people will perform it in the exact order that the designer had in mind. Vary the order of operations and actions, and report anything that might cause someone to lose data, effort, time… or their happiness and good will.

And, dear Quicken, please fix this crap.