Challenges and Legibility

Lately, James Bach and I have been issuing challenges to some of our colleagues on Twitter, typically based on something they’ve said or observed. I think James would agree that the results have been very exciting. In our community, people build credibility by responding to challenges and probing the issues more deeply, and it’s been tremendous to see how some of them have risen to the challenge. For me, recent examples include Joe Harter and his response to the question “Why keep testing when we’ve got a swarm of bugs?”; and David O’Dowd and his recent tweets on how to address disagreement over the “right” temperature for a cup of coffee. It goes both ways, of course: we expect other people to challenge us, too. That’s how we test ideas.

Recently, James turned me on to an interesting Web site, authored by a fellow named Venkatesh Rao, and in particular to this blog post. I was very excited by the concept of legibility, making things more readable in a metaphoric sense, more understandable. To me, legibility is a powerful idea because it seems to explain a central conundrum in testing and in the management of software development: a good deal of the effort that we spend, so it seems, is not in producing better stuff, but rather in attempting to make complex stuff more understandable. One approach to understanding complexity is to take the general systems view, and model the system of interest in terms of other, simpler systems, and look at the aspects of elements, relationships, control, feedback, and effects, and the relationships between all of these. Another approach is to close your eyes to the complexity (as French governments and tax collecters tried to do in the 1800s) and pay attention only to a couple of specific elements in the model. Yet another approach, often used by large organizations and bureaucracies such as nation states, is a wholesale attempt to make the system more legible by eliminating the complexity by eliminating elements (as Prussian forest managers did in the 19th century, or as the builders of Brasilia did in the 20th).

I ordered the book Seeing Like a State to which Venkatesh refers, and I’m finding it interesting. More on that later, perhaps.

Before I ordered the book, though, I thought the idea of legibility would be of interest to a general systems thinker, so I sent a link along to Jerry Weinberg. He surprised me a little by replying,

“Well, yes, but it’s a far over-simplified vision itself. For instance, it doesn’t seem to account for why the “recipe” actually succeeds (value to some persons or groups). Think it through.”

Here’s my reply:

[quote]

Thank you for the challenge. Let me see if I can answer it.

I think it does account for why the “recipe” actually succeeds, although it may gloss over the point somewhat.

• Success is subject to the Relative Rule. (As I described in my chapter of The Gift of Time, the Relative Rule states that “for any abstract X, X is X to some person”.) That is, success is success to some person(s).
• Success is measured by some persons at some time (a refinement of the Relative Rule that I identified and that Markus Gartner seized on). Any determination of success (at some time and for some purpose) is like observing the part of the curve that looks linear. We cant’t save we’ve achieved the end result because a) not all the data is in yet, and b) as I’ve heard you say on a number of occasions, “nothing is ever settled”. (I think I’d like to call this The Unsettling Rule.)
• Similarly, “complexity”, “reality”, “irrationality”, “orderliness”, “legibility”, etc. are all subject to the Relative Rule and the Unsettling Rule too. When Venkatesh says, “The big mistake in this pattern of failure is projecting your subjective lack of comprehension onto the object you are looking at, as ‘irrationality'”, that reminds me of your (Jerry’s) advice in the SHAPE Forum many years back: stop looking at it as “irrational”, and start looking at it as “rational from the perspective of a different set of values”.
• Says Venkatesh, “This failure mode is ideology-neutral, since it arises from a flawed pattern of reasoning rather than values.” Well, that’s all very well, but you can’t have the concept of “a flawed pattern of reasoning” without imposing a value judgement.
• By making something more legible, you might have a short-term effect that you consider negative, but which gives rise to a more “positive” long-term effect. For example, in the old days, anyone could cut down trees pretty much anywhere they liked. These days we seem to have a stricter sense of preserving some kinds of land so as not to be interfered with by the forestry business, and using other kinds of land for what is effectively tree farming. “Legibility” is always in flux.
• “Rational and unlivable grid-cities like Brasilia, versus chaotic and alive cities like Sao Paolo.” Yeah, but I’ve heard about problems in Sao Paolo, and I’m not convinced that Brasilia is less livable than Sao Paolo, based on those problems.

I could go on… but have I shown you some evidence of thinking it through?

[/quote]

Jerry’s response was,

“Well done.

You’ve got another blog post there, I think.”

So here is that blog post.

In his challenge to me, Jerry was encouraging me (and, by extension, Venkatesh) to think about things in a more complex and nuanced way. For me, the key lesson is to remember that whatever you see as “broken” is almost certainly working for someone. That person, being different from you, is to some degree looking at everything from the perspective of a different set of values. When you see a problem in a product, or organization, or system, addressing that problem is going to take some effort for someone, and that person might see neither the problem, nor its the cost, nor the value of change as you see it. That person might have political authority over the situation, and like all people, that person is driven not only by rationality but also by emotion. That person might not even see you.

For example, as a tester, when you say that a product has “too many bugs”, it’s important to ask, “Too many compared to what?” “Too many for whom?” “Too many according to whom?” “Too many to meet what goal?” That’s one of the reasons that test framing is so important: your testing won’t be valued if it’s not congruent with the mission, whether implicit or explicit, that your client has in mind.

Now, having to deal with all this uncertainty and subjectivity might require us to give up an idealist Platonic sense of Goodness and Order and Godliness, and might force us to deal with messy, complex, and human concerns. But considering that we all have to live with each other, and that “ideal” is only ideal to some person, at some time, that might be a good thing.

Thank you to Jerry for his persistent, patient reminders.