It’s Not About the Artifact

There’s a significant mistake that people might make when using LLMs to summarize a requirements document, or to produce a test report.

LLMs aren’t all that great at summarizing. That’s definintely a problem, and it would be a mistake to trust an LLM’s summary without reviewing the original document. The bigger mistake is in believing that the output, the artifact, is the important thing.

We might choose to share a summary, in the form of an artifact, with other people. The artifact might be useful and informative, but the artifact itself is not really the point. To paraphrase Eisenhower’s famous saying about planning, the summary is nothing; the summarizing is everything.

We don’t summarize a requirements document simply to get a shorter version of it. As an absurd example, we could summarize easily by going through the document and removing each run of nine words and leaving the tenth; or we could take out all but the first and last page. If brevity were the only purpose of a summary, those things would do fine. We could make the document perfectly brief by deleting it entirely; infinite brevity!

The most important aspect of summarizing is not to produce a shorter document, but to change and sharpen our minds and our thinking. In an article in the Atlantic, Douglas Hofstadter — computer scientist, pioneering AI researcher, and author of Godel, Escher, Bach puts it beautifully:

I frankly am baffled by the allure, for so many unquestionably insightful people (including many friends of mine), of letting opaque computational systems perform intellectual tasks for them. Of course it makes sense to let a computer do obviously mechanical tasks, such as computations, but when it comes to using language in a sensitive manner and talking about real-life situations where the distinction between truth and falsity and between genuineness and fakeness is absolutely crucial, to me it makes no sense whatsoever to let the artificial voice of a chatbot, chatting randomly away at dazzling speed, replace the far slower but authentic and reflective voice of a thinking, living human being.

Douglas Hofstadter, “Gödel, Escher, Bach, and AI”, The Atlantic, July 8, 2023

We go through the process of summarizing a document (or a set of them) to wrap our minds around what the document says, what it means, and what it implies. A good summary doesn’t just identify what might be there; it also identifies what’s missing, what’s unclear, what’s misleading, and what’s wrong. To arrive at all that, we must not only shorten and transform the text. We must bring what we already know to the table; then we must read, reflect, analyze; and then we must be to articulate the issues in our minds, speech, or text. That is, we must learn and transform ourselves.

Similarly, we don’t produce a test report simply to produce an artifact. We produce a test report to describe the status of the product; to describe what we tested and how we recognized problems; and to describe the quality of the testing work. We produce a test report to warn, to describe, to complain productively, or to ask for help.

Producing a good test report requires us to describe our knowledge, our awareness, our experience, and our problems. More importantly, it requires us to develop all of those by testing the product and finding problems in it. A worthwhile, truthful report on good testing work requires us to do the testing work. None of that personal development goes into a report generated by an LLM.

When we lean on an LLM to do it, we throw away the opportunity to learn, to reflect, and to develop our skills. This seems very risky to me. And it should seem even riskier to a manager who needs to know about the status of the product and potential risks to the business — and who needs skilled people to provide it.

A shorter version of this post originally appeared on LinkedIn. I developed deeper thoughts on the subject by producing this post instead of letting an LLM do it for me.