“We’re happy to acknowledge when we make errors, and we’ve made a few in this case for sure, but also that we are willing to address those. And we’ve taken immediate action in all of these cases,” he said. “And we certainly apologize for any effect or adverse effect that this has had on trust in the community, because we want people to be confident in what we’re doing.”
David Zibolski, (retired) Chief of Police, Fargo, North Dakota, USA, quoted in MPR News
Here some big problems with Our Modern World: deniability, malfeasance, euphemism, and the lack of consequences.
Here’s the story: Angela Lipps, a fifty-year-old grandmother from Tennessee, was flagged as a suspect in a bank fraud investigation in Fargo, North Dakota. A police officer identified here using facial recognition software from a company called Clearview AI.
Clearview AI had previously scraped images from sites including Instagram, Facebook, and LinkedIn to develop its software and its database — 40 billion of them, according to the New York Times. The software was used, and still is, by the FBI, the Department of Homeland Security, and police departments all over the country.
After the Times revealed the existence of the company, people were understandably upset about it. Class action lawsuits were launched, and eventually settled. One settlement in 2022 prevented the company from selling its data to individuals or businesses. Another, settled in 2024, gave victims a 23% stake in the company because Clearview (poor Clearview!) would have gone bankrupt before the case could be brought to trial. It’s not clear to me how many members were in the class, but 23% of the company’s value at the time amounted to $52 million — on the order of 15 cents for every member of the US population.
Back to Mrs. Lipps. After being identified by the police officer using Clearview and no other evidence, she was arrested in July 2025, held in jail in Tennessee until October, and then taken to North Dakota.
On Christmas Eve, after five months in prison, she was released. To get home, she depended on money donated from defense lawyers for food and accommodation, and a volunteer driver to take her halfway home, whereupon her family picked her up. Apparently her extended holiday in North Dakota prevented her from being home for Christmas.
According to the Times, Clearview AI claims that its technology is “designed to function as one tool within a broad investigative process. It generates leads; it does not make identifications, draw conclusions, or recommend arrests… When using Clearview AI’s platform, independent corroboration by trained law enforcement professionals is required.” Statements like this give Clearview deniability.
Look at the claims Clearview AI makes on its Web site. It says
We deliver facial recognition solutions that are
PRECISE
Leading facial recognition technology, excelling even in challenging photographic conditions, tested by NIST.
PROVEN
Trained on the largest and most diverse dataset and relied on by law enforcement in high-stakes scenarios.
Next to “NIST”, there’s a teensy-weensy little superscript link. When I click on it, it says
DISCLAIMER
[1] This refers to performance in the categories of Demographic Effects on Visa-Border and Mugshot photos in the NIST Facial Recognition Vendor Test in the 1:1 setting, as well as performance in the Mugshot-Mugshot, Mugshot-Webcam, Visa-Border, Border-Border (≥ 10 YRS), Mugshot-Mugshot (≥ 10 YRS) categories of the 1:N Investigative setting.
Documentation from the NIST is pretty clear about the patterns and risks of false positive photo matching if you’re familiar with statistics, and if you take the time to study the analysis. The message one should take from it is this: the technology is powerful and most not be trusted as conclusive; and statistical patterns can be misleading. Is that part of the training for police officers using it?
When you’re testing something that matters, it’s nice to gather evidence that the software can work; but that’s the easy bit, really, more of a demonstration than a test. “Can work” does not mean does work, and “does work” does not mean will work.
We know from experience that we cannot assume normal software systems — intentionally, deliberately, and consciously designed and written by human beings, to be okay. “AI” — software that contains machine learning algorithms that have not been deliberately and consciously written by human beings — requires a far greater degree of mistrust. When there’s risk on the line, the big deal is looking for evidence of problems. That is, the goal of excellent testing is to falsify the idea that the software is trouble-free.
But we must also assume that even when the system is working well, there is a risk of reasonably foreseeable misuse. For instance, imagine an algorithm that has been designed and tested to have a one-in-a-million chance of misidentifying someone. That sounds pretty reliable, right? But over the population of the United States, that amounts to about 340 chances to get the identification wrong.
People in positions of responsibility would benefit from training in testing and critical thinking. When something matters (like bank fraud), and you’re targeting a criminal, it’s important to gather evidence to support the idea that you’ve got the right person. When something else matters (like a person’s freedom from unlawful imprisonment) and you’re using AI, it’s important to systematically mistrust the software, and to to seek evidence showing that you’re targeting the wrong person — that is, to falsify the wrong idea that you’ve targeted the right person.
And in the case of Ms. Lipps, that evidence was easily available to anyone who bothered to looked for it. Before her flight from Tennessee to the jail in Fargo, she had never been on an airplan, and she had never been to North Dakota. Ms. Lipps’ family and friends had records to show that, while the bank fraud was taking place in Fargo, she had been performing transactions at home in Tennessee. All this would have been easy to determine if anyone had investigated seriously — or systematically mistrusted the software.
Now read Mr. Zibolski’s euphemistic statement above. He apologizes for any effect or adverse effect that this has had on trust in the community, rather than for any adverse affect that it had on an innocent person who was locked up for almost half a year.
All this is reminiscent of the Great Post Office Scandal in the UK. In both cases, software was assumed to provide sufficiently reliable evidence without any other evidence to deprive innocent people of their liberty and their time.
Taxpayers in Britain will end up paying something north of two billon pounds in compensation to the victims. The citizens of Fargo will likely have to pay a significant settlement to Ms. Lipps. Yet neither the people who misused the software nor the people who produce it suffer. They certainly won’t get sent off for an extended holiday in a prison. Mr. Zibolski retired, presumably at full pension. Meanwhile, there hasn’t even been an apology to Ms. Lipps; neither from the Fargo police department, nor from the former Chief of Police, nor from the unnamed police officers for whom he was responsible.
There must be consequences for this degree of irresponsibilty, incompetence, and callousness.
Further reading: Mistakes Were Made (But Not By Me), Tavris and Aronson