The fact that CEOs overestimate the size of this market is easy to understand, since "CEO" is the most laptop job of all laptop jobs. Having a chatbot summarize the boss's email is the 2025 equivalent of the 2000s gag about the boss whose secretary printed out the boss's email and put it in his in-tray so he could go over it with a red pen and then dictate his reply.

3/

The smart AI money is long on "decision support," whereby a statistical inference engine suggests to a human being what decision they should make. There's bots that are supposed to diagnose tumors, bots that are supposed to make neutral bail and parole decisions, bots that are supposed to evaluate student essays, resumes and loan applications.

The narrative around these bots is that they are there to help humans.

4/

In this story, the hospital buys a radiology bot that offers a second opinion to the human radiologist. If they disagree, the human radiologist takes another look. In this tale, AI is a way for hospitals to make fewer mistakes by spending more money. An AI assisted radiologist is *less* productive (because they re-run some x-rays to resolve disagreements with the bot) but *more* accurate.

5/

In automation theory jargon, this radiologist is a "centaur" - a human head grafted onto the tireless, ever-vigilant body of a robot

Of course, no one who invests in an AI company expects this to happen. Instead, they want *reverse-centaurs*: a human who acts as an assistant to a robot. The *real* pitch to hospital is, "Fire all but one of your radiologists and then put that poor bastard to work reviewing the judgments our robot makes at machine scale."

6/

No one seriously thinks that the reverse-centaur radiologist will be able to maintain perfect vigilance over long shifts of supervising automated process that rarely go wrong, but when they do, the error *must* be caught:

pluralistic.net/2024/04/01/hum…

The role of this "human in the loop" isn't to prevent errors. That human's is there to be *blamed* for errors:

pluralistic.net/2024/10/30/a-n…

The human is there to be a "moral crumple zone":

estsjournal.org/index.php/ests…

7/

The human is there to be an "accountability sink":

profilebooks.com/work/the-unac…

But they're not there to be radiologists.

This is bad enough when we're talking about radiology, but it's even worse in government contexts, where the bots are deciding who gets Medicare, who gets food stamps, who gets VA benefits, who gets a visa, who gets indicted, who gets bail, and who gets parole.

8/

The Unaccountability Machine - Profile Books

^{Profile Books}

That's because statistical inference is intrinsically *conservative*: an AI predicts the future by looking at its data about the past, and when that prediction is also an automated decision, fed to a Chaplainesque reverse-centaur trying to keep pace with a torrent of machine judgments, the prediction becomes a directive, and thus a self-fulfilling prophecy:

pluralistic.net/2023/03/09/aut…

9/

AIs want the future to be like the past, and AIs make the future like the past. If the training data is full of human bias, then the predictions will also be full of human bias, and then the outcomes will be full of human bias, and when those outcomes are copraphagically fed back into the training data, you get new, highly concentrated human/machine bias:

pluralistic.net/2024/03/14/inh…

10/

By firing skilled human workers and replacing them with spicy autocomplete, Musk is assuming his final form as both the kind of boss who can be conned into replacing you with a defective chatbot *and* as the fast-talking sales rep who cons your boss. Musk is transforming key government functions into high-speed error-generating machines whose human minders are only the payroll to take the fall for the coming tsunami of robot fuckups.

11/

This is the equivalent to filling the American government's walls with asbestos, turning agencies into hazmat zones that we can't touch without causing thousands to sicken and die:

pluralistic.net/2021/08/19/fai…

12/

I'm on a 20+ city book tour for my new novel *Picks and Shovels*.

Catch me in SAN DIEGO on Mar 24:

mystgalaxy.com/32425Doctorow

And in CHICAGO with PETER SAGAL on Apr 2:

exileinbookville.com/events/44…

More tour dates here:

martinhench.com

13/

Image:
Krd (modified)
commons.wikimedia.org/wiki/Fil…

CC BY-SA 3.0
creativecommons.org/licenses/b…

--

Cryteria (modified)
commons.wikimedia.org/wiki/Fil…

CC BY 3.0
creativecommons.org/licenses/b…

eof/

File:DASA 01.jpg - Wikimedia Commons

^{commons.wikimedia.org}