I'm not an expert on psychotherapy, but I *am* an expert on privacy and corporate misconduct, and holy *shit* is the idea of a chatbot psychotherapist running on some Big Tech cloud a *terrible* idea. Because while I'm no expert on therapy, I have benefited from therapy, and I know this for certain: therapy *requires* confidentiality.

2/

Shrinks are *incredibly* careful about privacy. For example: when my brother was getting married, my therapist was invited to the wedding. His daughter and my brother's fiancee were close friends, and my brother's fiancee had grown up staying over at their house and wanted her friend and her friend's parents at the wedding.

3/

My therapist sat me down and said, "Now listen, I take confidentiality very seriously. If you want me to, I will pretend not to know you at the wedding. No one needs to know that you're seeing me or - any therapist."

I told him I didn't mind people knowing I'd seen him, but just that little fastidious gesture confirmed the trust I'd put in Alan. It meant that I could openly and freely discuss things I'd never told anyone before, and that I never told anyone ever again.

4/

Having those truly open conversations transformed my life for the better.

Now consider the chatbot therapist: what are its privacy safeguards? Well, the companies make some promises about what they will and won't do with the transcripts of your AI sessions, but they are lying. Of course they're lying! AI companies lie about what their technology can do (of course). They lie about what their technologies *will* do. They lie about money. But most of all, *they lie about data*.

5/

There is no subject on which AI companies have been more consistently, flagrantly, grotesquely dishonest than training data. When it comes to getting more data, AI companies will lie, cheat and steal in ways that would seem hacky if you wrote them into fiction, like they were pulp-novel dope fiends:

arstechnica.com/ai/2025/03/dev…

6/

Open Source devs say AI crawlers dominate traffic, forcing blocks on entire countries

AI bots hungry for data are taking down FOSS sites by accident, but humans are fighting back.

^{Benj Edwards (Ars Technica)}

When an AI company tells you it won't use your intimate secrets as training data, they are lying. Of course they're lying! This isn't just *any* data, it's data that isn't replicated elsewhere on the internet. It's rare - it's *unique*. It's a competitive advantage. AI companies will 100%, without exception, totally use your private therapy data as training data.

7/

What's more: they will *leak* your therapy sessions. They will leak them because they can't figure out how to prevent models from vomiting up their training data verbatim:

theatlantic.com/technology/arc…

But they'll also leak because tech companies leak like hell. They are *crawling* with insider threats. If the AI company sticks around long enough, it'll leak your secrets.

8/

The Flaw That Could Ruin Generative AI

A technical problem known as “memorization” is at the heart of recent lawsuits that pose a significant threat to generative-AI companies.

^{Alex Reisner (The Atlantic)}

And if it goes bankrupt? That's even worse! When tech companies go bust, the first thing their creditors do is sell off their warehouses full of private data. The more private and compromising that data is, the harder they'll try to sell it:

eff.org/deeplinks/2025/03/how-…

Now, maybe you're thinking, "OK, but that's a small price to pay if we can finally get therapy for *everyone*."

9/

How to Delete Your 23andMe Data

This week, the genetic testing company 23andMe filed for bankruptcy, which means the genetic data the company collected on millions of users is now up for sale.

^{Electronic Frontier Foundation}

After all, the country - the world - is in the midst of a terrible mental health crisis and there's a dire shortage of therapists.

Now, let's stipulate for the moment to the idea that chatbots are substitutes for human therapists - that, at the very least, they're better than nothing. I don't think that's true, but let's say it is. Even so, this is a bad tradeoff.

10/

Here, try this thought-experiment: someone figures out a great business-model for to pay for therapy for poor people. "We turned therapy into a livestreamed reality TV show. If you're too poor to afford a therapist, you can go to one of our partially trained livestreamer therapists, who will broadcast all of your secrets to anyone who watches.

11/

"There's a permanent archive of these sessions, and the worst people in the world comb through it 24/7 looking for embarrassing stuff to repost and go viral with. What, you don't like that? Oh, I see: you just don't think poor people deserve mental health. I guess the perfect really *is* the enemy of the good."

12/

This gambit is called "predatory inclusion." Think of Spike Lee shilling cryptocurrency scams as a way to "build Black wealth" or Mary Kay promising to "empower women" by embroiling them in a bank-account-draining, multi-level marketing cult.

13/

Having your personal, intimate secrets sold, leaked, published or otherwise exploited is *worse* for your mental health than not getting therapy in the first place, in the same way that having your money stolen by a Bitcoin grifter or Mary Kay is worse than not being able to access investment opportunities in the first place.

But it's not just people struggling with their mental health who shouldn't be sharing sensitive data with chatbots - it's *everyone*.

14/

All those business applications that AI companies are pushing, the kind where you entrust an AI with your firm's most commercially sensitive data? Are you *crazy*? These companies will not only leak that data, they'll sell it to your competition. Hell, Microsoft *already* does this with Office365 analytics:

pluralistic.net/2021/02/24/gwb…

15/

These companies lie *all the time* about everything, but the thing they lie most about is how they handle sensitive data. It's wild that anyone has to be reminded of this. Letting AI companies handle your sensitive data is like turning arsonists loose in your library with a can of gasoline, a book of matches, and a pinky-promise that *this time*, they won't set anything on fire.

16/