Flutterby™! : LLMs and human connection\

Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers

Contrary to best practices in the medical community, LLMs 1) express stigma toward those with mental health conditions and 2) respond inappropriately to certain common (and critical) conditions in naturalistic therapy settings -- e.g., LLMs encourage clients' delusional thinking, likely due to their sycophancy. This occurs even with larger and newer LLMs, indicating that current safety practices may not address these gaps.

Via Jeremy Kahn @trochee@dair-community.social, who also notes:

It's frustrating to see a paper that arrives at a conclusion I consider foregone — that LLMs are no substitute for a human connection — while using very shady methods

They *don't even cite Weizenbaum* and ELIZA was exactly this