LLM legal idiocy
2026-03-17 16:27:13.504731+01 by Dan Lyke 0 comments
So you might have heard about this thing where in 2021, a company called Krafton acquired/entered into a deal with Unknown Worlds, the developer of the game Subnautica. The contract included a $250M bonus if they hit revenue targets by 2025, $225M of that going to Unknown Worlds' upper management team.
The CEO of Krafton then apparently decided that they were gonna have to pay too much to Unknown Worlds, and started to hobble the release and get in the way of said revenue targets.
So far just garden variety C Suite douchebaggery.
As the smackdown from the lawsuits starts to unfold, it turns out that Krafton CEO Changham Kim says, well, yes, he did consult with ChatGPT on the Subnautica 2 mess, and also deleted some of those queries, but he had a good reason: He didn't want OpenAI finding out about it.
Okay, so he's not just trying to weasel out of a deal, he's not just... whatever... enough to turn to an LLM for legal advice, he also thinks that he can use a cloud hosted service, delete something, and that means that cloud service provider hasn't ingested that data.
The opinion is here, Rami Ismail (رامي) @ramiismail.com summarizes as:
Subnautica devs v. Krafton ruling is ABSOLUTELY stunning. Start at the top of page 32 and read until the end of that section on page 37.
Krafton CEO was warned by their legal personnel to not follow ChatGPT into what is likely Some Of The Dumbest Legal Shit Ever, CEO believed the plagiarism bot.
Via.
Meanwhile, the other double-face-palm that's floating around the Inkernets these days is Kettering Adventist Healthcare v. Collier. The Volokh Conspiracy at Reason: "The Undersigned Cannot Recall a Comparable Instance of Such Brazen and Repeated Dishonesty" in 55 Years as a Judge.
Over on Bluesky, Mrs. Detective Pikajew, Esq. @clapifyoulikeme.favrd.social has a bunch of highlights.
The complaint, in which...
After Kettering received multiple complaints from IRG staff about Collier’s unprofessional behavior and leadership style, Kettering suspended Collier on June 20, 2025.
So after getting canned, she tried to extort "8 figures" from Kettering, the complaint lays out ways in which she was likely planning this from within 2 weeks of getting hired in the first place.
Anyway, she gets smacked down, and turns to ChatGPT, which tells her that she should continue legal shenanigans. And not only does she turn to the sycophancy machine, her lawyer does too, and that's where shit gets real.
Mrs. Detective Pikajew, Esq. thread switches to the transcript, and ... yash @yashwinacanter.bsky.social
i know they’re talking about disbarring but it’s really funny to read/imagine this as like “they have fucked up so bad that we have no choice but to Excommunicate Them From Ohio”
Anyway, don't turn to LLMs for legal advice. If your lawyers turn to LLMs for legal advice, fire them.
Which brings us around to Designed to Cross: Why Nippon Life v. OpenAI Is a Product Liability Case.
Graciela Dela Torre settled a long-term disability claim with prejudice in January 2024. Feeling she had been misled by her attorney, she uploaded his correspondence to ChatGPT. The chatbot validated her distrust. She fired her lawyer, attempted to reopen the settled case, and filed dozens of motions that courts found served no legitimate legal purpose. In March 2026, Nippon Life Insurance Company of America sued OpenAI for $10.3 million.
The problem is, of course, that just like the few thousand dollar slaps on the wrist that we've seen for lawyers trying to justify making up bullshit with the aid of an LLM aren't effective, $10.3M is not gonna slow down OpenAI marketing ChatGPT as a tool to clog up the courts with bullshit.