GPT5 is a terrible storyteller
2025-09-02 18:02:01.048+02 by Dan Lyke 0 comments
CHristoph Heilig: GPT-5 Is a Terrible Storyteller – And That's an AI Safety Problem, coming up with the theory that OpenAI is using LLMs to evaluate outputs in training, and like a high school English student writing the sorts of florid prose that they think their teacher is going to like, there's a feedback loop:
Do you remember the researchers that hid prompt-style instructions (e.g., in white or tiny text) inside arXiv drafts to make LLM-assisted reviewers output only positive evaluations and avoid mentioning negatives? It's almost as if GPT-5 accomplished something similar – to invent a kind of secret language that allows it to communicate with LLMs in a way that they will like GPT-5's stories even when they are utter nonsense.