Have LLMs peaked?
2026-04-15 17:46:22.549282+02 by Dan Lyke 0 comments
In response to Peter @peter@thepit.social
ChatGPT was released to the public four years ago and today i can't think of a single software feature or product that uses it that i would miss if it disappeared today.
Mal 甄/kalessin/Peri @perigee@rage.love writes:
@peter @Binder I've been in ML/data science since 2018, formally, but worked with big data in a scientific sense since the mid 90s and one thing that keeps striking me like a thunderclap is how no LLM bro seems to be aware that while there have been refinements in the statistics and efficiencies of architecture, there hasn't been significant improvement in the fundamental outcomes of the statistics since probably 2019?
The lack of progress defies Moore's "law" and no one in the pro LLM space wants to even mention how "progress" has seemingly halted. Or was never happening in the first place.
There's a paper from a year ago (I'll dig the citation out of Computerphile's archives in a bit) that posits that any significant difference from feeding LLMs more content asks for an impossible amount of new ingested (stolen) information if the aim is to train a general LLM. In other words the method has already peaked.
It is just one paper. But to me it explains further AI development more as a profiteering Ponzi scheme and not actual Golden Age of Humanity and Computing.
The paper is No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance which, it looks like, I haven't linked to before.