On LLM summarization
2024-07-23 02:14:17.868632+02 by Dan Lyke 0 comments
When ChatGPT summarizes, it actually does nothing of the kind.
... I just realised the situation is even worse. If I have 35 sentences of circumstance leading up to a single sentence of conclusion, the LLM mechanism will — simply because of how the attention mechanism works with the volume of those 35 — find the ’35’ less relevant sentences more important than the single key one. So, in a case like that it will actively suppress the key sentence.</blockqutoe>