Code completion gets it wrong
2025-07-29 17:17:14.6007+02 by Dan Lyke 1 comments
Ars Technica: Two major AI coding tools wiped out user data after making cascading mistakes — "I have failed you completely and catastrophically," wrote Gemini.. I was gonna get completely up in arms over that anthorpomorphization, but the author gets it:
It's worth noting that AI models cannot assess their own capabilities. This is because they lack introspection into their training, surrounding system architecture, or performance boundaries. They often provide responses about what they can or cannot do as confabulations based on training patterns rather than genuine self-knowledge, leading to situations where they confidently claim impossibility for tasks they can actually perform—or conversely, claim competence in areas where they fail.
If there are positive things that come out of the mass delusion that is AI, backups and version control are gonna be two of them...