Flutterby™! : LLMs are hacking us

LLMs are hacking us

2025-06-09 21:54:46.815667+02 by Dan Lyke 0 comments

Wilhelm Fitzpatrick @rafial@masto.hackers.townM>a<

Sitting here with gasping like a fish after reading a detailed description of using an AI coding tool to write a unit test for a simple function in which it is mentioned:

* The AI was apparently unable to "reason" about correct dependency injection or mocking
* 20 prompts were written to "iteratively refine" the solution
* 2000 lines of code were generated of which 50 were used

...and at the end the writer concludes "the final result was successful and saved time over doing it manually"

😱

Wilhelm Fitzpatrick @rafial@masto.hackers.town

I continue to be astounded by how LLMs seem to hack the basic reasoning processes in our own meat brains!

comments in ascending chronological order (reverse):

Add your own comment:

(If anyone ever actually uses Webmention/indie-action to post here, please email me)

Format with:

(You should probably use "Text" mode: URLs will be mostly recognized and linked, _underscore quoted_ text is looked up in a glossary, _underscore quoted_ (http://xyz.pdq) becomes a link, without the link in the parenthesis it becomes a <cite> tag. All <cite>ed text will point to the Flutterby knowledge base. Two enters (ie: a blank line) gets you a new paragraph, special treatment for paragraphs that are manually indented or start with "#" (as in "#include" or "#!/usr/bin/perl"), "/* " or ">" (as in a quoted message) or look like lists, or within a paragraph you can use a number of HTML tags:

p, img, br, hr, a, sub, sup, tt, i, b, h1, h2, h3, h4, h5, h6, cite, em, strong, code, samp, kbd, pre, blockquote, address, ol, dl, ul, dt, dd, li, dir, menu, table, tr, td, th

Comment policy

We will not edit your comments. However, we may delete your comments, or cause them to be hidden behind another link, if we feel they detract from the conversation. Commercial plugs are fine, if they are relevant to the conversation, and if you don't try to pretend to be a consumer. Annoying endorsements will be deleted if you're lucky, if you're not a whole bunch of people smarter and more articulate than you will ridicule you, and we will leave such ridicule in place.

Flutterby™ is a trademark claimed by

Dan Lyke

for the web publications at www.flutterby.com and www.flutterby.net.