Flutterby™! : LLM maybe hits median IQ?

Next unread comment / Catchup all unread comments User Account Info | Logout | XML/Pilot/etc versions | Long version (with comments) | Weblog archives | Site Map | | Browse Topics

LLM maybe hits median IQ?

2024-03-06 18:14:43.427428+01 by Dan Lyke 2 comments

I have been trying to figure out what people are seeing in LLMs. I'm like "if I wanted an overconfident 8th grader spewing the sort of bullshit they think would impress their English teacher, but doesn't, I'd find one of those". I'm extremely skeptical of the ability of LLMs to add value to any process. LLMs seem like the late night TV kitchen appliance thing, looks really cool when the shysters shyst with it, but in practice not as handy as good sharp tools.

I have told often the tale of talking with a brilliant and fairly well known mathematician back when the memory consumption difference between floats and doubles was a thing, about some details of entropy in coordinate spaces used to represent geometry, and I said "wait a minute, that doesn't feel right", and we both kinda thought and scribbled for a moment before coming to the same conclusion, but it was plain that we'd gotten to that result through different means: He was doing symbolic manipulation, I was visualizing.

And my visualizing worked in 3 dimensions, his symbolic manipulation could be generalized to more.

In learning square dance choreography, and all of the different ways that people maintain mental state around square dance choreography, I'm seeing similar things: People who track dancers, people who track Xs and Os, people who track resolve points.

So I think that part of what I'm seeing with LLMs and the praise for them is people whose mental models revolve around a certain sort of language and symbolic manipulation see them as pretty good at that. They don't do well with the mental models that I carry, so I look at their output and roll my eyes.

And then there's just that they show potential, and some people are seeing that potential carried forward, and some aren't.

Anyway: Maxim Lott: AIs ranked by IQ; AI passes 100 IQ for first time, with release of Claude-3

[ related topics: Children and growing up Interactive Drama Technology and Culture Television Education Artificial Intelligence ]

comments in ascending chronological order (reverse):

#Comment Re: LLM maybe hits median IQ? made: 2024-03-06 19:25:42.868873+01 by: Dan Lyke

Related: RT berserk du soleil @aetataureate@dosgame.club

BMI and IQ are both sometimes explained as "not good metrics, but the only baseline we have and therefore useful." Anyone who says this to you is full of shit. Measuring an arbitrary thing with no evidence-based meaning, then comparing it to another of the same measure, still means nothing. How many times did you look to the left today? How many times did you touch something with your pinky fingers? Who the fuck cares?

#Comment Re: LLM maybe hits median IQ? made: 2024-03-06 21:40:00.30488+01 by: spc476

As I read articles about testing or Agile or whatnot, I always attempt to "apply" it to the type of programming I do, and often I'm like, "Wait? What type of programming do you do? What language? And why do you write like what you do applies everywhere?"

Add your own comment:

(If anyone ever actually uses Webmention/indie-action to post here, please email me)

Format with:

(You should probably use "Text" mode: URLs will be mostly recognized and linked, _underscore quoted_ text is looked up in a glossary, _underscore quoted_ (http://xyz.pdq) becomes a link, without the link in the parenthesis it becomes a <cite> tag. All <cite>ed text will point to the Flutterby knowledge base. Two enters (ie: a blank line) gets you a new paragraph, special treatment for paragraphs that are manually indented or start with "#" (as in "#include" or "#!/usr/bin/perl"), "/* " or ">" (as in a quoted message) or look like lists, or within a paragraph you can use a number of HTML tags:

p, img, br, hr, a, sub, sup, tt, i, b, h1, h2, h3, h4, h5, h6, cite, em, strong, code, samp, kbd, pre, blockquote, address, ol, dl, ul, dt, dd, li, dir, menu, table, tr, td, th

Comment policy

We will not edit your comments. However, we may delete your comments, or cause them to be hidden behind another link, if we feel they detract from the conversation. Commercial plugs are fine, if they are relevant to the conversation, and if you don't try to pretend to be a consumer. Annoying endorsements will be deleted if you're lucky, if you're not a whole bunch of people smarter and more articulate than you will ridicule you, and we will leave such ridicule in place.

Flutterby™ is a trademark claimed by

Dan Lyke
for the web publications at www.flutterby.com and www.flutterby.net.