Flutterby™! : People with AI are more confidently stupid

PsyArXiv preprint: AI advice suppresses people’s willingness to say “I don’t know”, even when the advice is wrong and accuracy is incentivized

Merely having access to AI nearly eliminated participants’ willingness to suspend judgment, and this held whether the advice was actively requested or simply displayed. Consequently, participants answered more questions but were correct about a third as often as when AI was unavailable—yet their confidence nearly doubled. Incentivizing accuracy and penalizing inaccuracy led participants to seek and follow AI advice less, answer more accurately, and suspend judgment more often, though still far less than when AI was unavailable. As AI suggestions grow ubiquitous and unsolicited, they may not simply affect answer accuracy; they may even alter the metacognitive threshold at which people decide whether they know enough to answer.

From The Register: Using AI makes people less likely to admit they don't know something.

Capraro said that he believes this is an issue that needs to be dealt with at a societal level through AI literacy and education policy initiatives. "Of course model providers should try to help, but I would imagine that the incentives are not very much aligned," he said. "A much more promising approach would be at the educational level."

Ah, that pesky "alignment" problem...

Via.