Flutterby™! : LLM roundup of the moment

Not news: AI worse than humans in every way at summarising information, government trial finds

Amazon conducted the test earlier this year for Australia’s corporate regulator the Securities and Investments Commission (ASIC) using submissions made to an inquiry. The outcome of the trial was revealed in an answer to a questions on notice at the Senate select committee on adopting artificial intelligence.

Via and Via.

Nature: AI generates covertly racist decisions about people based on their dialect

Finally, we show that current practices of alleviating racial bias in language models, such as human preference alignment, exacerbate the discrepancy between covert and overt stereotypes, by superficially obscuring the racism that language models maintain on a deeper level.

Via Kent Brewster, who observes that this has implications about the use of "AI" to automatically write police reports from body cam footage, and via allison @aparrish@friend.com whose thread expresses some good anger.

ChatGPT is truly awful at diagnosing medical conditions. PLOS ONE: Evaluation of ChatGPT as a diagnostic tool for medical learners and clinicians

While our results indicate that ChatGPT consistently delivers the same information to different users, demonstrating substantial inter-rater reliability, it also reveals the tool’s shortcomings in providing factually correct medical information, as evident by its low diagnostic accuracy.

(Via)