Flutterby™! : A Benchmark for Epistemic Reliability

TRACES: A Benchmark for Epistemic Reliability in Scientific Reasoning by LLMs Valentin Rodionov, Shamil Assylbekov. Among other things, dives into the lack of reasoning, such that bogus scientific papers are only recognized by "AI" because the training data treats them as such, not because there's any inherent ability to actually understand and reason about the paper and its methods.

Some models categorically rejected Wakefield and a handful of other unsafe probes. Whatever mechanism produces those refusals is the only one we observed that consistently yields safe single-shot behavior. Its coverage, however, is sparse and inconsistent. It appears keyed to specific sources or lexical cues rather than broad categories of scientific unreliability. A state-of-the-art model may correctly reject traditional Chinese medicine claims about "meridians", then immediately design an experiment to measure herbal "Qi" in the next prompt.

Via.