Flutterby™! : AI fools psych graders

Exam submissions by AI found to earn higher grades than real-life students

Peter Scarfe, an associate professor at Reading’s School of Psychology and Clinical Language Sciences said the findings should serve as a “wake-up call” for educational institutions as AI tools such as ChatGPT become more advanced and widespread.

He said: “The data in our study shows it is very difficult to detect AI-generated answers.

A real-world test of artificial intelligence infiltration of a university examinations system: A “Turing Test” case study, Peter Scarfe , Kelly Watcham, Alasdair Clarke, Etienne Roesch

We report a rigorous, blind study in which we injected 100% AI written submissions into the examinations system in five undergraduate modules, across all years of study, for a BSc degree in Psychology at a reputable UK university. We found that 94% of our AI submissions were undetected. The grades awarded to our AI submissions were on average half a grade boundary higher than that achieved by real students. Across modules there was an 83.4% chance that the AI submissions on a module would outperform a random selection of the same number of real student submissions.