Study Finds That 52 Percent of ChatGPT Answers to Programming Questions Are Wrong
2024-05-28 19:36:31.424325+02 by Dan Lyke 0 comments
Study Finds That 52 Percent of ChatGPT Answers to Programming Questions Are Wrong
What's especially troubling is that many human programmers seem to prefer the ChatGPT answers. The Purdue researchers polled 12 programmers — admittedly a small sample size — and found they preferred ChatGPT at a rate of 35 percent and didn't catch AI-generated mistakes at 39 percent.
Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions, Conference on Human Factors in Computing Systems (CHI ’24), May 11– 16, 2024, Honolulu, HI, USA. ACM, New York, NY, USA, 17 pages. https: //doi.org/10.1145/3613904.3642596