People could barely tell AI from human in 72,000 judgments, until they asked follow-up questions

A study in Nature Human Behaviour led by Mengmi Zhang at Nanyang Technological University collected more than 72,000 judgments from 1,916 people on responses from 636 humans and 37 AI models. In the main language tests judges were right 53% to 57% of the time, close to chance. When judges could ask their own follow-up questions, they caught every one of the seven live conversations with GPT-3.5 Turbo. The data were collected in 2023 and 2024 on models two generations old, so the result is a floor on how hard detection has become. SourcesCA

Sources

• CAI often passes for human, but asking follow-up questions can trip it up (Tech Xplore)techxplore.com
• ANature Human Behaviour, DOI 10.1038/s41562-026-02590-6nature.com

This story ran in the Morning Brief of October 30th 2026.