Right now, it’s hard to rely on apps that claim to detect ChatGPT writing

RASHMEE ROSHAN LALL February 21, 2023
Screenshot of test I set GPTZero on Feb 21


Test set on January 30

On February 22, the creators of GPTZero, the  app that’s meant to spot robot-writing, will be talking about the future of education with Unesco, the UN body that deliberates on issues related to education, arts, sciences and culture.

Unesco, like everyone else, must surely be following the ongoing debate over student assignments. How does a university teacher ensure that summative assessment submissions genuinely reflect the thoughts and learning of their student? How to prevent the temptation to turn in work penned by artificial intelligence generative text tools? How to spot copycat drivel?

Tools such as GPTZero are supposed to provide a solution and Edward Tian, the Princeton student who created it very recently, assures that it’s much smarter now. No longer fed a diet of news articles, the “updated GPTZero model [is] trained and tested with student written essays.”

Now fine-tuned, the product will be tailored “for educators worldwide”, writes Mr Tian, and has “a 99 per cent accuracy classifying human text as human, and 85 per cent classifying AI text”.

GPTZero used to have the tagline: Detect AI Plagiarism. Accurately

Now, it declares: Humans Deserve the Truth

Just like before, I immediately went on to GPTZero and tested it with a sonnet written by ChatGPT. (Incidentally, I got the same result on January 30, as well. The screen grabs are above.)

The result was near-instantaneous. And wrong!

Just like before, GPTZero said the sonnet was “likely to be written entirely by a human.”

At least for now, it may be hard to beat gut human intuition.

Also read:

AI, your (overly) helpful, slightly batty editor?

How now, thou chatbot Bard?

The AI bubble is like the Dutch tulip mania of the 1630s

Will Dr Clever Clogs, otherwise known as ChatGPT, be at Davos 2024?

AI magic: Turner painting of a 21st century Greenwich neighbourhood

Can this ChatGPT detector unerringly spot robotic idioms? Nope

How to beat AI – for dummies. Hint: learn a craft

ChatGPT isn’t Keats & won’t reinvent the sonnet like Terrance Hayes…and that’s fine

‘Write a blog about the status of ChatGPT’

An AI breakthrough that may mean curtains for poets, journalists…and me