How to check if a bad answer was AI-generated

It was pretty much inevitable, but I've just seen the first profile building for SEO/spam purposes with answers generated by a large language model like ChatGPT. Such answers seem to make sense at first glance, with decent grammar and some logical structure, but to any topic expert, they have a good chance to stand out for being pretty wrong and not actually all that helpful. Since it most certainly won't be the last one, I figure it's worth pointing out how you can easily check for this:

1) Copy the entire text of the answer.
2) Go to https://www.zerogpt.com/
3) Paste the text into the large textarea
4) Hit the blue "Detect Text" button.

If the answer is "100% AI GPT*" (or for large answers, if the percentage is well above 50%, with long continuous blocks of yellow-highlighted text), then congratulations, you've caught yourself an AI answer. If you're a respected member / travel guru, please feel free to report the answer as "duplicate".

FWIW, to give you some feeling for what to expect, this is the text which gave away this particular account, in answer to a question about how to arrange booking two stopovers in Singapore when flying between two other destinations:

It really depends on how much you're willing to spend, but if you don't mind paying a bit more for the convenience, I would suggest buying individual direct flights. That way, you won't have to worry about making sure your connections line up, or having to rush to the airport if there's a delay. Plus, you won't have to worry about any additional luggage fees for hopping off in Singapore.

[ Edit: Edited on 1 Mar 2023, 19:20 GMT by Sander ]

Interesting!

When I've spotted a reply that looks like it's duplicated from the internet, I usually copy and paste it into Google and it often shows the exact webpage where the text was published. There is a reply from a couple of days ago which I might try in zerogpt now.

Thanks Sander

Quoting Borisborough

There is a reply from a couple of days ago which I might try in zerogpt now.

It's been picked up already in the Explore UK thread 'written' by the same spammer!

Really useful to know about this site. Thanks.

I tried it out on a post I'd previously been dubious about but google search hadn't worked. Result was 100% AI, duly reported as duplicate. Came back to this thread and it's the same spammer as mentioned above. :-)

Yes, I think we all get the sense of a "wrong 'un" but don't necessarily have anything concrete to point to or to justify a report.

I think for brevity in such cases I shall just be more liberal in flagging pointless posts. Other times, when I have more time, I'll play with Zerogpt out of curiosity, but as a general rule if they're posting crap I'll be happy flagging it.

I verified that that particular user was also reported for spamming elsewhere so definitely a correct result here.

These AI contributions are going to make our life really difficult ZeroGPT may work for a while, but I feel like it's such an arms race we're in now. In the future we may need something like a "Verified real human" badge that members can somehow unlock. How we verify.. not quite sure.

Well, I am pleased that I am not a moderator ! But when we pose a question to a forum we expect a range of replies. Not all will be relevant or helpful regardless of whether the authors are humans or robots. We can all sort these out for ourselves. So the occasional reply generated by AI won’t matter. Lots of replies generated by AI would be different.
However, the issue that worries me is the Community Travel Guide. I really like this feature. Entries are often a bit out of date, and they are sometimes incomplete, but I have never found one that is fabricated.
There are plenty of other travel sites where the author clearly has zero knowledge of their subject and is just phishing; or trying to cheat travellers by seeking bookings for non-existent attractions. I have an egregious example regarding my own village. If you Google ‘Kid Friendly Things to do in Palapye’ you will discover this….

"One of the most popular attractions is the Palapye Zoo. The Palapye Zoo is home to many different animals, including lions, tigers, and cheetahs. There is also a petting zoo, where children can interact with friendly animals. Another popular activity for kids is the Palapye Water Park. The water park has many different slides and pools, and is a great place to cool off on a hot day."
And later … "The Ditshiping Rock Paintings are some of the most ancient and important rock art in Botswana. Located just outside of Palapye, these paintings provide a unique opportunity to see how the Bushmen have lived for centuries."

This all sounds attractive. But sadly there is no Palapye Zoo. There never has been. There is no water park either. Ditshiping does exist but is 600 km away in the middle of the Okavango Delta. I would be surprised if there were any rocks there to paint on given the location. The author of this fictitious information is supposedly in the USA. But they could just as easily be in Nigeria ……

Humans bent on mischief could already post fiction like this in our community travel guide, but they are unlikely to bother. Too much effort for no real reward. Using ChatGPT to help create entries should dramatically reduce the effort required, but so long as the rewards are kept strictly at zero we shouldn’t be an attractive target.

Of course AI could be used quite benignly to simply help those with a limited command of English to communicate. If I had something that would help me write better French I would certainly use it.

But heaven help us if AI systems are left to create entries entirely by themselves. To know whether the above excerpts above are true or false you need local knowledge. AI systems don’t have this. They draw their knowledge from a training set of digital sources, which will frequently be whatever they can find on the internet. And then they uncritically blend the good, the bad and the ugly. Until someone can teach a computer to carefully weigh the credibility of its sources the old rule of garbage in, garbage out will always rule. And there is no shortage of garbage on the internet for a computer to use as the input. I can see the fictitious Palapye Zoo being endlessly recycled.

We can all do something to guard against this by occasionally checking the Travel Guide entries for places that we know well. But hopefully we are doing this anyway …. we can't leave it all to Utrecht.

FWIW, I flagged two more spambots using AI-generated answers today - although these were easy, as they included obvious spam links right in their first message, so would've been flagged even without the zerogpt check.

Wheee! I've now spotted our first (?) blog featuring a fully AI-generated trip report, including AI-generated photos.

(This one has the feel of a student exercise more than a spam setup, but as a proof of concept for the latter, it's disturbing.)

Reality ends here. Please keep your sanity inside the boat at all times.

[ Edit: Edited on 17 Mar 2023, 12:33 GMT by Sander ]