Man spreads fake news about train crash using ChatGPT, gets arrested

By Ankita ChakravartiA man in China was arrested for making up a story about a train crash and spreading it online using a technology called ChatGPT. This is believed to be the first arrest in China for misusing ChatGPT. Police in Gansu province said the suspect, whose surname is Hong, was arrested for creating false information using artificial intelligence technology. The story was first noticed by cyber security officers in Kongtong county when they saw a fake news article that claimed nine people died in a train accident. The story was posted by more than 20 accounts on a Chinese platform called Baijiahao. The stories were clicked more than 15,000 times by the time authorities found out.

The police traced the origins of the article to a company owned by Hong, who was detained after they searched his home and his computer. He confessed to bypassing Baijiahao’s duplication check function and posting the fake story on multiple accounts. He used ChatGPT to quickly produce different versions of the same fake story by inputting elements of trending social stories from past years.

ChatGPT is not directly available to Chinese IP addresses, but Chinese users can still access its service with a VPN connection. Chinese IT outlets were experimenting with their versions of ChatGPT after Microsoft and Google announced their innovations.

China closely monitors its social media through firewalls to ensure no critical content against the ruling Communist Party of China (CPC). China’s top internet regulator is concerned that unchecked use of deep synthesis technology could lead to its use in criminal activities such as online scams or defamation. As ChatGPT has become more popular, China’s law enforcement agencies have voiced suspicion and warnings about the technology. In February, Beijing police warned the public to be wary of rumors generated by ChatGPT.

The suspect, Hong, is accused of “picking quarrels and provoking trouble,” which carries a maximum sentence of five years in jail. However, offenders can be jailed for ten years and given additional penalties if the offense is deemed especially severe. This is the first time the public has been made aware of an arrest by Chinese authorities after Beijing’s first provisions to regulate the use of “deepfake” technology officially took effect in January.