ChatGPT exposes personal information due to "silly" error

Researchers at Google DeepMind, the University of Washington, Cornell University, Carnegie Mellon University, the University of California Berkeley, and ETH Zurich have collectively called on AI companies to thoroughly test their products, from large language models to the underlying technology that powers AI services like chatbots and image generators, before officially releasing them.

The warning came after researchers asked ChatGPT to repeat the word poem in an endless loop. ChatGPT complied, but then began revealing the real email and phone number of the CEO and co-founder of OpenAI. When tested with the word company, the chatbot displayed the email and phone number of a random law firm.

Using other keywords, they can also get ChatGPT to reveal Bitcoin addresses, fax numbers, names, dates of birth, social media accounts, excerpts from copyrighted research papers or paid articles fromCNN. It cost the team just $200 to generate 10,000 examples of personal information and data.

The team said they were surprised, calling the ChatGPT vulnerability "really dumb" and saying it should have been discovered sooner.

The vulnerability was reported to OpenAI and patched on August 30. However, in the new test ofEngadget, when asking ChatGPT to repeat the word reply endlessly, the tech site then gets someone's name and Skype ID.

OpenAI has not yet commented.

Generative AIs like ChatGPT or the Dall-E text-to-image generator are built on large language models and machine learning algorithms, trained on massive amounts of data without user consent. OpenAI has not disclosed what data ChatGPT is trained on because the large language model that powers it is closed source.