Fake disease “bixinomania” fooled AI systems and academic community
A researcher invented a fictitious eye disease called “bixinomania” and posted two fake articles about it on an academic server. Although the texts explicitly admitted the invention, major AI systems treated the disease as real for several weeks: Google Gemini linked it to blue light, Perplexity reported its prevalence, and ChatGPT provided symptom consultations to users.
The counterfeit study was even cited in a peer-reviewed journal, which had to retract the issue after intervention by Nature. Neither AI systems nor human researchers noticed the forgery, highlighting how people cite AI-generated references without verifying their content.
The US regulator FDA uses ChatGPT to draft official letters, and the CEO of a New York hospital plans to integrate AI algorithms into medical practice. ChatGPT is also often employed to consult patients.
This incident clearly illustrates the risks of excessive trust in AI-generated information, especially in science and medicine. It underscores the growing need for stricter control and data verification to prevent fake content from spreading.
Going forward, it will be crucial to develop methods to detect and prevent the use of fabricated scientific materials and to enhance critical thinking among both AI users and human researchers.
