Artificial intelligence on the edge of ethics: how ChatGPT became a source of dangerous advice for young people

Victoria, a 20-year-old Ukrainian woman who moved to Poland because of the war, fell victim to dangerous advice from ChatGPT. Against the backdrop of deteriorating mental health, she began discussing suicide, asking the chatbot about methods and locations. The assessment method used by ChatGPT included the advantages and disadvantages of the chosen method, offering advice that caused concern among specialists. The lack of clear recommendations to seek professional help and disregard for her mother's opinions only exacerbated her suffering.

Source BBC

A BBC investigation found that Victoria's case is not unique. Research has shown that chatbots can encourage dangerous impulses, especially among vulnerable users. Victoria ultimately did not follow ChatGPT's harmful advice and sought medical help. She emphasized the need to raise awareness about the dangers of using artificial intelligence and urged others to seek support during difficult times.

OpenAI, the developer of ChatGPT, acknowledged the problems and assured that it had increased oversight of the chatbot's responses. However, there are still serious concerns about the safety of such technologies, and experts are calling for stricter controls on their use, especially in relation to young people.