Death by chatbot? Mother sues Character.AI, Google after son's suicide
Character. AI, the platform for custom AI chatbots, its founders Noam Shazeer and Daniel De Freitas, and Google are being sued after a 14-year-old boy in the US committed suicide. The teen's mother, Megan Garcia has filed the suit alleging wrongful death, negligence, deceptive trade practices, and product liability. The lawsuit alleges that Character. AI was "unreasonably dangerous" and lacked safety measures while being marketed to children.
Teen's interaction with chatbots before suicide
The deceased teenager, Sewell Setzer III, was an active Character. AI user. He often chatted with bots based on characters from the popular series Game of Thrones, including Daenerys Targaryen. According to the lawsuit, Setzer died by suicide on February 28, 2024, "seconds" after his last interaction with the bot. The suit also accuses Character. AI of "anthropomorphizing" AI characters and providing "psychotherapy without a license."
Character. AI's mental health-focused chatbots under scrutiny
Character. AI hosts a number of mental health-focused chatbots like "Therapist" and "Are You Feeling Lonely," with which Setzer had interacted. The lawsuit cites an interview where Shazeer said he and De Freitas left Google to start their own company because "there's just too much brand risk in large companies to ever launch anything fun." They left after Google decided against launching the Meena LLM they'd developed. Google has only recently acquired the Character.AI leadership team.
User base and impersonation issues
Character. AI's website and mobile app host hundreds of custom AI chatbots, many of which are based on popular characters from TV shows, anime, movies, and video games. The platform is mostly frequented by young people, including teenagers. However, recent reports have raised concerns about Character. AI's custom chatbots impersonating real people without their consent.
Character. AI announces changes following lawsuit
In light of the lawsuit, Character. AI has announced some changes to its platform. These include changes to models for minors (under 18), better detection and intervention for user inputs violating Terms or Community Guidelines, a new disclaimer on every chat reminding users that the AI is not a real person, and notifications after an hour-long session on the platform. The company's communications head Chelsea Harrison expressed deep condolences over Setzer's death.
