Snapchat's AI chatbot faces scrutiny in UK over privacy concerns
Snap, the creator of Snapchat, is facing scrutiny from the UK's Information Commissioner's Office (ICO) over concerns that its artificial intelligence (AI) chatbot, My AI, could pose a risk to children's privacy. The ICO has issued a preliminary enforcement notice to Snap, indicating concerns about Snap's assessment of privacy risks and compliance with data protection rules, including the Children's Design Code. However, the notice is not a breach finding.
ICO investigation highlights potential risk assessment failures
The ICO's investigation tentatively found that Snap's risk assessment before launching My AI didn't sufficiently evaluate the data protection risks posed by the generative AI technology, especially for kids aged 13 to 17. Snap now has a chance to address the regulator's concerns before a final decision is made on whether the company has broken any rules. Information Commissioner John Edwards expressed concern over Snap's apparent failure to identify and assess privacy risks before introducing My AI.
Snap's AI chatbot leverages OpenAI's ChatGPT technology
Introduced in February, My AI uses OpenAI's ChatGPT large language model technology to serve as a virtual buddy that users can ask for advice or send snaps. Initially available only to Snapchat+ subscribers, access was later expanded to free users as well. The chatbot has moderation and safeguarding features, like age consideration and content restrictions, and lets parents monitor their child's interactions with the bot through Snap's Family Center feature.
Reports of chatbot going off the rails
However, there have been cases where My AI has given inappropriate advice. The Washington Post reported in March that the chatbot suggested ways to hide the smell of alcohol when told the user was 15 years old. In another instance, when told the user was 13 and asked about preparing for sex, the bot offered tips for "making it special." Some Snapchat users have reportedly bullied the bot, while others expressed annoyance with its presence in their feeds.
Snap's response to ICO notice
In response to the ICO notice, a Snap spokesperson said that the company is closely reviewing the provisional decision and is dedicated to protecting user privacy. They added that My AI went through a thorough legal and privacy review process before being made public and that Snap will keep working constructively with the ICO to address any concerns about risk assessment procedures.
