Google sets new rules for AI apps on Play Store
What's the story
Google has introduced new guidelines for developers creating AI apps distributed through its platform, Google Play.
The tech giant aims to curb the circulation of inappropriate and prohibited content by requiring these apps to prevent the generation of restricted content including sexual and violent material.
Additionally, these apps must provide a mechanism for users to report offensive content they encounter.
Google also stressed that developers need to "rigorously test" their AI tools and models to ensure user safety and privacy.
Advertising rules
Crackdown on inappropriate AI app advertising
Google is also taking action against apps that promote inappropriate use cases in their marketing materials.
Apps that suggest they can undress people or create nonconsensual nude images, regardless of whether they are actually capable of doing so, may be banned from Google Play.
This move is a response to the rising issue of AI-driven undressing apps advertising themselves on social media platforms.
The company's new guidelines aim to prevent the misuse and circulation of such content.
AI Misuse
Misuse of AI apps in schools
The use of AI apps has extended into schools across the US, with students reportedly using them to create and circulate deepfake nudes for bullying and harassment purposes.
In one instance, a racist AI deepfake of a school principal led to an arrest in Baltimore.
Google believes its new policies will help exclude apps featuring potentially harmful or inappropriate AI-generated content from Google Play, thereby addressing this growing issue.
Developer responsibilities
Guidelines for AI app developers
Google has outlined specific responsibilities for developers of AI apps.
These include not allowing the generation of any restricted content and providing users with a means to flag offensive and inappropriate content.
Developers are also prohibited from advertising their app's ability to break any of Google Play's rules.
Furthermore, they have been tasked with protecting their apps against prompts that might manipulate their AI features to create harmful and offensive content.
Testing protocols
Testing and documentation advice for developers
Google strongly advises developers to rigorously test their AI apps before launching.
The tech giant suggests using its closed testing feature to share early versions of apps with users for all kinds of feedback.
Not only should these tests be conducted, but they should also be documented as Google may request to review them in the future.
The tech giant is also releasing additional resources, such as its People + AI Guidebook, designed to assist developers in creating AI apps.