Telegram user misusing AI to generate paid nonconsensual adult content
A Telegram user is exploiting artificial intelligence (AI) to create adult content featuring any individual, provided they receive photographs of the person. This service, advertised on X, costs as little as $10 and extends beyond celebrities to everyday individuals. The user operates in a Telegram chat room named "IRL Fakes," where they display real photos alongside the AI-generated adult images.
Misuse of AI intensifies ethical concerns
The user's actions raise serious ethical issues, as they not only create nonconsensual adult content but also counterfeit nude images of minors. The intent behind this practice appears to be showcasing their skills to potential clients, and embarrassing the targeted women. This alarming misuse of generative AI tools underscores the need for stricter regulations and safeguards against such practices.
Potential customers can even try out 'samples'
The user operating the Telegram channel said in its "general" chat, "For my custom fakes, I need around 10 face photos of the subject (more is welcome), I can do it with less but results might vary." "If you want a sample before buying, I do 1 bikini/lingerie photo for free. If you have questions or want one done, DM me." However, when contacted by 404 Media, the user deleted their Discord server, as well as Telegram and Twitter accounts.
Price chart and tools used
The porn image creator in question used a free AI tool called IP-Adapter for image generation. This tool created by China's Tencent, permits users to "prompt a text-to-image generator with an image." The X-rated image-maker advertised their services across several platforms, including X, where they had over 1,000 followers and a "Price Sheet" offering "custom fakes." One picture cost $10, three for $25, five for $40, and 10 images cost $70.
How were the images made?
The images in the Telegram channel were made by borrowing photographs of real people. These pictures were used to train a Stable Diffusion model, in order to recreate the likeness of the people in those pictures. Later, text prompts were added so that their likeness could be recreated in any sexual scenario that could be described via words.
What did Telegram and Discord say?
In response to the incident, a Telegram spokesperson said to 404 Media, "Since its creation, Telegram has actively moderated harmful content on its platform including illegal pornography." "Moderators use a combination of proactive monitoring of public parts of the platform and user reports to remove content that breaches Telegram's terms of service," the company added. Meanwhile, Discord claimed, "Whether AI-generated or not, non-consensual adult intimate media has no place on Discord." X is yet to comment on the issue.
