A single prompt, a lifetime of trauma: The dark side of AI

Concerns have been raised about the alleged misuse of artificial intelligence tools on the platform X (formerly Twitter) to create sexually explicit images of women without their consent.

Men were found using the platform’s AI chatbot, Grok, to alter photographs of women and generate sexualised images. The images were created by prompting the tool to place women in bikinis or sexually suggestive situations, without consent.

While artificial intelligence is intended to improve creativity and productivity, it is instead being used to facilitate abuse. The ease of generating altered images through AI has reduced the effort required to cause harm, increasing the scale and speed of digital abuse.

Such tools are already being used to generate non-consensual sexual content of real people, including minors. Safeguards are lagging behind the rapid spread of such misuse.

The psychological impact of such content, where a single altered image could affect how a woman experiences the internet, public spaces and personal safety.

Under Indian law, altering a woman’s image to sexualise her without consent is illegal. Such acts attract criminal liability under the Information Technology Act, 2000, including provisions related to privacy violations and the transmission of obscene or sexually explicit content. The Bharatiya Nyaya Sanhita, 2023, criminalises voyeurism, online harassment, intimidation and defamation, including in cases involving AI-generated content.