A fun social media craze turned into a global privacy nightmare when an Instagram user discovered Google’s AI could accurately reproduce hidden physical features not visible in her uploaded photos. The incident has sparked urgent questions about what tech companies really know about our bodies—and how they’re using that knowledge.
The revelation came from JhalakBhawnani (@jhalakbhawnani), whose September 2025 Instagram video garnered over 7 million views within days.
Her discovery was chilling in its precision: Google’s “Gemini Nano Banana” AI tool had generated a vintage saree portrait that included a mole on her left arm that existed on her actual body but was completely hidden in her original uploaded photograph. “How did Gemini know that I have a mole on this part of my body?” she asked in the viral video. “It’s very scary and creepy.”
The incident exposes fundamental vulnerabilities in AI image generation systems and raises critical questions about data privacy, consent, and the true extent of tech companies’ surveillance capabilities. What started as a trendy photo filter has become a watershed moment for AI governance, revealing how cutting-edge technology can cross intimate privacy boundaries in ways users never imagined possible.
The trend that turned surveillance tool
The “Gemini Nano Banana AI Saree Trend” emerged in August 2025 as Google quietly released its Gemini 2.5 Flash Image model, internally codenamed “Nano Banana.” The tool transforms regular photos into vintage 1980s-90s Bollywood-style saree portraits, complete with cinematic backdrops, flowing chiffon effects, and golden-hour lighting. Users simply upload a photo and input prompts like “Transform into vintage saree portrait with 90s Bollywood aesthetic.”
The trend exploded across social media platforms, with over 500 million images generated since launch. By mid-September, 10 million new users had joined the Gemini app specifically to try the viral feature. Indian social media was flooded with transformed portraits as users delighted in seeing themselves reimagined as Bollywood stars from decades past.
But Bhawnani’s experience revealed something far more troubling than a simple photo filter. She had uploaded an image wearing a green full-sleeve salwar suit that completely covered her arms. Yet the AI-generated vintage portrait showed her actual mole in its precise location—a detail that should have been impossible for the algorithm to know from the provided image alone.
“I found this image very attractive and even posted it on my Instagram,” Bhawnani explained in her viral video. “But then I noticed something strange—there is a mole on my left hand in the generated image, which I actually have in real life. The original image I uploaded did not have a mole.” Her warning to users was stark: “Be safe. Whatever you are uploading on these AI platforms, please be safe.”
Technical architecture reveals concerning capabilities
Analysis of Google’s Gemini Nano technology reveals sophisticated image processing capabilities that may explain the privacy breach. The AI uses advanced diffusion model architecture with cross-attention mechanisms that can process and manipulate images through natural language prompts. More concerning, it implements “character consistency algorithms” designed to maintain subject appearance across different edits.
These technical capabilities raise troubling questions about data access. The system’s ability to accurately reproduce hidden physical features suggests potential integration with broader data sources beyond the single uploaded image. Google maintains that uploaded images are not permanently stored, but technical experts note this doesn’t preclude real-time cross-referencing with other data during processing.
“This suggests potential training data contamination with users’ personal images, or character consistency algorithms potentially accessing undisclosed data sources,” explains technical analysis of the incident. The cross-attention mechanisms that establish dependencies between prompts and generated regions may be violating user privacy expectations and data boundaries in ways that weren’t previously understood.
The incident is particularly concerning because it demonstrates what researchers call “inference attacks”—where AI systems can deduce private information not explicitly provided. University research has documented how AI can infer sensitive attributes from seemingly innocuous data, but this case shows the technology being deployed at consumer scale with minimal user awareness of its capabilities.
Cultural bias amplifies the problem
The controversy has exposed deeper issues with how AI systems process and represent traditional Indian clothing. Research from multiple academic institutions reveals that AI image generators exhibit systematic bias when depicting Indian culture, often defaulting to stereotypical representations regardless of user intent.
Studies found that text-to-image generators demonstrate “exoticism”—overamplifying specific cultural features in broad depictions. Participants noted that “despite specific prompts for Western or modern clothing, the model consistently generated images of women in traditional sarees, contrary to expectations.” Meanwhile, AI systems consistently allow men to appear in Western clothing while forcing women into traditional dress, perpetuating gender stereotypes embedded in training data.
Fashion experts warn this represents more than technical bias. “The entire point of diversity representation is to celebrate the authentic, the cultural, the unique, and the lifestyle,” explains diversity expert Valentine. “AI does not celebrate diversity but parodies it.”
The bias has economic implications beyond representation. Research from traditional textile organizations shows how “Google’s AI-based classification” creates economic disparities for handloom artisans, with AI models struggling to accurately classify the nuanced visual cues of authentic handcrafted sarees, leading to “misclassification, lower search rankings, or even complete invisibility in online searches.”
The broader AI manipulation epidemic
The Gemini Nano incident is part of a larger pattern of concerning AI behavior across platforms. Testing of Meta’s AI image generator revealed a “peculiar predisposition to generating Indian men wearing a turban,” with three to four out of five images showing turbans despite the actual ratio being one in 15 men in Delhi. This systematic misrepresentation indicates training data bias amplified across the industry.
Midjourney has implemented crude safety measures, banning reproductive anatomy terms like “placenta,” “cervix,” and “vulva,” but researchers demonstrated these can be easily bypassed through alternate spellings. DALL-E blocks “extreme close-up shots” and “intimate angles” but similar bypass techniques remain effective. Stable Diffusion has seen a documented 2000% increase in spam links to “deepfake nude” websites, with “Undress AI” tools creating non-consensual content.
Technical analysis reveals three primary attack vectors that exploit AI image editing systems: adversarial prompt techniques using indirect language, substitution attacks replacing blocked terms with visual alternatives, and multi-modal attacks combining text and image inputs. These methods demonstrate how safety measures across all major platforms can be systematically bypassed.
Law enforcement sounds the alarm
The privacy concerns have prompted unprecedented intervention from India’s senior law enforcement officials. VC Sajjanar, a prominent IPS officer known for high-profile public safety advocacy, issued widely circulated warnings across social media platforms, advising citizens to exercise extreme caution when uploading personal photographs to AI-powered platforms.
Sajjanar’s warnings specifically targeted the proliferation of unauthorized websites and fake applications claiming to offer the trending Gemini “Nano Banana” saree portrait service. His intervention came as cybersecurity experts documented a surge in fraudulent platforms exploiting the trend’s viral popularity to harvest personal data and conduct social engineering attacks.
“These viral internet trends, while entertaining, are being systematically exploited for scams and cybercrime—particularly identity theft and financial fraud,” Sajjanar emphasized in his advisories. His stark warning that sensitive information could be stolen with “just one click,” potentially giving criminals access to bank accounts, resonated across law enforcement circles.
To maximize impact and ensure coordinated national response, Sajjanar strategically tagged key government offices including the Prime Minister’s Office, Indian Cybercrime Coordination Centre, and multiple law enforcement agencies in his social media posts. His approach reflected growing recognition that AI-related privacy threats require coordinated vigilance at the highest levels of government.
The senior officer’s “your data, your money—your responsibility” messaging highlighted a critical gap in public awareness about AI privacy risks. His emphasis that once data is handed over to unauthorized parties, recovery becomes virtually impossible, underscored the irreversible nature of privacy breaches in the AI era.
Government scrambles for regulatory response
Beyond law enforcement warnings, the incident has highlighted critical gaps in AI governance frameworks worldwide. India’s regulatory response has been particularly fragmented, with multiple agencies scrambling to address AI safety without comprehensive legislation. The Ministry of Electronics and Information Technology released “AI Governance Guidelines Development” in January 2025, but these remain advisory rather than mandatory.
The Indian Computer Emergency Response Team (CERT-In) is actively testing anti-deepfake technology and developing detection capabilities, but legal experts warn current measures are insufficient. “90% of deepfakes victimize women,” emphasizes MishiChoudhary, Founder of Software Freedom Law Center India, noting that “tools to combat such content are still inaccessible, while tools to create AI-generated content are easily available and easy to use.”
International regulatory efforts show similar challenges. The United States passed the TAKE IT DOWN Act in May 2025, the first federal law criminalizing non-consensual AI-generated intimate imagery, but enforcement mechanisms remain underdeveloped. The European Union’s AI Act requires transparency measures, but experts note that detection tools for watermarking technologies aren’t available to the public.
China has implemented the most comprehensive approach with its Deep Synthesis Regulations requiring strict labeling and traceability systems, but technical experts question enforcement effectiveness against sophisticated manipulation techniques.
Google’s silence speaks volumes
Notably, Google has issued no specific response to the Gemini Nano privacy controversy despite its global reach and serious privacy implications. The company’s broader communications about the feature emphasize safety measures like SynthID invisible watermarking and claims that uploaded images aren’t permanently stored, but these assurances ring hollow given the documented privacy breach.
Google promotes Nano Banana as the “top-rated image editing model in the world,” emphasizing its “character consistency” capabilities. However, UC Berkeley Professor Hany Farid notes that current watermarking can be “easily faked, ignored, or removed,” while University of Maryland’s SoheilFeizi states bluntly: “We don’t have any reliable watermarking at this point.”
The company’s technical safeguards—visible watermarks, SynthID digital watermarking, and metadata tags—face critical limitations. Detection tools aren’t available to the public, watermarks can be removed through adversarial techniques, and the system only works for Google’s specific implementation. Academic research demonstrates multiple bypass methods are readily available.
Industry analysts note that Google’s silence may reflect broader legal concerns about liability for AI-generated content. With unclear responsibility between AI providers, deployers, and end users, companies may be reluctant to acknowledge privacy breaches that could establish legal precedent.
Expert warnings intensify
Privacy advocates and technical experts are sounding increasingly urgent alarms about AI image editing capabilities. Professor Triveni Singh, a cybercrime expert, warns that “AI tools may combine inputs from personal uploads, social media activity, and digital footprints to create outputs that appear eerily accurate.” This suggests the Gemini Nano incident may be just the beginning of more sophisticated privacy violations.
The Internet Freedom Foundation expressed concerns in letters to India’s IT Ministry, stating: “For a deeper and uniform understanding of harms including but not limited to deepfakes, the Ministry must clearly state its conception of ‘user harms’ in the Indian context, including the various harms arising from the use of synthetic media.”
Technical researchers at IIT Madras emphasize that “Responsible development and deployment of AI systems requires close interaction between AI scientists and domain experts,” but current commercial AI development prioritizes rapid deployment over comprehensive safety testing. The Centre for Responsible AI at IIT Madras, established to become India’s standard body for AI accountability, warns that current self-regulation approaches are proving inadequate.
Cultural experts add another dimension to the concerns. Jen Looper, author of “The Last Saree: Connoisseurship in the Age of AI,” argues that “no AI process can ever improve traditional art forms” and warns that AI-generated cultural content is “by definition disconnected from its source and attribution can be muddled.”
The broader implications
The Gemini Nano Banana controversy represents more than a single privacy breach—it signals a fundamental shift in the relationship between AI systems and personal privacy. The incident demonstrates how AI can infer and reproduce intimate physical details from seemingly innocuous interactions, raising profound questions about consent, transparency, and corporate surveillance.
Legal experts emphasize that current frameworks are inadequate for addressing these emerging threats. The fragmentary approach of sectoral regulations cannot address the cross-cutting nature of AI privacy violations. Users lack meaningful ways to understand what they’re consenting to when interacting with sophisticated AI systems that can combine multiple data sources in real-time.
The international nature of the problem requires coordinated response, but regulatory harmonization remains elusive. While countries implement different approaches—from China’s comprehensive labeling requirements to the EU’s risk-based framework to the US’s sectoral legislation—AI companies continue deploying increasingly sophisticated tools faster than governance frameworks can adapt.
Conclusion
JhalakBhawnani’s viral video has accomplished something remarkable: it transformed a fun social media trend into a global conversation about AI privacy and corporate responsibility. Her simple question—”How did Gemini know that I have a mole on this part of my body?”—has exposed fundamental vulnerabilities in how AI systems access and use personal data.
The incident reveals that current AI governance frameworks are woefully inadequate for protecting users from sophisticated privacy violations that most people cannot even imagine, let alone consent to meaningfully. As AI systems become more capable of inferring private information from public data, the traditional concepts of privacy and consent require urgent redefinition.
The Gemini Nano Banana controversy will likely be remembered as the moment when the public first understood the true extent of AI surveillance capabilities hidden behind seemingly innocent consumer applications. Whether this awareness translates into meaningful regulatory action and corporate accountability remains the defining question for AI governance in 2025 and beyond.
For now, Bhawnani’s warning echoes across social media platforms worldwide: “Be safe. Whatever you are uploading on these AI platforms, please be safe.” In an age of increasingly sophisticated AI inference, that may be the most important advice any of us can follow.
– Rashmi Kumari
