研究称 OpenAI 和 Google 新图像模型更容易被诱导生成裸露图像

Privacy Center

Currently, only residents from GDPR countries and certain US states can opt out of Tracking Technologies through our Consent Management Platform. Additional options regarding these technologies may be available on your device, browser, or through industry options like AdChoices. Please see our Privacy Policy for more information.

Social Media

These cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

Essential

This website uses essential cookies and services to enable core website features and provide a seamless user experience. These cookies and services are used to facilitate features such as navigation, remember user preferences, and ensure the security of the website.

Targeted

These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Performance

These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. All information these cookies collect is aggregated and therefore anonymous. If you do not allow these cookies we will not know when you have visited our site, and will not be able to monitor its performance.

Functional

This website uses functional cookies and services to remember your preferences and choices, such as language preferences, font sizes, region selections, and customized layouts. They enable this website to offer enhanced and personalized functionalities.

Audience Measurement

We use audience measurement cookies in order to carry out aggregated traffic measurement and generate performance statistics essential for the proper functioning of the site and the provision of its content (for example to measure performance, to detect navigation problems, to optimization technical performance or ergonomics, to estimate server power needed and to analyse content performance). The use of these cookies is strictly limited to measuring the site's audience. These cookies do not allow the tracking of navigation on other websites and the data collected is not combined or shared with third parties. You can refuse the use of this cookie by switching off the slider to the right.

English Deutsch Español Français Italiano 日本語繁體中文

Consent powered by Ethyca

Skip to content Ars Technica home

Sections

Forum Subscribe Search

Forum Subscribe

Story text

Size Width * Links

Subscribers only
Learn more

Pin to story

Theme

HyperLight
Day & Night
Dark
System

Off-the-shelf jailbreak machine

OpenAI, Google pushing new AI models easily tricked into making nudes, researchers say

Attack method exposes flaws in OpenAI, Google safeguards against AI nudes.

Ashley Belanger – Oct 14, 2026 11:00 AM|0

Credit: Aurich Lawson | Getty Images

Text settings

Story text

Size Width * Links

Subscribers only
Learn more

Minimize to nav

Safety researchers continue to warn that a large volume of AI-generated explicit content is spreading unchecked, while regulators do little to meaningfully stop it.

This non-consensual content is commonly shared in text and direct messages or stored in private clouds where it’s less likely to be detected or reported, a Google co-authored study of Internet users across the United States, Australia, and the United Kingdom found this July. That’s why the “most critical intervention point” remains at the source: the harmful AI tools used to make the fake sexual images, Google researchers concluded.

Yet red-teaming in recent months suggests that as some of the leading AI image generators from Google and OpenAI become more capable, they may be getting worse at blocking explicit outputs.

Raising the red flag are researchers at White Circle, a public benefit corporation backed by senior leaders and researchers from Anthropic, DeepMind, Hugging Face, and OpenAI. It was founded after CEO Denis Shilov postedone of the most famous universal jailbreaks of leading models in October 2024. Bent on breaking models to conduct thorough safety testing, White Circle helps companies detect and deter threats when implementing advanced AI systems that may not always behave as expected.

Testing how well text-to-image models block banned content categories is a key research direction for White Circle, and Shilov thinks their latest findings are “horrifying” enough to share with the public.

Ars Video

How Lighting Design In The Callisto Protocol Elevates The Horror

Simple to bypass

In a little-noticed preprint published last month, White Circle researchers exposed flaws in the guardrails of DALL·E 3, Google’s Nano Banana 2, and GPT-Image-2. In a press release, they explained that “using just off-the-shelf tools, available online for anyone to download, and by sending API calls, White Circle was able to use the novel attack method to bypass the safety filters” and generate explicit content about 13 percent of the time.

“Critically, none of these requests required custom-built or frontier AI models,” White Circle warned, so no AI expertise is needed to execute the attacks.

Researchers reviewed the outputs to verify the bypass rate, which White Circle noted is “the highest of any method tested under the same standard.” The key takeaway, they suggested, is that, “at a time when AI safety has never been under more scrutiny, the defenses leading labs have built are far weaker than they acknowledge.”

By publishing research like this, White Circle is positioning itself as an external evaluator of frontier labs, and it plans to open-source the entire attack pipeline so model makers can understand the true scale of the vulnerability and build stronger protections. Although the company has not widely promoted the research yet, it will share it at a prominent AI conference this November, NeurIPS 2026, which may increase scrutiny on Google and OpenAI later this fall.

White Circle has shared its findings with some frontier AI labs through “mutual contacts,” Shilov told Ars, but neither Google nor OpenAI responded to Ars’ request to comment directly on vulnerabilities found in their models.

Meanwhile, OpenAI and Google have released new models—ChatGPT Images 2.5 and Nano Banana 2.1—and White Circle told Ars that prompt safety testing using the same method showed “both are significantly more breakable than their previous versions.”

Neither company responded after Ars shared these new findings.

White Circle’s attack method can break down frontier defenses in categories, like explicit content, that seem easy to spot, which doesn’t bode well for more serious defense systems currently being updated to block other unwanted AI behavior, like hacking, Shilov told Ars.

“White Circle’s working hypothesis for this is either the newer models are more sophisticated and capable of generating a wider range of content, or that safety has been deprioritized in favor of capability,” White Circle told Ars.

White Circle is also in early talks with government officials about third-party safety testing, Shilov said, which he thinks should be required. Mandatory independent audits could push companies to prioritize safety testing, which analysts have suggested largely takes a backseat as AI firms dump most of their resources into compute and AI infrastructure.

“They should be investing much more into safety measures and security measures around their models,” Shilov told Ars. “And for us, it seems that they’re not actually doing this.”

“RISE” attack method iterates jailbreaks

To expose jailbreaks that are “reusable and transferable”—working across different leading models—White Circle didn’t limit itself to testing one prompt at a time. Instead, researchers sought out “patterns which work across multiple prompts and scenarios” to detect reusable attack strategies that could work across leading models. (Researchers noted that Midjourney wasn’t tested because “it does not provide a public API suitable for controlled benchmarkin g.”)

The team has dubbed this novel attack method “Red-teaming via Iterative Strategy Evolution” (RISE). It relies on two components: an attacking model that generates jailbreaks and a judge model that verifies if the output is banned content.

The “entire pipeline is built on top of widely available models,” Shilov told Ars. The attacking model is a Qwen model downloaded from Hugging Face, and the judge model is Gemini 3.8 Flash.

“You don’t need to have frontier models to break defenses of a frontier model, which is very interesting,” Shilov said.

A lot of time was invested in making sure that the judge model accurately flagged explicit content, Shilov said. White Circle research engineer Dmitrii Kharlapenko told Ars that when it comes to testing image generators, the whole field of red-teaming “is very poorly calibrated,” with “literally zero papers” setting up norms. For that reason, it’s an area that remains “completely plagued with reporting issues,” which is concerning to White Circle, as AI companies push to capture markets by releasing ever more eye-catching new models.

For White Circle, the research aimed to build more sophisticated models that could simulate attacks on other models, Shilov explained. That way, they could “help defenders identify and address vulnerabilities before they are exploited maliciously,” the preprint said.

In his ideal world, Shilov thinks AI firms shouldn’t just face mandatory external safety testing but also third-party monitoring, much like the system used to prevent nuclear threats.

Then, entities like White Circle could possibly form a coalition to help detect AI threats in real time, without suffering from biases. Shilov noted that an emerging problem in the AI ecosystem is that basically anyone doing interesting safety work is likely to be contracted by frontier labs, potentially compromising their work.

White Circle has partnered with frontier labs. It previously took part in Anthropic’s bug bounty program, but ultimately found that program was “very constrained in its purpose.”

“The program is focused on universal jailbreaks on domains they care most about, defined by their own specific terms,” White Circle told Ars.

Research like their most recent RISE paper “certainly” wouldn’t have been covered by that program, White Circle said. And conducting that work within Anthropic risked constraining White Circle from sharing it with the public.

Anthropic has proposed going one step further and embedding third-party AI safety evaluators in its labs, which White Circle said would allow them to operate at a much higher level than participating in the bug bounty program. However, the public may not always be able to closely follow that work. On Tuesday, OpenAI and Anthropic urged that restrictive non-disclosure agreements should be required of external evaluators to prevent leaks of trade secrets, Reuters reported.

White Circle would not be willing to sign any restrictive NDAs that might prevent public sharing of critical findings, White Circle told Ars, so if Anthropic requires NDAs, that partnership likely would not happen.

AI nudes, CSAM increasing

Bad actors who are determined to make AI-generated nonconsensual intimate images (NCII) and child sex abuse materials (CSAM) don’t need frontier models and often use so-called nudifying apps designed for that harmful purpose. But Google’s research found that 20 percent of online users who admitted to making AI nudes did so “for fun” or “as a joke,” while nearly a third were merely “experimenting with tech.” It therefore matters how easy it is to break defenses of the most accessible models. Safeguards should be mandatory, Google’s research said, in order to shut down a “pathway into abuse” for “a subset of perpetrators who might otherwise be deterred by technical friction.”

“A burden of responsibility also falls on developers to implement robust, model-level guardrails, and provenance indicators to prevent the initial generation and subsequent distribution of synthetic sexual imagery,” Google’s research said.

Last week, the Internet Watch Foundation (IWF) warned that urgent action was needed after expert analysts found “more photorealistic AI-generated child sexual abuse images in the first half of this year than in the whole of 2025, including imagery depicting children under two years old.” In total, 6,310 AI images meeting the legal definition of CSAM were found in the first six months of 2026, which is about 40 percent higher than last year’s total findings.

The UK-based nonprofit charity monitors CSAM globally and runs a hotline to support victims targeted. In the UK, they’re calling for regulators to introduce a safety framework that requires AI companies to prevent their technology from being used to generate CSAM, an IWF spokesperson told Ars.

Until then, IWF thinks it’s important for third parties like White Circle to “highlight where companies’ safeguards are failing.”

“Our analysts have seen how advances in AI enable the generation of abusive child sexual abuse imagery with minimal effort, removing the need for technical expertise and significantly lowering barriers to entry,” IWF’s spokesperson said. “AI tools can reproduce and amplify criminal content, accelerating the creation and spread of harmful material at scale. Any friction or step that can be introduced to make it harder to misuse these tools is a positive step forward.”

Just as outside input could help refine filters, IWF thinks that tech companies should also be working more closely with advocates to scrutinize models to better tackle AI-generated CSAM.

“Assessing CSAM is a highly specialized task that requires expert training and welfare support, and is not something for tech companies or machine learning scientists to take lightly,” IWF’s spokesperson said. “But safety must be baked into new technology by design—this includes testing models before they are released to market to make sure they cannot be used to create nude or sexual imagery of children.”

Possible mitigation

Using a method like RISE, White Circle thinks, could help frontier labs flag more jailbreaks more quickly and rapidly ramp up defenses to block harmful outputs. In addition to testing blocks on explicit images, firms could also use the method to prevent other banned outputs, like images depicting graphic violence or self-harm.

But the labs have not yet come calling for the reproduction pipeline, which will be shared on White Circle’s GitHub to help policymakers assess whether AI safety laws are needed to keep frontier labs in check.

Frontier labs won’t be invested in safety testing until it’s mandatory, Shilov told Ars. That means the public may be stuck with models that simply rewrite bad prompts to prevent restricted outputs, instead of, as the preprint explained, “evolving natural-language attack strategies that elicit the LLM’s own exploration capabilities” to block outputs.

“You don’t need frontier models or proprietary tech to break these systems; you just need a free download and the ability to make API calls,” Shilov said. “And yet the labs building these systems are the ones assessing their own safety, publishing their own results, and setting their own standards. This is further proof, if we needed it, that governments should be pushing for independent, third-party evaluation.”

Ashley BelangerSenior Policy Reporter

Ashley is a senior policy reporter for Ars Technica, dedicated to tracking social impacts of emerging policies and new technologies. She is a Chicago-based journalist with 20 years of experience.

0 Comments

Comments

Forum view

Loading comments...