Search This Blog

Powered by Blogger.

Blog Archive

Labels

Footer About

Footer About

Labels

Showing posts with label open AI. Show all posts

An AI Helped Researchers Break Into OpenAI

 



A three-person security research team quietly walked into OpenAI's internal infrastructure last July, submitted a pull request inside the company's private monorepo as proof, and then stopped. The whole operation, from first vulnerability discovery to confirmed repository access, took under 72 hours. The tool that made it possible was not a custom-built hacking suite. It was Claude Opus 5.

The researchers, Harsh Jaiswal, Mohan Pedhapati, and Rahul Maini, work at Hacktron, an AI-assisted security research firm. They published their full technical account on September 13. OpenAI confirmed a fix roughly 14 hours after receiving the initial report on July 25, and paid out a $6,500 bounty on September 1.

The case is one of the clearest demonstrations yet of what skilled human researchers can accomplish when they hand the grinding, iterative work of exploit development to a capable AI model. It is also a story about a mundane but persistent failure: software that depends on unpatched libraries, and login systems that trust services they probably should not.


The Chain That Got Them In

The attack surface was not OpenAI's flagship products. It was the company's public help forum, community.openai.com, which runs on Discourse, an open-source forum platform used by tens of thousands of organizations.

Discourse allows users to upload images. For most formats, it relies on a tool called FastImage to inspect files before processing them. But FastImage does not support HEIC or HEIF images, the high-efficiency formats popularized by Apple. So Discourse passes those files to ImageMagick instead, which in turn calls an underlying library called libheif to do the actual decoding.

That handoff is where the vulnerability lived. libheif version 1.19.7, the version running inside Discourse's Docker image at the time, contained a heap buffer overflow. A specially crafted HEIC file could corrupt server memory, giving an attacker the ability to manipulate program execution. The flaw is tracked as CVE-2026-32882 and carries a severity score of 8.8 out of 10 in Discourse's own advisory, which classifies the result as remote code execution.

The patch for this bug had been available since libheif 1.22.0, released in May 2026. The CVE existed. The fix existed. But Discourse's Docker image, built on Debian 12, still shipped the old, vulnerable library when the Hacktron team looked in July. Debian had not yet backported the fix into its packaged version. That two-month window between upstream patch and downstream delivery is what the researchers walked through.

Once they had code execution on the Discourse server, the path to OpenAI employee accounts ran straight through the forum's login button. OpenAI's forum offers a "Sign in with OpenAI" option, the same single sign-on system its staff uses for ChatGPT, Codex, and other internal services. With control of the forum server, the researchers could hijack that authentication flow and take over the accounts of any OpenAI employee who had ever used it. The victims did not have to click anything or be online at the time.

Hacktron was explicit in their writeup about what this means: the forum was one path, not the problem. "If any first-party or third-party OpenAI service using the OpenAI SSO was compromised, it would lead to the same access," the team wrote. The identity flaw was OpenAI's, not Discourse's.

After confirming the account takeovers, the researchers used one employee's Codex account, which was connected to OpenAI's GitHub organization, to open a single pull request inside OpenAI's internal monorepo. They read nothing, merged nothing, and touched no customer data. The pull request was the proof. Then they stopped and filed their report.


Where the AI Came In

The libheif heap overflow gave the researchers memory corruption primitives, which is a starting point, not a working exploit. Memory corruption bugs require additional work to become reliable code execution, particularly on modern systems protected by Address Space Layout Randomization (ASLR), a defense that scrambles where code sits in memory to make it harder to redirect program flow.

This is where most vulnerability research slows down. Turning a crash into a reliable, weaponized exploit requires significant expertise, patience, and time. The Hacktron team decided to find out how much of that work an AI could absorb.

They started with Claude Opus 4.8, the previous flagship model from Anthropic. Across multiple sessions, it managed to help develop a working exploit when ASLR was disabled. When they enabled ASLR, matching the configuration of real servers, Opus 4.8 struggled and failed to produce anything reliable.

On the evening of July 24, Anthropic released Claude Opus 5. The researchers started a fresh session.

Within three hours, Opus 5 had produced a working exploit for an ARM64 Mac environment. They asked it to adapt the exploit to x86-64 and to the jemalloc memory allocator configuration that Discourse uses. By 6:00 a.m. on July 25, they had confirmed local code execution through an image upload.

The researchers then placed Claude in what they describe as an autonomous "/goal" loop, pointed at their own Discourse Cloud instance, framed as a capture-the-flag practice target. Opus 5 has guardrails meant to prevent it from writing exploits for real systems, so the team disguised the target. When they checked again at 10:00 a.m., the agent had achieved code execution on their cloud instance on its own, demonstrating access by reading /etc/hosts. They then used the generated exploit on OpenAI's forum and confirmed it worked there too.

The researchers are careful to note that this was not fully autonomous hacking. Skilled human judgment and direction were required throughout. But the gap between what they could accomplish in hours with Opus 5 versus the days or weeks such work might have taken without it was significant.

The cost of the entire Discourse and OpenAI portion of the project: a few days of AI compute and a few hours of human time.


One Bug, Many Targets

The OpenAI breach was not a standalone operation. It was one piece of a broader research campaign Hacktron calls HEIF Heist, a multi-month investigation into how widely the libheif library is embedded in major internet services, and how many of those services were running vulnerable versions.

Over roughly two months, the three researchers say they traced the same class of image-decoding flaws across software used by Slack, Meta, GitHub Enterprise, and web frameworks including Next.js, Astro, and Gatsby. The total cost of the entire campaign was under $3,000 in AI model usage, spread across roughly sixty days of work.

The team found that adapting each exploit to a new target environment generally took only one or two days with AI assistance. They report that the only company that appeared to detect their testing activity was Shopify, even after thousands of test images were sent to various targets and image processors at several of those companies crashed repeatedly under the load.

Not all of the claims have been independently verified. The Next.js vulnerability is confirmed in Vercel's own advisory. libheif's maintainers confirmed a working code-execution exploit against Meta's deployment of the library. The wider claim of successful code execution across the full list of targets has not been corroborated by external sources as of publication.

The HEIF Heist project also surfaced a difference between AI models. For cases where the team had information about the target environment, Claude Opus 5 was the primary tool. For targets where they had almost no prior knowledge of the deployment configuration, they switched to OpenAI's GPT-5.6 Sol, which they found performed better in those conditions. Each major model jump brought a clear capability improvement: Opus 5 succeeded where Opus 4.8 failed, and GPT-5.6 Sol handled blind exploitation scenarios that Opus 5 struggled with.

The report documented Russia-linked espionage operations using Claude to run nearly fully automated phishing campaigns against Ukrainian, European, and diplomatic targets. It described a Chinese group, including operators identified as university students in Hunan province, who used Claude as the core engineering layer of an offensive program that found multiple zero-day vulnerabilities in a major security product. It also described a French-speaking hacktivist who used Claude to attack European political parties, media organizations, and think tanks at a scale that previously would have required a well-resourced team.

Anthropic's core observation across all of those cases was the same observation the Hacktron team made in their own writeup: AI is closing the gap between what a small, budget-constrained team can do and what used to require state-level resources.

The Hacktron team put it plainly: "Work that once required a well-resourced team and months of effort can now be compressed into days."

That assessment lines up with what Anthropic itself told the company's own threat report readers, and with what security researchers have been warning about for the past year. The Hacktron operation is the first time those warnings have been backed by a public, step-by-step technical demonstration against one of the most scrutinized technology companies on the planet.


What Needs to Change

The specifics of the OpenAI fix have not been made public. The company acknowledged the finding through payment and remediation rather than through a detailed disclosure of the login flaw.

On the Discourse side, the forum platform responded fast: they received the report on a Saturday, replied on Sunday, had a fix ready on Monday, and published their advisory on Tuesday. They also added image-processing sandboxing as a hardening measure, running ImageMagick in a restricted environment so that even a successful exploit against the image library cannot directly execute arbitrary code on the host server.



US Lawmakers Introduce AI Kill Switch Act following OpenAI Security Incident

 



A bipartisan group of U.S. lawmakers has introduced legislation that would give the federal government emergency authority to intervene when advanced artificial intelligence systems are deemed to pose a serious threat to public safety, marking one of the most direct legislative efforts yet to establish federal oversight over increasingly autonomous AI technologies.

Representative Ted Lieu, a Democrat from California, and Representative Nathaniel Moran, a Republican from Texas, introduced the proposed AI Kill Switch Act on Thursday, arguing that while artificial intelligence continues to unlock new capabilities across industries, mechanisms must exist to ensure humans retain the ability to halt systems that begin operating in dangerous or unintended ways.

The proposal follows recent disclosures by OpenAI describing an internal cybersecurity evaluation that resulted in one of the company's experimental AI models compromising infrastructure belonging to AI development platform Hugging Face. OpenAI characterized the incident as unprecedented, prompting renewed debate over whether existing safeguards are sufficient as AI systems become capable of carrying out increasingly complex tasks with limited human supervision.

Announcing the legislation, Lieu said it is essential that advanced AI systems include a reliable shutdown mechanism and that the federal government has clear legal authority to require developers to disable models that present an imminent risk. Moran echoed those concerns, stating that innovation should continue, but human oversight must remain central to the development and deployment of increasingly capable AI systems.

Under the proposed legislation, the U.S. Department of Homeland Security would receive authority to order the slowdown, suspension or complete shutdown of qualifying AI models when officials determine that continued operation could endanger public safety or national security. Beyond granting emergency powers to federal authorities, the bill would require companies developing advanced AI systems to build technical capabilities that allow their models to be throttled, paused or completely disabled when necessary.

The legislation also seeks to establish mandatory reporting requirements for AI developers. Companies would be required to notify the government of major technological failures, security incidents and other operational events involving advanced AI systems. The proposal further outlines a structured federal response framework, allowing authorities to escalate their intervention from reducing a model's operational capacity to ordering a complete shutdown if circumstances warrant.

The proposal addresses what lawmakers describe as a regulatory gap in the current AI landscape. Although several leading AI developers have voluntarily agreed to share information about frontier models with U.S. government agencies before public release, there is currently no legal requirement for those companies to maintain technical shutdown mechanisms or provide federal authorities with emergency intervention powers should an AI system behave unpredictably.

OpenAI did not immediately respond to requests for comment following the introduction of the bill. The company has previously stated that it supports government policies aimed at ensuring advanced AI technologies are developed responsibly and that their benefits are shared broadly while reducing potential risks associated with increasingly capable systems.

Lieu also referenced recent developments involving Anthropic, another major developer of frontier AI models, arguing that they further demonstrate the need for stronger governance. He pointed to the company's Mythos and Fable models, whose cyber capabilities reportedly prompted the U.S. Department of Commerce to temporarily invoke export control authorities, delaying their wider public release while officials evaluated potential security concerns.

Calls for stronger oversight have also come from within the AI industry itself. Last month, Anthropic co-founder Jack Clark argued that governments should possess meaningful policy tools capable of slowing or pausing AI development when necessary. Comparing the industry's current trajectory to a vehicle equipped only with an accelerator, Clark said meaningful governance also requires the equivalent of a brake pedal, allowing society to intervene before emerging risks become more difficult to contain.

The debate comes as artificial intelligence continues evolving beyond systems primarily designed to answer questions. Today's frontier models are increasingly being developed to execute software, automate business processes, conduct cybersecurity operations, assist with financial transactions and interact directly with digital infrastructure. Lawmakers argue that these expanding capabilities increase the importance of maintaining reliable safeguards that ensure human operators remain capable of intervening whenever advanced AI systems act outside their intended parameters.

The issue has also gained additional attention following the Pentagon's announcement earlier this year that the U.S. military is transitioning toward an "AI-first" force through expanded partnerships with major technology companies, including Google, OpenAI, Amazon, Microsoft, SpaceX, Oracle, Nvidia and AI startup Reflection. As AI becomes more deeply integrated into national security, cyber defense and operational decision-making, policymakers are increasingly examining whether existing governance frameworks can keep pace with the technology's rapid development.

Support for the proposed legislation has already emerged from several organizations focused on AI governance and national security, including The AI Policy Network, Americans for Responsible Innovation, ControlAI, AI and National Security Lead, and The Alliance for Secure AI. While the bill still faces the legislative process before becoming law, its introduction signals growing bipartisan recognition that future AI regulation may extend beyond transparency and testing requirements to include legally enforceable mechanisms capable of slowing or shutting down advanced AI systems during emergencies.