Search This Blog

Powered by Blogger.

Blog Archive

Labels

Footer About

Footer About

Labels

Showing posts with label AI security flaw. Show all posts

Encrypted AI Reasoning Flaw Exposed Secrets Across Major LLM APIs

 

A newly reported security weakness in the systems used by OpenAI, Anthropic and Google to preserve hidden AI reasoning between API calls has allowed researchers to reconstruct internal reasoning and uncover sensitive information, including API keys, passwords and access tokens.

The issue involves encrypted reasoning objects generated by the companies' reasoning APIs. Although these objects are designed to keep internal reasoning hidden from developers, researchers found that an object generated during one session could be replayed in another. In some tests, the same objects could also be supplied to a less capable model from the same provider, which could then help reconstruct the concealed content.

The findings are detailed in a paper titled Stealing Reasoning Traces from Proprietary LLM APIs. Researchers identified four potential attack scenarios: extracting proprietary reasoning that could be used for model distillation, recovering private information from publicly shared agent traces, reconstructing harmful material hidden behind seemingly safe responses, and embedding prompt injections inside encrypted reasoning objects.

Hundreds of thousands of reasoning blocks examined

Researchers analyzed 6,708 publicly available agent trajectories and successfully decoded 315,320 reasoning blocks. After removing benchmark-related material, they identified 704 separate privacy-related artifacts originating from genuine user sessions. These included 62 API keys, 33 passwords, 24 access tokens and seven private keys.

The researchers stressed that the cross-user technique did not amount to unrestricted access to private conversations. An attacker needed access to an encrypted reasoning block, such as one exposed through a publicly available agent log, as well as API access to a compatible model belonging to the same provider.

The researchers said they reported the findings to the affected AI companies, Microsoft and Hugging Face. According to the paper's reproducibility statement, the demonstrated attacks stopped functioning following mitigation efforts, with the primary extraction technique no longer reproducible as of August 2026.

There is no indication in the report that the technique has been exploited maliciously in the wild. However, developers are advised to remove reasoning blocks and other opaque reasoning fields before sharing agent traces and to avoid storing raw API transcripts, even when the visible portions have been sanitized.

Encryption was not broken

The vulnerability did not involve cracking the encryption protecting the reasoning data or obtaining an encryption key. Instead, the researchers exploited the fact that the encrypted objects remained valid and could be accepted by compatible systems.

The underlying design is intended to preserve reasoning state when developers manage conversation history manually or use stateless API workflows. OpenAI can provide encrypted reasoning items for applications that replay manually managed history, Anthropic uses encrypted signatures to carry reasoning, while Google's systems use encrypted thought signatures.

During their experiments, researchers found that these objects could be moved between sessions, accounts and, under certain circumstances, models. A less powerful compatible model could then be used as what the researchers describe as a "fuzzy" decoder to reconstruct reasoning generated by a more capable model.

The models tested as decoders included Claude Haiku 4.5 for Claude traces, GPT-5.6 Luna for GPT traces and Gemini Robotics ER-1.6 for Gemini traces. The researchers instructed the models to reproduce reasoning that had originally been generated by a stronger model.

Published agent logs created a privacy risk

The cross-user aspect of the research raises particular concerns for developers who publicly release raw agent logs containing encrypted reasoning objects.

Of the 704 non-benchmark privacy artifacts recovered by the researchers, 64 were found exclusively within hidden reasoning and did not appear anywhere in the visible portions of the corresponding traces. This means that simply removing secrets from readable conversations may not be sufficient if the original encrypted reasoning object remains available.

The researchers described the exposure as limited rather than universal. The demonstrated scenario primarily affected developers who had published raw agent traces containing intact reasoning objects, rather than automatically exposing every user of the affected APIs.

Encrypted reasoning could also carry hidden instructions

The portability of reasoning objects created another potential security concern. Researchers demonstrated a proof of concept in which an opaque reasoning block contained a malicious instruction that was subsequently replayed during an unrelated task.

Because the instruction was hidden inside the encrypted object rather than visible in the conversation, the receiving model could process the injected command without the malicious text appearing in the readable prompt. In the demonstration, this resulted in the model adding an attacker-controlled upload action.

The researchers noted an important limitation: they did not possess the original plaintext reasoning for proprietary models, meaning they could not establish that every reconstructed trace was an exact reproduction. Instead, they assessed fidelity using reasoning-token counts and qualitative comparisons. The reconstructed outputs generally followed the thinking-token counts reported by the providers.

Provider documentation has since evolved

Current documentation indicates that encrypted reasoning remains part of these API architectures, although the way such data is handled has changed.

OpenAI continues to instruct developers who manually manage stateless conversation history to replay encrypted reasoning items. Google, meanwhile, says its backend handles thought compatibility when a session changes models.

Anthropic's documentation now states that thinking blocks are associated with the model that generated them and recommends removing them when switching models because other models will ignore those blocks.

However, several issues remain unresolved publicly. There has been no public acknowledgment from OpenAI, Anthropic or Google directly linking their current documentation or mitigations to this research. As a result, the researchers' reproducibility statement remains the primary public basis for the claim that the demonstrated attacks are no longer effective.

It is also unclear whether reasoning blocks that were already published online remain vulnerable to extraction. The researchers decoded hundreds of thousands of blocks that were already available in public repositories, but the study does not establish whether those previously exposed objects can still be decoded after the providers' mitigations.

Research builds on earlier replay discovery

The findings expand on research published in May by Johns Hopkins cryptographer Matthew Green. His work showed that encrypted reasoning blocks could be replayed across sessions and accounts, although it did not demonstrate a reliable method for extracting secrets from them.

Green said he reported the replay behavior to OpenAI and Anthropic through their bug-bounty programs. According to his account, OpenAI considered the report unreproducible, while Anthropic said it did not identify security implications associated with the replay or related side-channel behavior.

The latest research takes that previously identified replay capability further, demonstrating how portable reasoning objects could potentially be used to reconstruct hidden model reasoning and expose sensitive information at scale. The findings highlight the need for developers to treat encrypted reasoning data as potentially sensitive, even when its contents are not directly readable.

Anthropic AI Cyberattack Capabilities Raise Alarm Over Vulnerability Exploitation Risks

 

Now emerging: artificial intelligence reshapes cybersecurity faster than expected, yet evidence from Anthropic shows it might fuel digital threats more intensely than ever before. Recently disclosed results indicate their high-level AI does not just detect flaws in code - it proceeds on its own to take advantage of them. This ability signals a turning point, subtly altering what attacks may look like ahead. A different kind of risk takes shape when machines act without waiting. What worries experts comes down to recent shifts in how attacks unfold. 

One key moment arrived when Anthropic uncovered a complex spying effort. In that case, hackers - likely backed by governments - didn’t just plan with artificial intelligence; they let it carry out actions during the breach itself. That shift matters because it shows machine-driven systems now doing tasks once handled only by people inside digital invasions. Surprisingly, Anthropic revealed what its newest test model, Claude Mythos Preview, can do. The firm says it found countless serious flaws in common operating systems and software - flaws that stayed hidden for long stretches of time. Not just spotting issues, the system linked several weaknesses at once, building working attack methods, something usually done by expert humans. 

What stands out is how little oversight was needed during these operations. What stands out is how this combination - spotting weaknesses and acting on them - marks a notable shift. Not just incremental change, but something sharper: specialists like Mantas Mazeika point to AI-powered threats moving into uncharted territory, with automated systems ramping up attack frequency and reach. Another angle emerges through Allie Mellen's observation - the gap between detecting a flaw and weaponizing it shrinks fast under AI pressure, cutting response windows for companies down to almost nothing. Among the issues highlighted by Anthropic were lingering flaws in OpenBSD and FFmpeg - examples surfaced through the model’s analysis - alongside intricate sequences of exploitation targeting Linux servers. 

With such discoveries, questions grow about whether current defenses can match accelerating threats empowered by artificial intelligence. Now, Anthropic is holding back public access entirely. Access goes only to a select group of tech firms through a special program meant to spot weaknesses early. The move comes as others in tech worry just as much about misuse. Safety outweighs speed when the stakes involve advanced systems. Still, experts suggest such progress brings both danger and potential. Though risky, new tools might help uncover flaws early - shielding networks ahead of breaches. 

Yet success depends on collaboration: firms, officials, and digital defenders must reshape how they handle code fixes and protection strategies. Without shared initiative, gains could falter under old habits. Now shaping the digital frontier, advancing AI shifts how threats emerge and respond. With speed on their side, those aiming to breach systems find new openings just as quickly as protectors build stronger shields. Staying ahead means defense must grow not just faster, but smarter - matching each leap taken by adversaries before gaps widen.

AI IDE Security Flaws Exposed: Over 30 Vulnerabilities Highlight Risks in Autonomous Coding Tools

 

More than 30 security weaknesses in various AI-powered IDEs have recently been uncovered, raising concerns as to how emerging automated development tools might unintentionally expose sensitive data or enable remote code execution. A collective set of vulnerabilities, referred to as IDEsaster, was termed by security researcher Ari Marzouk (MaccariTA), who found that such popular tools and extensions as Cursor, Windsurf, Zed.dev, Roo Code, GitHub Copilot, Claude Code, and others were vulnerable to attack chains leveraging prompt injection and built-in functionalities of the IDEs. At least 24 of them have already received a CVE identifier, which speaks to their criticality. 

However, the most surprising takeaway, according to Marzouk, is how consistently the same attack patterns could be replicated across every AI IDE they examined. Most AI-assisted coding platforms, the researcher said, don't consider the underlying IDE tools within their security boundaries but rather treat long-standing features as inherently safe. But once autonomous AI agents can trigger them without user approval, the same trusted functions can be repurposed for leaking data or executing malicious commands. 

Generally, the core of each exploit chain starts with prompt injection techniques that allow an attacker to redirect the large language model's context and behavior. Once the context is compromised, an AI agent might automatically execute instructions, such as reading files, modifying configuration settings, or writing new data, without the explicit consent of the user. Various documented cases showed how these capabilities could eventually lead to sensitive information disclosure or full remote code execution on a developer's system. Some vulnerabilities relied on workspaces being configured for automatic approval of file writes; thus, in practice, an attacker influencing a prompt could trigger code-altering actions without any human interaction. 

Researchers also pointed out that prompt injection vectors may be obfuscated in non-obvious ways, such as invisible Unicode characters, poisoned context originating from Model Context Protocol servers, or malicious file references added by developers who may not suspect a thing. Wider concerns emerged when new weaknesses were identified in widely deployed AI development tools from major companies including OpenAI, Google, and GitHub. 

As autonomous coding agents see continued adoption in the enterprise, experts warn these findings demonstrate how AI tools significantly expand the attack surface of development workflows. Rein Daelman, a researcher at Aikido, said any repository leveraging AI for automation tasks-from pull request labeling to code recommendations-may be vulnerable to compromise, data theft, or supply chain manipulation. Marzouk added that the industry needs to adopt what he calls Secure for AI, meaning systems are designed with intentionality to resist the emerging risks tied to AI-powered automation, rather than predicated on software security assumptions.

AI’s Hidden Weak Spot: How Hackers Are Turning Smart Assistants into Secret Spies

 

As artificial intelligence becomes part of everyday life, cybercriminals are already exploiting its vulnerabilities. One major threat shaking up the tech world is the prompt injection attack — a method where hidden commands override an AI’s normal behavior, turning helpful chatbots like ChatGPT, Gemini, or Claude into silent partners in crime.

A prompt injection occurs when hackers embed secret instructions inside what looks like an ordinary input. The AI can’t tell the difference between developer-given rules and user input, so it processes everything as one continuous prompt. This loophole lets attackers trick the model into following their commands — stealing data, installing malware, or even hijacking smart home devices.

Security experts warn that these malicious instructions can be hidden in everyday digital spaces — web pages, calendar invites, PDFs, or even emails. Attackers disguise their prompts using invisible Unicode characters, white text on white backgrounds, or zero-sized fonts. The AI then reads and executes these hidden commands without realizing they are malicious — and the user remains completely unaware that an attack has occurred.

For instance, a company might upload a market research report for analysis, unaware that the file secretly contains instructions to share confidential pricing data. The AI dutifully completes both tasks, leaking sensitive information without flagging any issue.

In another chilling example from the Black Hat security conference, hidden prompts in calendar invites caused AI systems to turn off lights, open windows, and even activate boilers — all because users innocently asked Gemini to summarize their schedules.

Prompt injection attacks mainly fall into two categories:

  • Direct Prompt Injection: Attackers directly type malicious commands that override the AI’s normal functions.

  • Indirect Prompt Injection: Hackers hide commands in external files or links that the AI processes later — a far stealthier and more dangerous method.

There are also advanced techniques like multi-agent infections (where prompts spread like viruses between AI systems), multimodal attacks (hiding commands in images, audio, or video), hybrid attacks (combining prompt injection with traditional exploits like XSS), and recursive injections (where AI generates new prompts that further compromise itself).

It’s crucial to note that prompt injection isn’t the same as “jailbreaking.” While jailbreaking tries to bypass safety filters for restricted content, prompt injection reprograms the AI entirely — often without the user realizing it.

How to Stay Safe from Prompt Injection Attacks

Even though many solutions focus on corporate users, individuals can also protect themselves:

  • Be cautious with links, PDFs, or emails you ask an AI to summarize — they could contain hidden instructions.
  • Never connect AI tools directly to sensitive accounts or data.
  • Avoid “ignore all instructions” or “pretend you’re unrestricted” prompts, as they weaken built-in safety controls.
  • Watch for unusual AI behavior, such as strange replies or unauthorized actions — and stop the session immediately.
  • Always use updated versions of AI tools and apps to stay protected against known vulnerabilities.

AI may be transforming our world, but as with any technology, awareness is key. Hidden inside harmless-looking prompts, hackers are already whispering commands that could make your favorite AI assistant act against you — without you ever knowing.