Search This Blog

Powered by Blogger.

Blog Archive

Labels

Footer About

Footer About

Labels

Showing posts with label GPT. Show all posts

Researchers Uncover Compaction Smuggling Flaw That Exposes OpenAI Reasoning Traces

 


Researchers at Xint identified a vulnerability in the OpenAI API that could expose hidden reasoning traces from advanced AI models, creating a potential route for large-scale model distillation. The technique, named “compaction smuggling,” bypassed safeguards introduced after earlier research demonstrated methods for extracting encrypted reasoning data. OpenAI confirmed the issue and deployed a fix on October 23, 2026. Xint received a $6,000 bounty for the discovery. 

Reasoning models such as GPT-5.6 Sol generate intermediate reasoning before producing answers. OpenAI does not provide API users with the complete raw reasoning in readable form, instead offering optional summaries. This restriction helps protect against distillation, a process in which developers train smaller models using outputs from more advanced systems. Full reasoning traces can provide particularly useful training data because they contain intermediate steps, incorrect approaches and self-corrections. 

The API stores raw reasoning in encrypted items that clients can return during later interactions but cannot directly decode. OpenAI also encrypts conversation compactions, which condense lengthy exchanges into smaller items while preserving relevant context. The discovery follows an August 2026 study titled Stealing Reasoning Traces from Proprietary LLM APIs. The researchers behind that study demonstrated that encrypted reasoning could be replayed into weaker models from the same provider and reproduced as text. 

The approach affected models from OpenAI, Anthropic and Google, although OpenAI’s safeguards made extraction more difficult. Subsequent protections reportedly stopped the original technique from working. Xint discovered that the compaction feature offered another route. The researchers first generated reasoning using GPT-5.6 Sol and retained its encrypted reasoning item. They then removed the original task from the conversation history and instructed the compaction process to preserve the reasoning verbatim in a JSON object. Replacing spaces with tab characters helped the content evade usage-policy classifiers. 

The resulting encrypted compaction was then submitted to GPT-4.1 and GPT-4o, which were instructed to reproduce the contents exactly. After decoding the JSON and converting tabs back into spaces, the researchers recovered the reasoning trace. The technique worked across all tested reasoning-effort settings, from minimal to maximum. In the demonstration, the original reasoning item contained 405 tokens, compared with 612 tokens in the compaction and 609 tokens in each echoed output. Xint compared outputs from two models using different prompts and obtained identical results. 

Although the original plaintext was unavailable for a definitive token-by-token comparison, the recovered trace retained its false starts, corrections and characteristic writing style. The researchers developed the technique while investigating GhostCompact and examining how encrypted compactions could be transferred between models. The findings raise concerns about large-scale extraction of proprietary AI capabilities. 

In September 2026, U.S. agencies warned about industrial-scale model distillation campaigns. Anthropic subsequently reported seven campaigns attributed to China-based laboratories, while OpenAI disclosed disrupting an effort that generated approximately 16,000 requests from more than 4,000 accounts in July. 

Xint concluded that server-side processes handling decrypted reasoning while accepting client-controlled instructions can become potential extraction pathways. The researchers said compaction features require protections comparable to those applied to raw reasoning traces.

OpenAI Temporarily Lifts GPT-5.6 Sol Usage Limits Following Surge in Demand

 


OpenAI has temporarily relaxed usage restrictions for its GPT-5.6 Sol model after experiencing a significant surge in demand over the past 48 hours. The move is aimed at giving users greater access to the company's most advanced AI model while it works on improving its efficiency.

On Sunday, OpenAI announced that it is temporarily removing the five-hour usage limit for users on Plus, Pro, and Business plans. In addition to lifting the restriction, the company has also reset current usage for eligible users, allowing them to continue using GPT-5.6 Sol without being affected by their previous usage during the current cycle.

"The last 48 hours of Codex and ChatGPT Work have been intense," OpenAI product lead Tibo said in a post on X. "[We're] temporarily removing the 5 hour usage limit restriction for all Plus, Business and Pro plans."

Under OpenAI's standard usage policy, Codex and ChatGPT count both local messages and cloud-based tasks against a shared usage allowance. ChatGPT typically operates on a rolling five-hour usage window, while additional weekly usage limits may also apply depending on the user's subscription plan and the AI model being used.

With the temporary removal of the five-hour restriction, eligible users can continue working without being interrupted after reaching the short-term usage cap. However, OpenAI has not stated that all usage limits have been removed, meaning other plan-specific restrictions may still remain in place.

Alongside the temporary policy change, OpenAI is also rolling out updates designed to make GPT-5.6 Sol more efficient. According to the company, these improvements will reduce the amount of usage consumed during tasks, enabling users to complete more work before reaching their available limits.

"[We are] rolling out changes that will make GPT-5.6 Sol more efficient across the board and that will be reflected in less usage being used so that it can take you further," Tibo said.

OpenAI has not disclosed the technical details behind these efficiency improvements. However, the changes are expected to involve lower token consumption, allowing GPT-5.6 Sol to process tasks more efficiently while using fewer resources.

As part of the update, the company has also issued a one-time usage reset for all eligible users. This provides additional capacity for developers and professionals using GPT-5.6 Sol for coding, automation, and agentic workflows. While the temporary removal of the five-hour cap and the usage reset offer more flexibility, OpenAI has not indicated that GPT-5.6 Sol is now available with unlimited access, as broader plan-based usage limits may continue to apply.

OpenAI Limits GPT-5.6 Release While U.S. Reviews AI Safety

 



OpenAI has postponed the extensive public rollout of its latest frontier artificial intelligence model, GPT-5.6, after the U.S. government requested an opportunity to examine the technology before it reaches a wider audience. Rather than making the model immediately available to all users, the company will begin with a restricted deployment involving a small number of carefully vetted partners whose identities have been disclosed to federal authorities.

The temporary decision surfaces an increasingly cautious approach toward highly capable AI systems as governments evaluate their potential impact on national security. Policymakers have become more concerned that advanced generative AI models, while offering substantial benefits across research, software development and cybersecurity, could also be exploited to support sophisticated cyberattacks, automate vulnerability discovery, generate convincing phishing campaigns or assist other malicious activities if deployed without adequate safeguards.

According to OpenAI, the limited rollout is intended to provide government officials with an opportunity to study the model's capabilities and assess possible security risks before broader public access is granted. The company said it has already briefed the U.S. government on GPT-5.6 and its expected capabilities and described the current arrangement as an interim measure while it works with Washington to establish a more structured framework for releasing future frontier AI models.

Chief Executive Officer Sam Altman publicly expressed support for rigorous safety evaluations but questioned whether government agencies should determine which organizations receive early access. In a post on X, Altman said extensive testing of advanced AI systems is appropriate, while arguing that customer selection should remain outside government control.

The latest development follows an executive order signed earlier this month by President Donald Trump establishing a voluntary process under which developers of designated "covered frontier models" may provide the U.S. government with access to their systems for up to 30 days before they are released to trusted external partners. The initiative is designed to give officials time to evaluate emerging security concerns and strengthen oversight of increasingly capable AI technologies before wider deployment.

OpenAI stated that restricting access during this initial period represents what it believes is the most practical route toward making GPT-5.6 more broadly available in the coming weeks while discussions continue with the Administration on implementing the cyber-focused executive order and developing a repeatable review process for future launches.

The company added that engineering teams will continue conducting extensive safety evaluations and work closely with early partners throughout the testing phase. At the same time, OpenAI cautioned that the current level of government access should remain a temporary measure rather than becoming a permanent requirement for future AI releases. It also declined to identify the organizations participating in the initial rollout.

OpenAI further warned that prolonged restrictions on access to frontier AI systems could slow innovation across multiple sectors. The company noted that developers, businesses, cybersecurity professionals and international collaborators all rely on access to advanced models to build defensive security tools, strengthen research, develop enterprise applications and accelerate responsible AI adoption.

Leading the new product family is GPT-5.6 Sol, which OpenAI describes as its most capable model to date. The release also includes Terra, positioned as a mid-range model, and Luna, a lower-cost alternative intended to make advanced AI capabilities available at a lower price point across a wider range of use cases.

The government's heightened scrutiny extends beyond OpenAI. Earlier this month, Anthropic was instructed by U.S. authorities to suspend access to its frontier AI models for foreign nationals because of national security concerns. The company continues to face an ongoing legal and regulatory dispute with the government over those restrictions, illustrating the growing debate surrounding oversight of advanced artificial intelligence systems.

The developments come as both OpenAI and Anthropic have confidentially submitted paperwork for U.S. initial public offerings. Separately, The New York Times reported that OpenAI is considering postponing its public market debut until next year.

The developing relationship between AI developers and governments illustrates how the deployment of frontier models is becoming closely linked with cybersecurity and national security policy. While companies continue to pursue increasingly powerful AI capabilities, regulators are placing greater emphasis on evaluating how these systems could influence cyber defense, critical infrastructure protection and the misuse of AI by malicious actors before they are released at scale.

Ethics and Tech: Data Privacy Concerns Around Generative AI

Ethics and Tech: Data Privacy Concerns Around Generative AI

The tech industry is embracing Generative AI, but the conversation around data privacy has become increasingly important. The recent “State of Ethics and Trust in Technology” report by Deloitte highlights the pressing ethical considerations that accompany the rapid adoption of these technologies. 30% of organizations have adjusted new AI projects, and 25% have modified existing ones in response to the AI Act, the report mentions.

The Rise of Generative AI

54% of professionals believe that generative AI poses the highest ethical risk among emerging technologies. Additionally, 40% of respondents identified data privacy as their top concern. 

Generative AI, which includes technologies like GPT-4, DALL-E, and other advanced machine learning models, has shown immense potential in creating content, automating tasks, and enhancing decision-making processes. 

These technologies can generate human-like text, create realistic images, and even compose music, making them valuable tools across industries such as healthcare, finance, marketing, and entertainment.

However, the capabilities of generative AI also raise significant data privacy concerns. As these models require vast amounts of data to train and improve, the risk of mishandling sensitive information increases. This has led to heightened scrutiny from both regulatory bodies and the public.

Key Data Privacy Concerns

Data Collection and Usage: Generative AI systems often rely on large datasets that may include personal and sensitive information. The collection, storage, and usage of this data must comply with stringent privacy regulations such as GDPR and CCPA. Organizations must ensure that data is anonymized and used ethically to prevent misuse.

Transparency and Accountability: One of the major concerns is the lack of transparency in how generative AI models operate. Users and stakeholders need to understand how their data is being used and the decisions being made by these systems. Establishing clear accountability mechanisms is crucial to build trust and ensure ethical use.

Bias and Discrimination: Generative AI models can inadvertently perpetuate biases present in the training data. This can lead to discriminatory outcomes, particularly in sensitive areas like hiring, lending, and law enforcement. Addressing these biases requires continuous monitoring and updating of the models to ensure fairness and equity.

Security Risks: The integration of generative AI into various systems can introduce new security vulnerabilities. Cyberattacks targeting AI systems can lead to data breaches, exposing sensitive information. Robust security measures and regular audits are essential to safeguard against such threats.

Ethical Considerations and Trust

80% of respondents are required to complete mandatory technology ethics training, marking a 7% increase since 2022.  Nearly three-quarters of IT and business professionals rank data privacy among their top three ethical concerns related to generative AI:

  • Developing and implementing ethical frameworks for AI usage is crucial. These frameworks should outline principles for data privacy, transparency, and accountability, guiding organizations in the responsible deployment of generative AI.
  • Engaging with stakeholders, including employees, customers, and regulatory bodies, is essential to build trust. Open dialogues about the benefits and risks of generative AI can help in addressing concerns and fostering a culture of transparency.
  • The dynamic nature of AI technologies necessitates continuous monitoring and improvement. Regular assessments of AI systems for biases, security vulnerabilities, and compliance with privacy regulations are vital to ensure ethical use.

Custom GPTs Might Coarse Users into Giving up Their Data


In a recent study by Northwestern University, researchers uncovered a startling vulnerability in customized Generative Pre-trained Transformers (GPTs). While these GPTs can be tailored for a wide range of applications, they are also vulnerable to rapid injection attacks, which can divulge confidential data.

GPTs are advanced AI chatbots that can be customized by OpenAI’s ChatGPT users. They utilize the Large Language Model (LLM) at the heart of ChatGPT, GPT-4 Turbo, but are augmented with more, special components that impact their user interface, such as customized datasets, prompts, and processing instructions, enabling them to perform a variety of specialized tasks.

However, the parameters and sensitive data that a user might use to customize the GPT could be left vulnerable to a third party. 

For instance, Decrypt used a simple prompt hacking technique—asking for the "initial prompt" of a custom, publicly shared GPT— to access the entire prompt and confidential data of a custom.

In their study, the researchers tested over 200 custom GPTs wherein the high risk of such attacks was revealed. These jailbreaks might also result in the extraction of initial prompts and unauthorized access to uploaded files.

The researchers further highlighted the risks of these assaults since they jeopardize both user privacy and the integrity of intellectual property. 

“The study revealed that for file leakage, the act of asking for GPT’s instructions could lead to file disclosure,” the researchers found. 

Moreover, the researchers revealed that attackers can cause two types of disclosures: “system prompt extraction” and “file leakage.” While the first tricks the model into sharing basic configuration and prompts, the second coerces the model into revealing its confidential training datasets. 

The researchers further note that the existing defences, like defensive prompts, prove insufficient in front of the sophisticated adversarial prompts. The team said that this will require a more ‘robust and comprehensive approach’ to protect the new AI models. 

“Attackers with sufficient determination and creativity are very likely to find and exploit vulnerabilities, suggesting that current defensive strategies may be insufficient,” the report further read.  "To address these issues, additional safeguards, beyond the scope of simple defensive prompts, are required to bolster the security of custom GPTs against such exploitation techniques." The study prompted the broader AI community to opt for more robust security measures.

Although there is much potential for customization of GPTs, this study is an important reminder of the security risks involved. AI developments must not jeopardize user privacy and security. For now, it is advisable for users to keep the most important or sensitive GPTs to themselves, or at least not train them with their sensitive data.