GPTs are advanced AI chatbots that can be customized by OpenAI’s ChatGPT users. They utilize the Large Language Model (LLM) at the heart of ChatGPT, GPT-4 Turbo, but are augmented with more, special components that impact their user interface, such as customized datasets, prompts, and processing instructions, enabling them to perform a variety of specialized tasks.

However, the parameters and sensitive data that a user might use to customize the GPT could be left vulnerable to a third party. 

For instance, Decrypt used a simple prompt hacking technique—asking for the "initial prompt" of a custom, publicly shared GPT— to access the entire prompt and confidential data of a custom.

In their study, the researchers tested over 200 custom GPTs wherein the high risk of such attacks was revealed. These jailbreaks might also result in the extraction of initial prompts and unauthorized access to uploaded files.

The researchers further highlighted the risks of these assaults since they jeopardize both user privacy and the integrity of intellectual property. 

“The study revealed that for file leakage, the act of asking for GPT’s instructions could lead to file disclosure,” the researchers found. 

Moreover, the researchers revealed that attackers can cause two types of disclosures: “system prompt extraction” and “file leakage.” While the first tricks the model into sharing basic configuration and prompts, the second coerces the model into revealing its confidential training datasets. 

The researchers further note that the existing defences, like defensive prompts, prove insufficient in front of the sophisticated adversarial prompts. The team said that this will require a more ‘robust and comprehensive approach’ to protect the new AI models. 

“Attackers with sufficient determination and creativity are very likely to find and exploit vulnerabilities, suggesting that current defensive strategies may be insufficient,” the report further read.  "To address these issues, additional safeguards, beyond the scope of simple defensive prompts, are required to bolster the security of custom GPTs against such exploitation techniques." The study prompted the broader AI community to opt for more robust security measures.

Although there is much potential for customization of GPTs, this study is an important reminder of the security risks involved. AI developments must not jeopardize user privacy and security. For now, it is advisable for users to keep the most important or sensitive GPTs to themselves, or at least not train them with their sensitive data.