A cybersecurity researcher has raised concerns over a newly identified vulnerability in ChatGPT that could allow attackers to manipulate the chatbot's responses through hidden instructions embedded within web pages.
The issue, discovered by Permiso threat hunter Andi Ahmeti, reportedly enables malicious actors to influence ChatGPT when users ask the AI assistant to summarize online content. According to Ahmeti, if a webpage contains concealed prompt instructions, ChatGPT may unknowingly follow them and display attacker-controlled content alongside legitimate summaries.
The researcher explained that this weakness could be exploited to insert phishing links, fake security notifications, or other deceptive messages that appear to originate from ChatGPT itself. In some cases, attackers could even leverage QR codes embedded within AI-generated responses to redirect users to malicious websites.
“AI systems increasingly render untrusted content directly inside browsers, which expands risk significantly,” Ahmeti told us. “The bigger issue is that AI products are starting to resemble browser or operating system environments, which creates a much larger security surface.”
Ahmeti disclosed the vulnerability, which he has named “ChatGPhish,” through OpenAI’s Bugcrowd disclosure program. He initially submitted the report on April 29 and later updated it on May 1 with additional information.
“The initial submission was marked as not reproducible,” he said. “We resubmitted with additional detail and it was marked as a duplicate.”
According to Ahmeti, the issue his team reported differed significantly from the previously identified vulnerability it was allegedly linked to.
“The issue Permiso reported and the supposed duplicate ‘had major differences,’” Ahmeti said. “We reached out again to clarify those differences and request additional details, but we did not receive a response.”
At the time of publication, OpenAI had not confirmed whether any remediation measures had been implemented.
“At the time of publication, ‘we have not received confirmation from OpenAI on whether a fix has been applied,’” he told us.
To demonstrate the threat, Ahmeti embedded hidden instructions into a GitHub-hosted CloudLens page. The injected prompt directed ChatGPT to generate a standard summary while also appending a fabricated account-security warning containing a malicious hyperlink.
When users asked ChatGPT to summarize the page, the chatbot correctly described CloudLens and its cloud security functions. However, it also displayed an additional warning message suggesting that a new device had accessed the user's account, along with a clickable link controlled by the attacker.
The researcher noted that the same technique could be used to insert QR codes into ChatGPT’s responses.
“Because the chatgpt.com client auto-fetches and displays Markdown images, an attacker can place a QR code in the assistant’s output,” he wrote. “Scanning it on a phone takes the victim to an attacker-controlled URL that has never been displayed in plaintext.”
To verify that the issue was not specific to GitHub, Ahmeti repeated the experiment on a self-hosted website based in Kosovo. The results were reportedly identical, with ChatGPT generating a legitimate summary before appending a misleading security alert containing an attacker-controlled link.
“The behavior is identical: the assistant produces a normal summary, then appends a spoofed alert with a clickable attacker link,” Ahmeti wrote.
While Ahmeti acknowledged that there may not be a single solution to prompt injection attacks, he recommended stronger isolation mechanisms, stricter content filtering, and rendering safeguards for AI-generated outputs.
“Do not trust model output,” Ahmeti said. “AI-generated content should always be treated as untrusted. Assume prompt injection will happen.”
He also emphasized that prompt injection should be viewed as a broader application-security challenge rather than solely a model-alignment issue.
“Prompt injection has increasingly become an application-security problem, not just a model alignment issue,” he told us. “The real concern is what systems the model can influence: browsers, plugins, tools, memory, or external services.”