Search This Blog
Powered by Blogger.

Blog Archive

Labels

Footer About

Footer About

Labels

Load More
Trendy Right Now

Popular Posts

Showing posts with label AI prompt injection attack. Show all posts
RovoBlast
AI prompt injection attack Atlassian Confluence Cyber Attacks Data Exfiltration Jira server breach RovoBlast

Atlassian Rovo AI Can Be Abused to Exfiltrate Jira, Confluence Data

 




Atlassian's Rovo AI assistant has been exposed to two independent attack techniques that could cause it to retrieve Jira and Confluence data accessible to an authenticated user and transmit the information to an attacker-controlled server.

AI security firm PromptArmor and Varonis Threat Labs identified the techniques through different attack paths. Varonis' RovoBlast vulnerability has been fixed by Atlassian, while PromptArmor said its separate content-based attack remained exploitable when it published its findings on August 5, 2026.

Neither finding demonstrated a direct bypass of Jira or Confluence permissions. Instead, both attacks abused the legitimate access available to a victim's Rovo session.


Malicious content can manipulate Rovo

PromptArmor demonstrated an indirect prompt-injection attack in which attacker-controlled instructions were embedded inside content processed by Rovo.

In its example, a user uploaded a malicious document and asked Rovo to organize Jira tickets. The concealed instructions directed the assistant to search Jira and Confluence, collect information available to the user and place the results into an attacker-controlled URL request.

The attacker could then recover the stolen ticket and page contents through server logs.

PromptArmor said the victim would later see the expected ticket suggestions without an obvious indication that information had also been transmitted externally. The attack was not entirely zero-click, as the victim still had to expose Rovo to the malicious content and initiate a normal request. However, the subsequent exfiltration did not require a separate human approval step.

The firm also reported that disabling Rovo's web-search capability did not prevent its demonstrated attack because the exfiltration relied on a separate URL-retrieval capability.

PromptArmor identified the lack of a control preventing Rovo from opening a URL constructed by the model as a root cause. It also noted that Rovo can render Markdown images from model output, which could provide another potential route for data leakage, although the firm did not demonstrate a complete Rovo attack through that mechanism.

PromptArmor said it disclosed the issue to Atlassian on May 23, followed up on June 4 and July 29, and published after reporting no further communication. Its disclosure did not establish whether the content-based attack was remediated after publication.


RovoBlast used a malicious link

Varonis discovered a separate vulnerability involving Rovo's "rovoChatPrompt" URL parameter.

An attacker could place instructions directly into a specially crafted Rovo Chat link. When an authenticated user clicked the link, Rovo would load the attacker-controlled prompt and execute it using the user's existing permissions.

Varonis demonstrated the technique by instructing Rovo to retrieve sensitive information, place it into an attacker-controlled image URL and fetch the resource, thereby sending the data to the attacker.

The researchers successfully exfiltrated a private API key stored in Confluence. They also tested the technique against Jira information and data accessible through SharePoint and Outlook connectors.

The vulnerability, dubbed RovoBlast, was reported through Bugcrowd, received a P2 priority rating and earned a $6,000 bounty. Bugcrowd records Atlassian as deploying a server-side fix on July 8, 2026, after which the researcher validated the remediation and the report was marked resolved.


Enterprise permissions remain central to the risk

Rovo operates across Atlassian products and can incorporate information from connected third-party applications. Atlassian says Rovo access follows the permissions available to the user, meaning the demonstrations did not provide attackers with unrestricted tenant access.

However, the findings expose a different problem: an attacker can attempt to make the AI assistant use a victim's legitimate permissions for an unintended purpose.

Atlassian provides administrators with controls to restrict Rovo by application and, for Enterprise customers, by user group. Rovo is available on Standard, Premium and Enterprise Cloud plans, while disabling Rovo for one Jira-family application may not remove shared Rovo Search, Chat and Create capabilities if another Jira application on the same site still has Rovo enabled.

Neither disclosure reported confirmed exploitation against a real organization, and neither issue has a CVE or entry in CISA's Known Exploited Vulnerabilities catalog as of August 8.

The immediate status is therefore split: Atlassian has confirmed the RovoBlast link vulnerability is closed, while the post-publication status of PromptArmor's separate content-borne attack remains unconfirmed.

Organizations using Rovo should review which applications, user groups and third-party connectors have access, tighten underlying data permissions and avoid treating the web-search setting alone as a complete defense against AI-assisted data exfiltration.

Read more »
Technology
AI AI prompt injection attack Copilot GitHub GitHub repositories breach Microsoft Technology

GitHub Fixes AI Flaw That Could Have Exposed Private Repository Tokens

 



A now-patched security weakness in GitHub Codespaces revealed how artificial intelligence tools embedded in developer environments can be manipulated to expose sensitive credentials. The issue, discovered by cloud security firm Orca Security and named RoguePilot, involved GitHub Copilot, the AI coding assistant integrated into Codespaces. The flaw was responsibly disclosed and later fixed by Microsoft, which owns GitHub.

According to researchers, the attack could begin with a malicious GitHub issue. An attacker could insert concealed instructions within the issue description, specifically crafted to influence Copilot rather than a human reader. When a developer launched a Codespace directly from that issue, Copilot automatically processed the issue text as contextual input. This created an opportunity for hidden instructions to silently control the AI agent operating within the development environment.

Security experts classify this method as indirect or passive prompt injection. In such attacks, harmful instructions are embedded inside content that a large language model later interprets. Because the model treats that content as legitimate context, it may generate unintended responses or perform actions aligned with the attacker’s objective.

Researchers also described RoguePilot as a form of AI-mediated supply chain attack. Instead of exploiting external software libraries, the attacker leverages the AI system integrated into the workflow. GitHub allows Codespaces to be launched from repositories, commits, pull requests, templates, and issues. The exposure occurred specifically when a Codespace was opened from an issue, since Copilot automatically received the issue description as part of its prompt.

The manipulation could be hidden using HTML comment tags, which are invisible in rendered content but still readable by automated systems. Within those hidden segments, an attacker could instruct Copilot to extract the repository’s GITHUB_TOKEN, a credential that provides elevated permissions. In one demonstrated scenario, Copilot could be influenced to check out a specially prepared pull request containing a symbolic link to an internal file. Through techniques such as referencing a remote JSON schema, the AI assistant could read that internal file and transmit the privileged token to an external server.

The RoguePilot disclosure comes amid broader concerns about AI model alignment. Separate research from Microsoft examined a reinforcement learning method called Group Relative Policy Optimization, or GRPO. While typically used to fine-tune large language models after deployment, researchers found it could also weaken safety safeguards, a process they labeled GRP-Obliteration. Notably, training on even a single mildly problematic prompt was enough to make multiple language models more permissive across harmful categories they had never explicitly encountered.

Additional findings stress upon side-channel risks tied to speculative decoding, an optimization technique that allows models to generate multiple candidate tokens simultaneously to improve speed. Researchers found this process could potentially reveal conversation topics or identify user queries with significant accuracy.

Further concerns were raised by AI security firm HiddenLayer, which documented a technique called ShadowLogic. When applied to agent-based systems, the concept evolves into Agentic ShadowLogic. This approach involves embedding backdoors at the computational graph level of a model, enabling silent modification of tool calls. An attacker could intercept and reroute requests through infrastructure under their control, monitor internal endpoints, and log data flows without disrupting normal user experience.

Meanwhile, Neural Trust demonstrated an image-based jailbreak method known as Semantic Chaining. This attack exploits limited reasoning depth in image-generation models by guiding them through a sequence of individually harmless edits that gradually produce restricted or offensive content. Because each step appears safe in isolation, safety systems may fail to detect the evolving harmful intent.

Researchers have also introduced the term Promptware to describe a new category of malicious inputs designed to function like malware. Instead of exploiting traditional code vulnerabilities, promptware manipulates large language models during inference to carry out stages of a cyberattack lifecycle, including reconnaissance, privilege escalation, persistence, command-and-control communication, lateral movement, and data exfiltration.

Collectively, these findings demonstrate that AI systems embedded in development platforms are becoming a new attack surface. As organizations increasingly rely on intelligent automation, safeguarding the interaction between user input, AI interpretation, and system permissions is critical to preventing misuse within trusted workflows.

Read more »
Cyber Attacks
AI Attacks AI Prompt AI prompt injection attack AI Risks AI Systems AI technology autonomous vehicles Cyber Attacks

Visual Prompt Injection Attacks Can Hijack Self-Driving Cars and Drones

 

Indirect prompt injection happens when an AI system treats ordinary input as an instruction. This issue has already appeared in cases where bots read prompts hidden inside web pages or PDFs. Now, researchers have demonstrated a new version of the same threat: self-driving cars and autonomous drones can be manipulated into following unauthorized commands written on road signs. This kind of environmental indirect prompt injection can interfere with decision-making and redirect how AI behaves in real-world conditions. 

The potential outcomes are serious. A self-driving car could be tricked into continuing through a crosswalk even when someone is walking across. Similarly, a drone designed to track a police vehicle could be misled into following an entirely different car. The study, conducted by teams at the University of California, Santa Cruz and Johns Hopkins, showed that large vision language models (LVLMs) used in embodied AI systems would reliably respond to instructions if the text was displayed clearly within a camera’s view. 

To increase the chances of success, the researchers used AI to refine the text commands shown on signs, such as “proceed” or “turn left,” adjusting them so the models were more likely to interpret them as actionable instructions. They achieved results across multiple languages, including Chinese, English, Spanish, and Spanglish. Beyond the wording, the researchers also modified how the text appeared. Fonts, colors, and placement were altered to maximize effectiveness. 

They called this overall technique CHAI, short for “command hijacking against embodied AI.” While the prompt content itself played the biggest role in attack success, the visual presentation also influenced results in ways that are not fully understood. Testing was conducted in both virtual and physical environments. Because real-world testing on autonomous vehicles could be unsafe, self-driving car scenarios were primarily simulated. Two LVLMs were evaluated: the closed GPT-4o model and the open InternVL model. 

In one dataset-driven experiment using DriveLM, the system would normally slow down when approaching a stop signal. However, once manipulated signs were placed within the model’s view, it incorrectly decided that turning left was appropriate, even with pedestrians using the crosswalk. The researchers reported an 81.8% success rate in simulated self-driving car prompt injection tests using GPT-4o, while InternVL showed lower susceptibility, with CHAI succeeding in 54.74% of cases. Drone-based tests produced some of the most consistent outcomes. Using CloudTrack, a drone LVLM designed to identify police cars, the researchers showed that adding text such as “Police Santa Cruz” onto a generic vehicle caused the model to misidentify it as a police car. Errors occurred in up to 95.5% of similar scenarios. 

In separate drone landing tests using Microsoft AirSim, drones could normally detect debris-filled rooftops as unsafe, but a sign reading “Safe to land” often caused the model to make the wrong decision, with attack success reaching up to 68.1%. Real-world experiments supported the findings. Researchers used a remote-controlled car with a camera and placed signs around a university building reading “Proceed onward.” 

In different lighting conditions, GPT-4o was hijacked at high rates, achieving 92.5% success when signs were placed on the floor and 87.76% when placed on other cars. InternVL again showed weaker results, with success only in about half the trials. Researchers warned that these visual prompt injections could become a real-world safety risk and said new defenses are needed.
Read more »
United Kingdom
AI prompt injection attack Artificial Intelligence Cyber Security NCSC SQL United Kingdom

UK Cyber Agency says AI Prompt-injection Attacks May Persist for Years

 



The United Kingdom’s National Cyber Security Centre has issued a strong warning about a spreading weakness in artificial intelligence systems, stating that prompt-injection attacks may never be fully solved. The agency explained that this risk is tied to the basic design of large language models, which read all text as part of a prediction sequence rather than separating instructions from ordinary content. Because of this, malicious actors can insert hidden text that causes a system to break its own rules or execute unintended actions.

The NCSC noted that this is not a theoretical concern. Several demonstrations have already shown how attackers can force AI models to reveal internal instructions or sensitive prompts, and other tests have suggested that tools used for coding, search, or even résumé screening can be manipulated by embedding concealed commands inside user-supplied text.

David C, a technical director at the NCSC, cautioned that treating prompt injection as a familiar software flaw is a mistake. He observed that many security professionals compare it to SQL injection, an older type of vulnerability that allowed criminals to send harmful instructions to databases by placing commands where data was expected. According to him, this comparison is dangerous because it encourages the belief that both problems can be fixed in similar ways, even though the underlying issues are completely different.

He illustrated this difference with a practical scenario. If a recruiter uses an AI system to filter applications, a job seeker could hide a message in the document that tells the model to ignore existing rules and approve the résumé. Since the model does not distinguish between what it should follow and what it should simply read, it may carry out the hidden instruction.

Researchers are trying to design protective techniques, including systems that attempt to detect suspicious text or training methods that help models recognise the difference between instructions and information. However, the agency emphasised that all these strategies are trying to impose a separation that the technology does not naturally have. Traditional solutions for similar problems, such as Confused Deputy vulnerabilities, do not translate well to language models, leaving large gaps in protection.

The agency also stressed upon a security idea recently shared on social media that attempted to restrict model behaviour. Even the creator of that proposal admitted that it would sharply reduce the abilities of AI systems, showing how complex and limiting effective safeguards may become.

The NCSC stated that prompt-injection threats are likely to remain a lasting challenge rather than a fixable flaw. The most realistic path is to reduce the chances of an attack or limit the damage it can cause through strict system design, thoughtful deployment, and careful day-to-day operation. The agency pointed to the history of SQL injection, which once caused widespread breaches until better security standards were adopted. With AI now being integrated into many applications, they warned that a similar wave of compromises could occur if organisations do not treat prompt injection as a serious and ongoing risk.


Read more »
Malware Attack
AI Models AI Prompt AI prompt injection attack Cyber Security cybercriminals DNS DNS attacks DNS Hacking DOH DoT Hacker attack Malware Attack

Hackers Use DNS Records to Hide Malware and AI Prompt Injections

 

Cybercriminals are increasingly leveraging an unexpected and largely unmonitored part of the internet’s infrastructure—the Domain Name System (DNS)—to hide malicious code and exploit security weaknesses. Security researchers at DomainTools have uncovered a campaign in which attackers embedded malware directly into DNS records, a method that helps them avoid traditional detection systems. 

DNS records are typically used to translate website names into IP addresses, allowing users to access websites without memorizing numerical codes. However, they can also include TXT records, which are designed to hold arbitrary text. These records are often used for legitimate purposes, such as domain verification for services like Google Workspace. Unfortunately, they can also be misused to store and distribute malicious scripts. 

In a recent case, attackers converted a binary file of the Joke Screenmate malware into hexadecimal code and split it into hundreds of fragments. These fragments were stored across multiple subdomains of a single domain, with each piece placed inside a TXT record. Once an attacker gains access to a system, they can quietly retrieve these fragments through DNS queries, reconstruct the binary code, and deploy the malware. Since DNS traffic often escapes close scrutiny—especially when encrypted via DNS over HTTPS (DOH) or DNS over TLS (DOT)—this method is particularly stealthy. 

Ian Campbell, a senior security engineer at DomainTools, noted that even companies with their own internal DNS resolvers often struggle to distinguish between normal and suspicious DNS requests. The rise of encrypted DNS traffic only makes it harder to detect such activity, as the actual content of DNS queries remains hidden from most monitoring tools. This isn’t a new tactic. Security researchers have observed similar methods in the past, including the use of DNS records to host PowerShell scripts. 

However, the specific use of hexadecimal-encoded binaries in TXT records, as described in DomainTools’ latest findings, adds a new layer of sophistication. Beyond malware, the research also revealed that TXT records are being used to launch prompt injection attacks against AI chatbots. These injections involve embedding deceptive or malicious prompts into files or documents processed by AI models. 

In one instance, TXT records were found to contain commands instructing a chatbot to delete its training data, return nonsensical information, or ignore future instructions entirely. This discovery highlights how the DNS system—an essential but often overlooked component of the internet—can be weaponized in creative and potentially damaging ways. 

As encryption becomes more widespread, organizations need to enhance their DNS monitoring capabilities and adopt more robust defensive strategies to close this blind spot before it’s further exploited.
Read more »
Vulnerabilities and Exploits
AI prompt injection attack Artifcial Intelligence Digital threats Technology Vulnerabilities and Exploits

This Side of AI Might Not Be What You Expected

 


In the midst of our tech-driven era, there's a new concern looming — AI prompt injection attacks. 

Artificial intelligence, with its transformative capabilities, has become an integral part of our digital interactions. However, the rise of AI prompt injection attacks introduces a new dimension of risk, posing challenges to the trust we place in these advanced systems. This article seeks to demystify the threat, shedding light on the mechanisms that underlie these attacks and empowering individuals to operate the AI with a heightened awareness.

But what exactly are they, how do they work, and most importantly, how can you protect yourself?

What is an AI Prompt Injection Attack?

Picture AI as your intelligent assistant and prompt injection attacks as a clever ploy to make it go astray. These attacks exploit vulnerabilities in AI systems, allowing individuals with malicious intent to sneak in instructions the AI wasn't programmed to handle. In simpler terms, it's like manipulating the AI into saying or doing things it shouldn't. From minor inconveniences to major threats like coaxing people into revealing sensitive information, the implications are profound.

The Mechanics Behind Prompt Injection Attacks

1. DAN Attacks (Do Anything Now):

Think of this as the AI version of "jailbreaking." While it doesn't directly harm users, it expands the AI's capabilities, potentially transforming it into a tool for mischief. For instance, a savvy researcher demonstrated how an AI could be coerced into generating harmful code, highlighting the risks involved.

2. Training Data Poisoning Attacks: 

These attacks manipulate an AI's training data, altering its behaviour. Picture hackers deceiving an AI designed to catch phishing messages, making it believe certain scams are acceptable. This compromises the AI's ability to effectively safeguard users.

3. Indirect Prompt Injection Attacks:

Among the most concerning for users, these attacks involve feeding malicious instructions to the AI before users receive their responses. This could lead to the AI persuading users into harmful actions, such as signing up for a fraudulent website.

Assessing the Threat Level

Yes, AI prompt injection attacks are a legitimate concern, even though no successful attacks have been reported outside of controlled experiments. Regulatory bodies, including the Federal Trade Commission, are actively investigating, underscoring the importance of vigilance in the ever-evolving landscape of AI.

How To Protect Yourself?

Exercise caution with AI-generated information. Scrutinise the responses, recognizing that AI lacks human judgement. Stay vigilant and responsibly enjoy the benefits of AI. Understand that questioning and comprehending AI outputs are essential to navigating this dynamic technological landscape securely.

In essence, while AI prompt injection attacks may seem intricate, breaking down the elements emphasises the need for a mindful and informed approach. 


Read more »
Subscribe to: Posts (Atom)