Search This Blog
Powered by Blogger.

Blog Archive

Labels

About Me

Load More
Trendy Right Now

Popular Posts

Showing posts with label LLM. Show all posts
Russian APT28
Data Theft LameHug LLM malware Russian APT28

LameHug Malware Crafts Real-Time Windows Data-Theft Commands Using AI LLM

 

LameHug, a novel malware family, generates commands for execution on compromised Windows systems using a large language model (LLM). 

Russia-backed threat group APT28 (also known as Sednit, Sofacy, Pawn Storm, Fancy Bear, STRONTIUM, Tsar Team, and Forest Blizzard) was attributed for the assaults after LameHug was identified by Ukraine's national cyber incident response team (CERT-UA). Written in Python, the malware communicates with the Qwen 2.5-Coder-32B-Instruct LLM via the Hugging Face API, which allows it to generate commands in response to prompts. 

Alibaba Cloud developed the LLM, which is open-source and designed to produce code, reason, and follow coding-focused instructions. It can translate natural language descriptions into executable code (in several languages) or shell commands. CERT-UA discovered LameHug after receiving reports on July 10 of malicious emails received from hacked accounts impersonating ministry officials and attempting to disseminate malware to executive government organisations.

The emails include a ZIP attachment that contains a LameHub loader. CERT-UA identified at least three variants: 'Attachment.pif,' 'AI_generator_uncensored_Canvas_PRO_v0.9.exe,' and 'image.py.’ 

With a medium degree of confidence, the Ukrainian agency links this action to the Russian threat group APT28. In the reported attacks, LameHug was tasked with carrying out system reconnaissance and data theft directives generated dynamically by the LLM. LameHug used these AI-generated instructions to gather system information and save it to a text file (info.txt), recursively search for documents in critical Windows directories (Documents, Desktop, Downloads), then exfiltrate the data over SFTP or HTTP POST. 

LameHug is the first publicly known malware that uses LLM to carry out the attacker's duties. From a technical standpoint, this could signal a new attack paradigm in which threat actors can modify their techniques throughout a compromise without requiring new payloads. 

Furthermore, employing Hugging Face infrastructure for command and control may help to make communication more stealthy, allowing the intrusion to remain undetected for a longer period of time. The malware can also avoid detection by security software or static analysis tools that search for hardcoded commands by employing dynamically generated commands. CERT-UA did not specify if LameHug's execution of the LLM-generated commands was successful.
Read more »
Privacy
Data protection EU GDPR hamburg LLM Meta AI Privacy

Legal Battle Over Meta’s AI Training Likely to Reach Europe’s Top Court

 


The ongoing debate around Meta’s use of European data to train its artificial intelligence (AI) systems is far from over. While Meta has started training its large language models (LLMs) using public content from Facebook and Instagram, privacy regulators in Europe are still questioning whether this is lawful and the issue may soon reach the European Court of Justice (ECJ).

Meta began training its AI using public posts made by users in the EU shortly after getting the go-ahead from several privacy watchdogs. This approval came just before Meta launched AI-integrated products, including its smart glasses, which rely heavily on understanding cultural and regional context from online data.

However, some regulators and consumer groups are not convinced the approval was justified. A German consumer organization had attempted to block the training through an emergency court appeal. Although the request was denied, that was only a temporary decision. The core legal challenges, including one led by Hamburg’s data protection office, are still expected to proceed in court.

Hamburg’s commissioner, who initially supported blocking the training, later withdrew a separate emergency measure under Europe’s data protection law. He stated that while the training has been allowed to continue for now, it’s highly likely that the final ruling will come from the EU’s highest court.

The controversy centers on whether Meta has a strong enough legal reason, known as "legitimate interest" to use personal data for AI training. Meta’s argument was accepted by Irish regulators, who oversee Meta’s EU operations, on the condition that strict privacy safeguards are in place.


What Does ‘Legitimate Interest’ Mean Under GDPR?

Under the General Data Protection Regulation (GDPR), companies must have a valid reason to collect and use personal data. One of the six legal bases allowed is called “legitimate interest.” 

This means a company can process someone’s data if it’s necessary for a real business purpose, as long as it does not override the privacy rights of the individual.

In the case of AI model training, companies like Meta claim that building better products and improving AI performance qualifies as a legitimate interest. However, this is debated, especially when public data includes posts with personal opinions, cultural expressions, or identity-related content.

Data protection regulators must carefully balance:

1. The company’s business goals

2. The individual’s right to privacy

3. The potential long-term risks of using personal data for AI systems


Some experts argue that this sets a broader precedent. If Meta can train its AI using public data under the concept of legitimate interest, other companies may follow. This has raised hopes among many European AI firms that have felt held back by unclear or strict regulations.

Industry leaders say that regulatory uncertainty, specifically surrounding Europe’s General Data Protection Regulation (GDPR) and the upcoming AI Act has been one of the biggest barriers to innovation in the region. Others believe the current developments signal a shift toward supporting responsible AI development while protecting users’ rights.

Despite approval from regulators and support from industry voices, legal clarity is still missing. Many legal experts and companies agree that only a definitive ruling from the European Court of Justice can settle whether using personal data for AI training in this way is truly lawful.


Read more »
Technology
Artifcial Intelligence CyberCrime Cybersecurity CyberThreat Google Google Gemini AI LLM Phishing Content Technology

Google Gemini Exploit Enables Covert Delivery of Phishing Content

 


An AI-powered automation system in professional environments, such as Google Gemini for Workspace, is vulnerable to a new security flaw. Using Google’s advanced large language model (LLM) integration within its ecosystem, Gemini enables the use of artificial intelligence (AI) directly with a wide range of user tools, including Gmail, to simplify workplace tasks. 

A key feature of the app is the ability to request concise summaries of emails, which are intended to save users time and prevent them from becoming fatigued in their inboxes by reducing the amount of time they spend in it. Security researchers have, however identified a significant flaw in this feature which appears to be so helpful. 

As Mozilla bug bounty experts pointed out, malicious actors can take advantage of the trust users place in Gemini's automated responses by manipulating email content so that the AI is misled into creating misleading summaries by manipulating the content. As a result of the fact that Gemini operates within Google's trusted environment, users are likely to accept its interpretations without question, giving hackers a prime opportunity. This finding highlights what is becoming increasingly apparent in the cybersecurity landscape: when powerful artificial intelligence tools are embedded within widely used platforms, even minor vulnerabilities can be exploited by sophisticated social engineers. 

It is the vulnerability at the root of this problem that Gemini can generate e-mail summaries that seem legitimate but can be manipulated so as to include deceptive or malicious content without having to rely on conventional red flags, such as suspicious links or file attachments, to detect it. 

An attack can be embedded within an email body as an indirect prompt injection by attackers, according to cybersecurity researchers. When Gemini's language model interprets these hidden instructions during thesummarisationn process, it causes the AI to unintentionally include misleading messages in the summary that it delivers to the user, unknowingly. 

As an example, a summary can falsely inform the recipient that there has been a problem with their account, advising them to act right away, and subtly direct them to a phishing site that appears to be reliable and trustworthy. 

While prompt injection attacks on LLMs have been documented since the year 2024, and despite the implementation of numerous safeguards by developers to prevent these manipulations from occurring, this method continues to be effective even today. This tactic is persisting because of the growing sophistication of threat actors as well as the challenge of fully securing generative artificial intelligence systems that are embedded in critical communication platforms. 

There is also a need to be more vigilant when developing artificial intelligence and making sure users are aware of it, as traditional cybersecurity cues may no longer apply to these AI-driven environments. In order to find these vulnerabilities, a cybersecurity researcher, Marco Figueroa, identified them and responsibly disclosed them through Mozilla's 0Din bug bounty program, which specialises in finding vulnerabilities in generative artificial intelligence. 

There is a clever but deeply concerning method of exploitation demonstrated in Figueroa's proof-of-concept. The attack begins with a seemingly harmless e-mail sent to the intended victim that appears harmless at first glance. A phishing prompt disguised in white font on a white background is hidden in a secondary, malicious component of the message, which conceals benign information so as to avoid suspicion of the message.

When viewed in a standard email client, it is completely invisible to the human eye and is hidden behind benign content. The malicious message is strategically embedded within custom tags, which are not standard HTML elements, but which appear to be interpreted in a privileged manner by Gemini's summarization function, as they are not standard HTML elements. 

By activating the "Summarise this email" feature in Google Gemini, a machine learning algorithm takes into account both visible and hidden text within the email. Due to the way Gemini handles input wrapped in tags, it prioritises and reproduces the hidden message verbatim within the summary, placing it at the end of the response, as it should. 

In consequence, what appears to be a trustworthy, AI-generated summary now contains manipulative instructions which can be used to entice people to click on phishing websites, effectively bypassing traditional security measures. A demonstration of the ease with which generative AI tools can be exploited when trust in the system is assumed is demonstrated in this attack method, and it further demonstrates the importance of robust sanitisation protocols as well as input validation protocols for prompt sanitisation. 

It is alarming how effectively the exploitation technique is despite its technical simplicity. An invisible formatting technique enables the embedding of hidden prompts into an email, leveraging Google Gemini's interpretation of raw content to capitalise on its ability to comprehend the content. In the documented attack, a malicious actor inserts a command inside a span element with font-size: 0 and colour: white, effectively rendering the content invisible to the recipient who is viewing the message in a standard email client. 

Unlike a browser, which renders only what can be seen by the user, Gemini process the entire raw HTML document, including all hidden elements. As a consequence, Gemini's summary feature, which is available to the user when they invoke it, interprets and includes the hidden instruction as though it were part of the legitimate message in the generated summary.

It is important to note that this flaw has significant implications for services that operate at scale, as well as for those who use them regularly. A summary tool that is capable of analysing HTML inline styles, such as font-size:0, colour: white, and opacity:0, should be instructed to ignore or neutralise these styles, which render text visually hidden. 

The development team can also integrate guard prompts into LLM behaviour, instructing models not to ignore invisible content, for example. In terms of user education, he recommends that organisations make sure their employees are aware that AI-generated summaries, including those generated by Gemini, serve only as informational aids and should not be treated as authoritative sources when it comes to urgent or security-related instructions. 

A vulnerability of this magnitude has been discovered at a crucial time, as more and more tech companies are increasingly integrating LLMs into their platforms to automate productivity. In contrast to previous models, where users would manually trigger AI tools, the new paradigm is a shift to automated AI tools that will run in the background instead.

It is for this reason that Google introduced the Gemini side panel last year in Gmail, Docs, Sheets, and other Workspace apps to help users summarise and create content within their workflow seamlessly. A noteworthy change in Gmail's functionality is that on May 29, Google enabled automatic email summarisation for users whose organisations have enabled smart features across Gmail, Chat, Meet, and other Workspace tools by activating a default personalisation setting. 

As generative artificial intelligence becomes increasingly integrated into everyday communication systems, robust security protocols will become increasingly important as this move enhances convenience. This vulnerability exposes an issue of fundamental inadequacy in the current guardrails used for LLM, primarily focusing on filtering or flagging content that is visible to the user. 

A significant number of AI models, including the Google Gemini AI model, continue to use raw HTML markup, making them susceptible to obfuscation techniques such as zero-font text and white-on-white formatting. Despite being invisible to users, these techniques are still considered valid input to the model by the model-thereby creating a blind spot for attackers that can easily be exploited by attackers. 

Mozilla's 0Din program classified the issue as a moderately serious vulnerability by Mozilla, and said that the flaw could be exploited by hackers to harvest credential information, use vishing (voice-phishing), and perform other social engineering attacks by exploiting trust in artificial intelligence-generated content in order to gain access to information. 

In addition to the output filter, a post-processing filter can also function as an additional safeguard by inspecting artificial intelligence-generated summaries for signs of manipulation, such as embedded URLs, telephone numbers, or language that implies urgency, flagging these suspicious summaries for human review. This layered defence strategy is especially vital in environments where AI operates at scale. 

As well as protecting against individual attacks, there is also a broader supply chain risk to consider. It is clear that mass communication systems, such as CRM platforms, newsletters, and automated support ticketing services, are potential vectors for injection, according to researcher Marco Figueroa. There is a possibility that a single compromised account on any of these SaaS systems can be used to spread hidden prompt injections across thousands of recipients, turning otherwise legitimate SaaS services into large-scale phishing attacks. 

There is an apt term to describe "prompt injections", which have become the new email macros according to the research. The exploit exhibited by Phishing for Gemini significantly underscores a fundamental truth: even apparently minor, invisible code can be weaponised and used for malicious purposes. 

As long as language models don't contain robust context isolation that ensures third-party content is sandboxed or subjected to appropriate scrutiny, each piece of input should be viewed as potentially executable code, regardless of whether it is encoded correctly or not. In light of this, security teams should start to understand that AI systems are no longer just productivity tools, but rather components of a threat surface that need to be actively monitored, measured, and contained. 

The risk landscape of today does not allow organisations to blindly trust AI output. Because generative artificial intelligence is being integrated into enterprise ecosystems in ever greater numbers, organisations must reevaluate their security frameworks in order to address the emerging risks that arise from machine learning systems in the future. 

Considering the findings regarding Google Gemini, it is urgent to consider AI-generated outputs as potential threat vectors, as they are capable of being manipulated in subtle but impactful ways. A security protocol based on AI needs to be implemented by enterprises to prevent such exploitations from occurring, robust validation mechanisms for automated content need to be established, and a collaborative oversight system between development, IT, and security teams must be established to ensure this doesn't happen again. 

Moreover, it is imperative that AI-driven tools, especially those embedded within communication workflows, be made accessible to end users so that they can understand their capabilities and limitations. In light of the increasing ease and pervasiveness of automation in digital operations, it will become increasingly essential to maintain a culture of informed vigilance across all layers of the organisation to maintain trust and integrity.
Read more »
LLM security
AI Models AI technology AI tools CISO Cyber Security Cybersecurity Researchers GenAI Generative AI generative AI tools LLM LLM Model LLM security

Security Teams Struggle to Keep Up With Generative AI Threats, Cobalt Warns

 

A growing number of cybersecurity professionals are expressing concern that generative AI is evolving too rapidly for their teams to manage. 

According to new research by penetration testing company Cobalt, over one-third of security leaders and practitioners admit that the pace of genAI development has outstripped their ability to respond. Nearly half of those surveyed (48%) said they wish they could pause and reassess their defense strategies in light of these emerging threats—though they acknowledge that such a break isn’t realistic. 

In fact, 72% of respondents listed generative AI-related attacks as their top IT security risk. Despite this, one in three organizations still isn’t conducting regular security evaluations of their large language model (LLM) deployments, including basic penetration testing. 

Cobalt CTO Gunter Ollmann warned that the security landscape is shifting, and the foundational controls many organizations rely on are quickly becoming outdated. “Our research shows that while generative AI is transforming how businesses operate, it’s also exposing them to risks they’re not prepared for,” said Ollmann. 
“Security frameworks must evolve or risk falling behind.” The study revealed a divide between leadership and practitioners. Executives such as CISOs and VPs are more concerned about long-term threats like adversarial AI attacks, with 76% listing them as a top issue. Meanwhile, 45% of practitioners are more focused on immediate operational challenges such as model inaccuracies, compared to 36% of executives. 

A majority of leaders—52%—are open to rethinking their cybersecurity strategies to address genAI threats. Among practitioners, only 43% shared this view. The top genAI-related concerns identified by the survey included the risk of sensitive information disclosure (46%), model poisoning or theft (42%), data inaccuracies (40%), and leakage of training data (37%). Around half of respondents also expressed a desire for more transparency from software vendors about how vulnerabilities are identified and patched, highlighting a widening trust gap in the AI supply chain. 

Cobalt’s internal pentest data shows a worrying trend: while 69% of high-risk vulnerabilities are typically fixed across all test types, only 21% of critical flaws found in LLM tests are resolved. This is especially alarming considering that nearly one-third of LLM vulnerabilities are classified as serious. Interestingly, the average time to resolve these LLM-specific vulnerabilities is just 19 days—the fastest across all categories. 

However, researchers noted this may be because organizations prioritize easier, low-effort fixes rather than tackling more complex threats embedded in foundational AI models. Ollmann compared the current scenario to the early days of cloud adoption, where innovation outpaced security readiness. He emphasized that traditional controls aren’t enough in the age of LLMs. “Security teams can’t afford to be reactive anymore,” he concluded. “They must move toward continuous, programmatic AI testing if they want to keep up.”
Read more »
malware
BrowserVenom CyberCrime Cybersecurity CyberThreat DeepSeek-R1 Google Ads Kapersky LLM malware

Fake DeepSeek AI Installers Deliver BrowserVenom Malware



Cybersecurity researchers have released a warning about a sophisticated cyberattack campaign in which users are attempted to access DeepSeek-R1, a widely recognized large language model (LLM), which has been identified as a large language model. Cybercriminals have launched a malicious operation designed to exploit unsuspecting users through deceptive tactics to capitalise on the soaring global interest in artificial intelligence tools, and more specifically, open-source machine learning models (LLMs). 


As a result of a detailed investigation conducted by Kaspersky, a newly discovered Windows-based malware strain known as BrowserVenom is distributed by threat actors utilising a combination of malvertising and phishing techniques to distribute. In addition to intercepting and manipulating web traffic, this sophisticated malware enables attackers to stealthily retrieve sensitive data from users, including passwords, browsing history, and personal information.

It has been reported that cybercriminals are using Google Adwords to redirect users to a fraudulent website that has been carefully designed to replicate the official DeepSeek homepage by using a website name deepseek-platform[.]com. They are deceiving victims into downloading malicious files by imitating the branding and layout of a legitimate DeepSeek-R1 model installation, and they are deceiving them into doing so. 

The emergence of BrowserVenom has a significant impact on the cyber threat landscape, as attackers are utilising the growing interest in artificial intelligence technologies to deliver malware in order to increase the level of exposure. Aside from highlighting the sophistication of social engineering tactics that are becoming increasingly sophisticated, this campaign also serves as an effective reminder to verify the sources of software and tools that may be related to artificial intelligence. 

An analysis of security threats has revealed that attackers behind the BrowserVenom attack have created a deceptive installer posing as the authentic DeepSeek-R1 language model in order to deliver malicious payloads. This malicious software installer has been carefully disguised to make it seem authentic, and it contains a recently identified malware called BrowserVenom, an advanced malware that reroutes all browser traffic through the attacker's servers. 

Using this redirection capability, cybercriminals can intercept and manipulate internet traffic, giving them direct access to the sensitive personal information of millions of people. Despite the fact that BrowserVenom is an important piece of malware, its scope of functionality is especially worrying. Once embedded within a system, the malware can monitor user behaviour, harvest login credentials, retrieve session cookies, and steal financial data, emails, and documents that may even be transmitted in plaintext. 

As a result of this level of access, cybercriminals are able to access all the information they need to commit financial fraud, commit identity theft, or sell stolen data on underground marketplaces. Kaspersky reports that the campaign has already compromised systems in a number of countries. They have confirmed infection reports in Brazil, Cuba, Mexico, India, Nepal, South Africa, and Egypt, highlighting the threat’s global reach. 

An infection vector for DeepSeek is a phishing site that is designed to look just like DeepSeek's official platform, which is the primary channel through which it gets infected, inducing users to download the trojanized installer. Because BrowserVenom is still spreading, experts warn that it poses a persistent and ongoing threat to users worldwide, especially those who use open-source AI tools without verifying the authenticity of the source they are using. 

According to a comprehensive investigation of the BrowserVenom campaign, it appears that a highly orchestrated infection chain has been crafted which begins at a malicious phishing website hosted at https[:]//deepseek-platform[.]com. Malvertising tactics have been employed by the attackers to place sponsored search results strategically atop pages when users search for terms like "DeepSeek R1" and similar. 

Deceptive strategies are designed to take advantage of the growing popularity of open-source artificial intelligence models and trick users into visiting a lookalike website that is convincingly resembling the DeepSeek homepage in order to trick them into visiting a website based on a fake DeepSeek lookalike website. Upon arrival at the fake site, the fake site detects the operating system of the visitor silently. 

A single prominent button labelled “Try now” is displayed on the interface for Windows users - the primary targets of this attack - in order to get a DeepSeek-R1 model for free. There have been occurrences of the site serving slightly modified layouts on other platforms, but all versions share the same goal of luring users into clicking and unintentionally initiating an infection, regardless of which platform they're on. This malware was developed by the operators of the BrowserVenom malware to enhance the credibility of the malicious campaign and reduce the suspicion of users. 

To accomplish this, multiple CAPTCHA mechanisms have been integrated into the attack chain at various points to confuse the user. In addition to providing the fake DeepSeek-R1 download website with a sense of legitimacy, this clever use of CAPTCHA challenges is also a form of social engineering, implying that it is secure and trustworthy, which in turn reinforces the illusion of security. When a user clicks the "Try Now" button on the fraudulent DeepSeek platform, the first CAPTCHA will be triggered, according to cybersecurity researchers.

It is at this point that a victim is presented with a fake CAPTCHA page that mimics the appearance of a standard bot-verification interface. Interestingly enough, this isn't just a superficial challenge for the victim. By using an embedded snippet of JavaScript code, the embedded code evaluates whether a person is actually conducting the interaction, performing several verification checks to identify and block automated access to the system. 

Once users click the button, they will be redirected to a CAPTCHA verification page, which is allegedly designed to stop automated robots from accessing the download. However, there is a layer of heavily obfuscated JavaScript behind this screen that performs advanced checks to ensure that a visitor is actually a human, and not a security scanner, by performing advanced checks. The attackers have been operating similar malicious campaigns in the past using dynamic scripts and evasion logic, which emphasises the campaign's technical sophistication. 

A user is redirected to a secondary page located at proxy1.php once they have completed the CAPTCHA, where a “Download now” button appears once they have completed the CAPTCHA. When users click on this final prompt, they are prompted to download the tampered executable file AI_Launcher_1.21.exe, which they can find at 
https://r1deepseek-ai[.]com/gg/cc/AI_Launcher_1.21.exe. 

Using this executable, the malware can be successfully installed in the browser. This entire process, from the initial search to the installation of the malware, has been cleverly disguised to appear as a legitimate user experience to illustrate how cybercriminals are using both social engineering as well as technical sophistication to spread their malware on an international scale. 

Once a user has successfully completed the initial CAPTCHA, they are directed to a secondary page which displays the "Download" button to what is supposed to be an official DeepSeek installer. It should be noted, however, that if users click on this link, they are downloading a trojanized executable file called AI-Launcher-1.21.exe, which stealthily installs BrowserVenom malware. As part of this process, a second CAPTCHA is required. In this case, the prompt resembles the Cloudflare Turnstile verification, complete with the familiar “I am not a robot” checkbox. As a result, the user is misled throughout the entire infection process, creating an illusion of safety. 

It is the victim's choice to choose between two AI deployment platforms after the second CAPTCHA has been completed- "Ollama" or "LM Studio," both of which are legitimate tools for running local versions of AI models like DeepSeek. However, regardless of which option users select, the end result is the same - BrowserVenom malware is silently downloaded and executed in the background without being noticed. 

Cybercriminals are increasingly weaponising familiar security mechanisms to disguise malicious activity in cybercrime, and this sophisticated use of fake CAPTCHAs indicates a broader trend. There has actually been a rise in similar attacks over the past few years, including recent phishing attacks involving Cloudflare CAPTCHA pages that coax users into executing malicious commands with the hope of getting them to do so. 

As soon as the installer is executed, it entails the installation of a dual-layered operation that mixes both visual legitimacy and covert malicious activity. The user is presented with a convincing installation interface which appears to be a large language model deployment tool, but a hidden background process simultaneously deploys the browser malware, thereby presenting the false appearance of a legitimate tool. During this behind-the-scenes sequence, an attempt is made to bypass traditional security measures to maintain stealth while bypassing traditional security measures. 

A crucial evasion technique is used in the installation of the infection: the installer executes an AES-encrypted PowerShell command to exclude the Windows Defender scan of the user's directory. In this case, attackers improve the likelihood that malware will install undetected and successfully if the malware's operating path is removed from routine antivirus oversight.

Once the malware is installed, the installer then proceeds to download additional payloads from obfuscated scripts, further complicating the detection and analysis of the malware. Ultimately, the payload, BrowserVenom, is injected directly into system memory using a sophisticated technique which avoids putting the malicious code on disk, thus evading signature-based antivirus detections. 

Once embedded in the system, BrowserVenom's primary function is to redirect all browser traffic towards a proxy server controlled by the attacker. As part of this process, the malware installs a rogue root certificate that facilitates HTTPS interceptions and modifies the configuration of browsers on multiple platforms, including Google Chrome, Microsoft Edge, Mozilla Firefox, and other Chromium and Gecko-based browsers. 

By making these changes, the malware can intercept and manipulate secure web traffic without raising the suspicion of users. Furthermore, the malware updates user preferences as well as browser shortcuts to ensure persistence, even if the computer is rebooted or manual removal attempts are made. Researchers have found elements of Russian-language code embedded within the phishing website and distribution infrastructure of the malware that strongly suggests that Russian-speaking threat actors are involved in its development. 

This is the first case of confirmed infections reported by the FBI in Brazil, Cuba, Mexico, India, Nepal, South Africa, and Egypt, demonstrating the campaign's global spread and aggressive campaign strategy. In addition to communicating with a command-and-control (C2) infrastructure at the IP address 141.105.130[.]106, the malware also uses port 37121 as its primary port to communicate, which is hardcoded into the proxy settings it uses. This allows BrowserVenom to hijack and route victim traffic through attacker-controlled channels without user knowledge. 

The growing threat of cyberattacks that exploit the AI boom, particularly the increasing use of popular LLM tools as bait, is emphasised by security experts. It is strongly recommended that users adhere to strict digital hygiene, which includes verifying URLs, checking SSL certificates, and avoiding downloading software from unauthorised sources or advertisements.

A growing interest in artificial intelligence has led to a surge in abuse by sophisticated cybercriminal networks, which has made proactive vigilance essential for users throughout all geographies and industries. In light of the recent BrowserVenom incident, which highlights the deceptive tactics that cybercriminals are using in order to get the user to take action, it highlights the urgency for users to be more aware of AI-related threats. 

Today, adversaries are blending authentic interfaces, advanced evasion methods, and social engineering into one seamless attack, which makes traditional security habits no longer sufficient to thwart them. The cybersecurity mindset of organizations and individuals alike requires a combination of real-time threat intelligence, behavioral detection tools, and cautious digital behavior that is based on real-time threat intelligence. Increasingly sophisticated artificial intelligence is changing the landscape of artificial intelligence threats, which requires continuous vigilance to prevent a malicious innovation from getting a step ahead.
Read more »
Software
agentic AI AI API Artifical Intelligence Cyber Security Cyberattacks CyberThreat Data collection LLM RAG Software

The Strategic Imperatives of Agentic AI Security


 

In terms of cybersecurity, agentic artificial intelligence is emerging as a transformative force that is fundamentally transforming the way digital threats are perceived and handled. It is important to note that, unlike conventional artificial intelligence systems that typically operate within predefined parameters, agentic AI systems can make autonomous decisions by interacting dynamically with digital tools, complex environments, other AI agents, and even sensitive data sets. 

There is a new paradigm emerging in which AI is not only supporting decision-making but also initiating and executing actions independently in pursuit of achieving its objective in this shift. As the evolution of cybersecurity brings with it significant opportunities for innovation, such as automated threat detection, intelligent incident response, and adaptive defence strategies, it also poses some of the most challenging challenges. 

As much as agentic AI is powerful for defenders, the same capabilities can be exploited by adversaries as well. If autonomous agents are compromised or misaligned with their targets, they can act at scale in a very fast and unpredictable manner, making traditional defence mechanisms inadequate. As organisations increasingly implement agentic AI into their operations, enterprises must adopt a dual-security posture. 

They need to take advantage of the strengths of agentic AI to enhance their security frameworks, but also prepare for the threats posed by it. There is a need to strategically rethink cybersecurity principles as they relate to robust oversight, alignment protocols, and adaptive resilience mechanisms to ensure that the autonomy of AI agents is paired with the sophistication of controls that go with it. Providing security for agentic systems has become more than just a technical requirement in this new era of AI-driven autonomy. 

It is a strategic imperative as well. In the development lifecycle of Agentic AI, several interdependent phases are required to ensure that the system is not only intelligent and autonomous but also aligned with organisational goals and operational needs. Using this structured progression, agents can be made more effective, reliable, and ethically sound across a wide variety of use cases. 

The first critical phase in any software development process is called Problem Definition and Requirement Analysis. This lays the foundation for all subsequent efforts in software development. In this phase, organisations need to be able to articulate a clear and strategic understanding of the problem space that the artificial intelligence agent will be used to solve. 

As well as setting clear business objectives, defining the specific tasks that the agent is required to perform, and assessing operational constraints like infrastructure availability, regulatory obligations, and ethical obligations, it is imperative for organisations to define clear business objectives. As a result of a thorough requirements analysis, the system design is streamlined, scope creep is minimised, and costly revisions can be avoided during the later stages of the deployment. 

Additionally, this phase helps stakeholders align the AI agent's technical capabilities with real-world needs, enabling it to deliver measurable results. It is arguably one of the most crucial components of the lifecycle to begin with the Data Collection and Preparation phase, which is arguably the most vital. A system's intelligence is directly affected by the quality and comprehensiveness of the data it is trained on, regardless of which type of agentic AI it is. 

It has utilised a variety of internal and trusted external sources to collect relevant datasets for this stage. These datasets are meticulously cleaned, indexed, and transformed in order to ensure that they are consistent and usable. As a further measure of model robustness, advanced preprocessing techniques are employed, such as augmentation, normalisation, and class balancing to reduce bias, es and mitigate model failures. 

In order for an AI agent to function effectively across a variety of circumstances and edge cases, a high-quality, representative dataset needs to be created as soon as possible. These three phases together make up the backbone of the development of an agentic AI system, ensuring that it is based on real business needs and is backed up by data that is dependable, ethical, and actionable. Organisations that invest in thorough upfront analysis and meticulous data preparation have a significantly greater chance of deploying agentic AI solutions that are scalable, secure, and aligned with long-term strategic goals, when compared to those organisations that spend less. 

It is important to note that the risks that a systemic AI system poses are more than technical failures; they are deeply systemic in nature. Agentic AI is not a passive system that executes rules; it is an active system that makes decisions, takes action and adapts as it learns from its mistakes. Although dynamic autonomy is powerful, it also introduces a degree of complexity and unpredictability, which makes failures harder to detect until significant damage has been sustained.

The agentic AI systems differ from traditional software systems in the sense that they operate independently and can evolve their behaviour over time as they become more and more complex. OWASP's Top Ten for LLM Applications (2025) highlights how agents can be manipulated into misusing tools or storing deceptive information that can be detrimental to the users' security. If not rigorously monitored, this very feature can turn out to be a source of danger.

It is possible that corrupted data penetrates a person's memory in such situations, so that future decisions will be influenced by falsehoods. In time, these errors may compound, leading to cascading hallucinations in which the system repeatedly generates credible but inaccurate outputs, reinforcing and validating each other, making it increasingly challenging for the deception to be detected. 

Furthermore, agentic systems are also susceptible to more traditional forms of exploitation, such as privilege escalation, in which an agent may impersonate a user or gain access to restricted functions without permission. As far as the extreme scenarios go, agents may even override their constraints by intentionally or unintentionally pursuing goals that do not align with the user's or organisation's goals. Taking advantage of deceptive behaviours is a challenging task, not only ethically but also operationally. Additionally, resource exhaustion is another pressing concern. 

Agents can be overloaded by excessive queues of tasks, which can exhaust memory, computing bandwidth, or third-party API quotas, whether through accident or malicious attacks. When these problems occur, not only do they degrade performance, but they also can result in critical system failures, particularly when they arise in a real-time environment. Moreover, the situation is even worse when agents are deployed on lightweight frameworks, such as lightweight or experimental multi-agent control platforms (MCPs), which may not have the essential features like logging, user authentication, or third-party validation mechanisms, as the situation can be even worse. 

When security teams are faced with such a situation, tracking decision paths or identifying the root cause of failures becomes increasingly difficult or impossible, leaving them blind to their own internal behaviour as well as external threats. A systemic vulnerability in agentic artificial intelligence must be considered a core design consideration rather than a peripheral concern, as it continues to integrate into high-stakes environments. 

It is essential, not only for safety to be ensured, but also to build the long-term trust needed to enable enterprise adoption, that agents act in a transparent, traceable, and ethical manner. Several core functions give agentic AI systems the agency that enables them to make autonomous decisions, behave adaptively, and pursue long-term goals. These functions are the foundation of their agency. The essence of agentic intelligence is the autonomy of agents, which means that they operate without being constantly overseen by humans. 

They perceive their environment with data streams or sensors, evaluate contextual factors, and execute actions that are in keeping with the predefined objectives of these systems. There are a number of examples in which autonomous warehouse robots adjust their path in real time without requiring human input, demonstrating both situational awareness and self-regulation. The agentic AI system differs from reactive AI systems, which are designed to respond to isolated prompts, since they are designed to pursue complex, sometimes long-term goals without the need for human intervention. 

As a result of explicit or non-explicit instructions or reward systems, these agents can break down high-level tasks, such as organising a travel itinerary, into actionable subgoals that are dynamically adjusted according to the new information available. In order for the agent to formulate step-by-step strategies, planner-executor architectures and techniques such as chain-of-thought prompting or ReAct are used by the agent to formulate strategies. 

In order to optimise outcomes, these plans may use graph-based search algorithms or simulate multiple future scenarios to achieve optimal results. Moreover, reasoning further enhances a user's ability to assess alternatives, weigh tradeoffs, and apply logical inferences to them. Large language models are also used as reasoning engines, allowing tasks to be broken down and multiple-step problem-solving to be supported. The final feature of memory is the ability to provide continuity. 

Using previous interactions, results, and context-often through vector databases-agents can refine their behavior over time by learning from their previous experiences and avoiding unnecessary or unnecessary actions. An agentic AI system must be secured more thoroughly than incremental changes to existing security protocols. Rather, it requires a complete rethink of its operational and governance models. A system capable of autonomous decision-making and adaptive behaviour must be treated as an enterprise entity of its own to be considered in a competitive market. 

There is a need for rigorous scrutiny, continuous validation, and enforceable safeguards in place throughout the lifecycle of any influential digital actor, including AI agents. In order to achieve a robust security posture, it is essential to control non-human identities. As part of this process, strong authentication mechanisms must be implemented, along with behavioural profiling and anomaly detection, to identify and neutralise attempts to impersonate or spoof before damage occurs. 

As a concept, identity cannot stay static in dynamic systems, since it must change according to the behaviour and role of the agent in the environment. The importance of securing retrieval-augmented generation (RAG) systems at the source cannot be overstated. As part of this strategy, organisations need to enforce rigorous access policies over knowledge repositories, examine embedding spaces for adversarial interference, and continually evaluate the effectiveness of similarity matching methods to avoid data leaks or model manipulations that are not intended. 

The use of automated red teaming is essential to identifying emerging threats, not just before deployment, but constantly in order to mitigate them. It involves adversarial testing and stress simulations that are designed to expose behavioural anomalies, misalignments with the intended goals, and configuration weaknesses in real-time. Further, it is imperative that comprehensive governance frameworks be established in order to ensure the success of generative and agentic AI. 

As a part of this process, the agent behaviour must be codified in enforceable policies, runtime oversight must be enabled, and detailed, tamper-evident logs must be maintained for auditing and tracking lifecycles. The shift towards agentic AI is more than just a technological evolution. The shift represents a profound change in the way decisions are made, delegated, and monitored in the future. A rapid adoption of these systems often exceeds the ability of traditional security infrastructures to adapt in a way that is not fully understood by them.

Without meaningful oversight, clearly defined responsibilities, and strict controls, AI agents could inadvertently or maliciously exacerbate risk, rather than delivering what they promise. In response to these trends, organisations need to ensure that agents operate within well-defined boundaries, under continuous observation, and aligned with organisational intent, as well as being held to the same standards as human decision-makers. 

There are enormous benefits associated with agentic AI, but there are also huge risks associated with it. Moreover, these systems should not just be intelligent; they should also be trustworthy, transparent, and their rules should be as precise and robust as those they help enforce to be truly transformative.
Read more »
Technology
AI AI technology Airbnb LLM Technology

AI Powers Airbnb’s Code Migration, But Human Oversight Still Key, Say Tech Giants

 

In a bold demonstration of AI’s growing role in software development, Airbnb has successfully completed a large-scale code migration project using large language models (LLMs), dramatically reducing the timeline from an estimated 1.5 years to just six weeks. The project involved updating approximately 3,500 React component test files from Enzyme to the more modern React Testing Library (RTL). 

According to Airbnb software engineer Charles Covey-Brandt, the company’s AI-driven pipeline used a combination of automated validation steps and frontier LLMs to handle the bulk of the transformation. Impressively, 75% of the files were migrated within just four hours, thanks to robust automation and intelligent retries powered by dynamic prompt engineering with context-rich inputs of up to 100,000 tokens. 

Despite this efficiency, about 900 files initially failed validation. Airbnb employed iterative tools and a status-tracking system to bring that number down to fewer than 100, which were finally resolved manually—underscoring the continued need for human intervention in such processes. Other tech giants echo this hybrid approach. Google, in a recent report, noted a 50% speed increase in migrating codebases using LLMs. 

One project converting ID types in the Google Ads system—originally estimated to take hundreds of engineering years—was largely automated, with 80% of code changes authored by AI. However, inaccuracies still required manual edits, prompting Google to invest further in AI-powered verification. Amazon Web Services also highlighted the importance of human-AI collaboration in code migration. 

Its research into modernizing Java code using Amazon Q revealed that developers value control and remain cautious of AI outputs. Participants emphasized their role as reviewers, citing concerns about incorrect or misleading changes. While AI is accelerating what were once laborious coding tasks, these case studies reveal that full autonomy remains out of reach. 

Engineers continue to act as crucial gatekeepers, validating and refining AI-generated code. For now, the future of code migration lies in intelligent partnerships—where LLMs do the heavy lifting and humans ensure precision.
Read more »
Mailchimp
API Key AWS Cyber Security LLM Mailchimp

Private API Keys and Passwords Discovered in a Popular AI Training dataset

 

The Common Crawl dataset, which is used to train several artificial intelligence models, has over 12,000 legitimate secrets, including API keys and passwords. The Common Crawl non-profit organisation maintains a vast open-source archive of petabytes of web data collected since 2008, which is free to use. 

Because of the huge dataset, various artificial intelligence initiatives, including OpenAI, DeepSeek, Google, Meta, Anthropic, and Stability, may rely on the digital archive to train large language models (LLMs).

Truffle Security researchers discovered legitimate secrets after scanning 400 terabytes of data from 2.67 billion web pages in the Common Crawl December 2024 database. They uncovered 11,908 secrets that were successfully authenticated and were hardcoded by developers, highlighting that LLMs could be trained on insecure code.

It should be noted that LLM training data is not used in its raw form; instead, it is cleaned and filtered to remove extraneous content such as useless data, duplicate, malicious, or sensitive data. Despite these efforts, removing confidential data is challenging, and the method does not guarantee that all personally identifiable information (PII), financial data, medical records, and other sensitive content will be erased from the huge dataset. 

Truffle Security discovered legitimate API keys for the WalkScore, MailChimp, and Amazon Web Services (AWS) services after examining the scanned data. In the Common Crawl dataset, TruffleHog found 219 different secret kinds in total, with MailChimp API keys being the most prevalent. 

Cybersecurity researchers explain that the developers made a mistake by hardcoding them into HTML forms and JavaScript snippets rather than using server-side environment variables. An attacker could exploit these keys for nefarious purposes like phishing and brand impersonation. Furthermore, disclosing such knowledge could result in data exfiltration. Another feature of the paper is the high reuse rate of the uncovered secrets, with 63% found on several pages. 

However, a WalkScore API key "appeared 57,029 times across 1,871 subdomains." The researchers also discovered a homepage with 17 unique live Slack webhooks, which should be kept private because they allow apps to submit messages to Slack. After conducting the research, Truffle Security got in touch with the affected suppliers and collaborated with them to remove the keys belonging to their users. 

The researchers claim to have "successfully assisted those organisations collectively in rotating/revoke several thousand keys." Truffle Security's findings are a warning that insecure coding mistakes can affect the LLM's behaviour, even if an AI model uses older archives than the dataset the researchers analysed.
Read more »
Subscribe to: Posts (Atom)