Search This Blog

Powered by Blogger.

Blog Archive

Labels

Footer About

Footer About

Labels

Showing posts with label PyPI Compromise. Show all posts

PyTorch Lightning and Intercom Client Users Exposed to Credential Stealing Campaign


 

Python's software supply chain has been compromised, which targeted the popular PyPI package Lightning and exposed downstream machine learning environments to covert credential theft through a sophisticated software supply chain compromise. 

In conjunction with Aikido Security, OX Security, Socket, and StepSecurity researchers, versions 2.6.2 and 2.6.3, both published on April 30, 2026, have been modified maliciously as part of a broader intrusion related to the "Mini Shai-Hulud" campaign. 

A day earlier, the attack emerged through compromised SAP-related npm packages, underlining an ongoing trend of coordinated cross-ecosystem supply chain threats targeting high-value development environments. As a result of this compromise, organizations that utilize PyTorch Lightning, an open-source abstraction layer over PyTorch with over 31,000 stars on Github, face significant risk. 

In addition to being frequently embedded in dependency trees facilitating image classification, fine-tuning of large language models, diffusion workloads, and forecasting, Lightning's ubiquity increased the scope of the attack. 

A standard pip install lightning command was sufficient for the activation of the malicious chain exploitation did not require a sophisticated trigger. Upon installation of the compromised package, a hidden _runtime directory containing obfuscated JavaScript was created and executed automatically upon module import. This behavior was embedded within the package's initialization logic, ensuring that no additional user interaction was required to execute the script. 

Upon receiving the payload, a Python script (start.py) downloaded the Bun JavaScript runtime from external sources, followed by an 11 MB obfuscated file (router_runtime.js) which carried out the attack sequence in stages. An execution model utilizing JavaScript within a Python package utilizing cross-language JavaScript marks a significant evolution in attacker tradecraft. This complicates detection mechanisms focusing on single-language threats.

The malware's primary objective was credential harvesting. Analysis indicates that the malware targeted GitHub tokens, cloud service credentials spanning Amazon Web Services (AWS), Google Cloud Platform (GCP), and Azure, SSH keys, NPM tokens, Kubernetes configurations, Docker credentials, and environment variables systematically. Moreover, it was also capable of accessing cryptocurrency wallets and developer secrets stored within local and continuous integration/continuous delivery environments. 

By exploiting compromised credentials, stolen data was exfiltrated, often by automating commits to attacker-controlled GitHub repositories, which effectively concealed malicious activity within legitimate developer workflows, effectively masking malicious activity. There were distinctive markers that linked the campaign to the "Shai-Hulud" identity. 

Infected environments were observed creating public repositories with unusual naming conventions, including EveryBoiWeBuildIsaWormBoi and descriptions such as "A Mini Shai-Hulud has appeared." Attackers seem to be able to track compromised systems using these artifacts both as infection indicators and as signalling mechanisms. 

An effort has been made to link the activity to a financial motivated threat group referred to as TeamPCP, who has consistently demonstrated a focus on credential-rich development environments. According to OX Security, approximately 8.3 million downloads are likely to have been exposed as a result of the incident. 

As a result of the attack, Intercom-Client was compromised on the same day, further demonstrating the coordinated nature of the campaign. These incidents are the culmination of a series of supply chain breaches affecting npm, PyPI, and Docker Hub occurring between April 21 and 23 that suggest that a deliberate and sustained effort was made to infiltrate widely trusted software distribution channels between April 21 and 23.

The router_runtime.js payload was further examined in order to uncover extensive obfuscation and a clear focus on credential access and repository manipulation. Approximately 700 references were found to process and environment variables, over 460 references were identified to authentication tokens, and approximately 330 references were found to code repositories. 

Shai-Hulud operations are closely related to these patterns, which emphasize code reuse and iterative refinement of attack techniques. Furthermore, the payload was also capable of poisoning GitHub repositories and propagating through npm packages, raising concerns about secondary infection vectors beyond data exfiltration. 

The Lightning-AI GitHub repository became aware of the compromise when a user reported suspicious behavior under issue #21689 titled “Possible supply chain attack on version 2.6.3.” The report described a hidden execution chain that involved downloading the Bun runtime and executing a large obfuscated payload during module import. Despite this, the issue was later closed without clarification, thereby creating uncertainty concerning the project's initial response to the matter. 

Following Socket's disclosure in the Lightning-AI/pytorch-lightning repository, an even more unusual outcome occurred. In a matter of seconds, an account identified as pl-ghost closed the issue warning about compromised versions, and then posted a meme entitled "SILENCE DEVELOPER." This behavior has raised immediate concerns about potential account compromise since it was seen as anomalous. 

It was discovered that additional suspicious activity was related to the same account, including six rapid branch creations and deletions across multiple repositories within approximately 70 minutes, which were associated with this account. Several of these branches followed random 10-character lowercase naming conventions, which is consistent with the behavior of the Shai-Hulud worm, which probes for write access. 

As well as the branch impersonating Dependabot, another contained inconsistencies such as a misspelled identifier and incorrect naming structure, and all branches were deleted within seconds of being created, and none of them triggered workflows, indicating that automated probing was not being used in development. This combined evidence strongly suggests that the maintainer account may have been compromised, possibly using the same stolen credentials that enabled the malicious package publication on PyPI to be published. 

Upon learning of the incident, Python Package Index administrators quarantined Lightning versions that may have been affected. According to the maintainers, an investigation is underway in order to determine the cause, as the compromised releases introduced functionality that was consistent with credential harvesting methods. 

In the meantime, it is highly recommended that developers remove versions 2.6.2 and 2.6.3 from their environments, downgrade to version 2.6.1, and rotate any potentially exposed credentials across multiple cloud and development platforms, including API keys, tokens, and access credentials. Besides Python, the campaign is evolving beyond Python.

Researchers have confirmed that version 7.0.4 of the intercom-client package within the Node ecosystem has also been compromised, using a preinstall hook to execute credentials-stealing malware. Packagist also has been affected by the attack, where the intercom/intercom-php package (version 5.0.2) has been altered to include a Composer plugin that downloads the Bun runtime using a shell script (setup-intercom.sh) and executes the same obfuscated payload during installation and updates. 

As a result of encryption and exfiltration of stolen data to a remote server endpoint, the campaign's adaptability across ecosystems was further demonstrated. It has been determined that the GitHub account "nhur" has likely been compromised, and that the malicious intercom-client package was published through an automated Continuous Integration workflow triggered by a now-deleted branch of GitHub.

It appears that technical overlap exists among the npm, PyPI, and PHP ecosystems, with similarities in exfiltration techniques based on GitHub, credential targeting patterns, and payload structures. Furthermore, researchers have found similarities between these attacks and previous ones affecting organizations such as Checkmarx, Bitwarden, Telnyx, LiteLLM, and Aqua Security's Trivy, which supports the hypothesis that a single threat actor is responsible. 

Upon suspension from mainstream platforms, TeamPCP reportedly launched an onion-based platform on the dark web to expand its presence. Additionally, the actors have publicly referenced their ties with other cybercriminal groups, including LAPSUS$, while marketing their own tooling infrastructure. 

The developments suggest that the threat landscape is becoming increasingly organized and persistent, with supply chain attacks not just isolated incidents but a broader strategy for infiltrating and monetizing developer ecosystems. Lightning and Intercom compromises remain a stark reminder of the fragility of modern software supply chains as investigations continue. 

In light of the increasingly capable of pivoting across ecosystems and exploiting trusted distribution channels by attackers, organizations operating in cloud-native environments and AI-based environments have become increasingly reliant on robust dependency auditing, real-time monitoring, and rapid incident response. 

The incident highlights a critical juncture in software supply chain security, at which trusted ecosystems are increasingly being weaponised through stealthy, cross-language attack chains that are emerging from across the globe. The coordinated compromises of PyPI, npm, and Packagist packages, together with evidence of maintainer account abuse and automated propagation techniques, demonstrate a high level of operational maturity that challenges traditional methods of detection and response. 

It is now necessary to take proactive measures to guard against threats such as TeamPCP, who have demonstrated their capability to infiltrate developer workflows on a large scale. These include rigorous dependency auditing, tighter access controls, and continuous monitoring of build environments. 

It is imperative to safeguard the integrity of open-source components in order to maintain confidence in modern software development in the present threat landscape.

Security Flaw in Popular Python Library Threatens User Machines


 

The software ecosystem experienced a brief but significant breach on March 24, 2026 that went almost unnoticed, underscoring how fragile even well-established development pipelines have become. As a result of a threat actor operating under the name TeamPCP successfully compromising the PyPI credentials of the maintainer, malicious code has been quietly seeded into newly published versions of the popular LiteLLM Python package versions 1.82.7 and 1.82.8.

LiteLLM itself was not the victim of the intrusion, but rather a previous breach involving Trivy, an open source security scanner integrated into the project's CI/CD pipeline, which effectively made a defensive tool into a channel for an attack. 

PyPI quarantined the tainted packages only after a limited period of approximately three hours when they were live, but the extent of potential exposure was significant due to the staggering number of downloads and installs of LiteLLM, which exceeds 3.4 million per day and 95 million per month, respectively. 

A powerful and unified interface for interacting with multiple large language model providers is provided by LiteLLM, a tool deeply embedded within modern artificial intelligence development environments. LiteLLM frequently operates in environments containing highly sensitive assets such as API credentials, cloud configurations, and proprietary information. 

The incident illustrates not only a fleeting compromise; it also illustrates a broader and increasingly urgent reality that the open source supply chain remains vulnerable to exactly the types of indirect, multi-stage attacks that are the most difficult to detect and the most damaging when they are successful in a global software development environment. This incident was not simply the result of code tampering; it was a carefully designed, multi-stage intrusion intended to exploit environments that are heavily automated and trusted. 

The threat group TeamPCP leveraged its access in order to introduce two trojanized versions of LiteLLM - versions 1.82.7 and 1.82.8 - which contained obfuscated payloads embedded in core components of the package, namely within the module litellm/proxy/proxy_server.py. 

While the insert was subtle, positioned between legitimate code paths, and encoded so as to evade immediate attention, it ensured execution at import, an important point in the development lifecycle that virtually ensures activation in production environments. 

An even more durable mechanism was introduced in the subsequent version by the attackers as a malicious .pth file directly embedded within the site-packages directory, which was used to extend their foothold. As a result of exploiting Python's internal initialization behavior, the payload executed automatically upon every interpreter startup, regardless of whether LiteLLM itself was ever invoked again. Using detached subprocess calls, the malicious logic was able to operate without visibility, effectively bypassing conventional monitoring tools which focus on application execution. 

Designing the payload reflected an in-depth understanding of cloud-native architectures and the dense concentrations of sensitive information contained within them. When activated, the code acted as a comprehensive orchestration layer capable of conducting reconnaissance, credential harvesting, and environment mapping.

Through a systematic process of traversing the host system, SSH keys, cloud provider credentials, Kubernetes configurations, container registry secrets, and environment variables were extracted. Additionally, managed services were probed further for information.

Cloud-based environments utilize native authentication mechanisms, such as AWS instance metadata, to generate signed requests and retrieve secrets directly from services such as Secrets Manager and Parameter Store, extending its reach beyond traditional disk-based storage or network access. 

A comprehensive collection process was conducted, including infrastructure-as-code artifacts, continuous integration and continuous delivery configurations as well as cryptographic material, database credentials, and developer shell histories, effectively turning each compromised device into an extensive repository of exploitable information. 

Data exfiltration was highly sophisticated, utilizing layered encryption and infrastructure that blended seamlessly into legitimate traffic patterns to exfiltrate data. After compression, encryption, and asymmetric key wrapping, stolen data was transmitted to a domain fabricated to resemble legitimate LiteLLM infrastructure before being encrypted.

As a consequence, even intercepted traffic would be of little value without access to the attacker's private key, complicating the forensic analysis and response process. Furthermore, the operation demonstrated a clear emphasis on persistence and lateral expansion, particularly within Kubernetes environments. 

As service account tokens were present in the payload, it initiated cluster-wide reconnaissance, deployed privileged pods across all nodes, including control-plane systems, and mounted host filesystems and bypassed scheduling restrictions. It then introduced a secondary persistence layer that was disguised as a benign system telemetry service within user-level configurations of systemd.

During periodic communication with a remote command-and-control endpoint, this component provided operators with the ability to deliver additional payloads, update tooling, or terminate the activity by using a built-in kill switch. In summary, the incident indicates that operational maturity extends beyond opportunistic exploitation, demonstrating a level of operational maturity. 

The team PCP successfully maximized the return on each compromised host by targeting LiteLLM, a gateway technology at the intersection of multiple artificial intelligence providers. This allowed them access not only to infrastructure credentials, but also to a wide variety of API keys that cover numerous large language model platforms. 

As a result, the compromise of one, widely trusted component can have alarming ripple effects across entire development and production environments with alarming speed and precision in an ecosystem increasingly characterized by interconnected dependencies. Organizations must reevaluate trust boundaries within their software supply chains in the aftermath of the incident, as remediation is no longer the only priority for organizations.

As security teams are increasingly being encouraged to adopt a zero-trust approach towards third-party dependencies, verification does not end when the product is installed, but continues throughout the entire execution lifecycle. 

Among these measures are the enforcing of strict version pins, verifying package integrity using trusted sources, and developing continuous monitoring mechanisms that will detect anomalous behavior at runtime as opposed to simply relying on static analysis. 

The strengthening of continuous integration/continuous delivery pipelines—especially their tools—has emerged as a critical control point, as this attack demonstrated how upstream compromise can cascade downstream without significant resistance. 

An institutionalization of rapid response playbooks is equally important in order to ensure that credentials are rotated, systems are isolated, and forensic validation is conducted without delay when anomalies are discovered. 

As the use of interconnected AI frameworks continues to increase, security responsibilities are shifting from reactive patching to proactive resilience, where detection, containment, and recovery of supply chain intrusions become as essential as preventing them.