Search This Blog

Powered by Blogger.

Blog Archive

Labels

Footer About

Footer About

Labels

Showing posts with label openAI. Show all posts

Railway Tells 4.8 Million Users to Rotate Secrets After Possible Breach of Environment Variables







Railway, the cloud platform used by developers to deploy and run applications, APIs, and databases, has issued an urgent warning to its roughly 4.8 million users: rotate every secret stored on the platform right now. The company detected suspicious activity involving the systems that hold customer environment variables and says an unauthorized party may have accessed the encrypted files, and in some cases, obtained their decrypted contents.

The company published a security disclosure on Wednesday, October 21, describing the intrusion as affecting systems that store customer environment variables. Those files are where developers keep the sensitive strings that power their deployments: API keys for third-party services, database connection strings, private registry credentials, and authentication tokens. Railway paused all deployments on the platform immediately after the detection, bringing down builds for its entire user base while the investigation got underway.

"We strongly recommend that you immediately roll all environment variables, including any API keys."
[Railway, Security Disclosure — October 21, 2026]

By October 22, builds had been restored for Pro and Enterprise workspace customers, though Hobby, Free, and Trial tiers remained offline, with Railway saying it planned to restore them in that order. A follow-up post published October 23 clarified that any code commits pushed during the deployment freeze had not been deployed, meaning developers would need to manually redeploy their most recent commits once access was restored, and also reconnect GitHub integrations and recreate CLI and CI credentials that were issued during that window.

Railway says it believes the incident is part of a coordinated series of attacks specifically targeting cloud platforms that run AI inference workloads. The company stated it had directly notified upstream AI inference providers and asked them to revoke all keys as a precaution - a step that suggests the attackers were harvesting credentials to gain free access to AI model APIs, rather than pursuing user data for conventional financial fraud.

That theory fits a pattern that emerged just the week before on a competing platform. Zeabur, another cloud deployment service, disclosed on October 15 that an attacker had used leaked internal credentials to access its shared cluster infrastructure between October 14 and 15. The Zeabur post-mortem detailed how the intruder moved through the environment with unusual speed, running commands inside 865 user database containers and reading the environment variables and secrets of services across nine regions. Investigators noted the attacker specifically searched the platform database for AI service API keys, attempting to call Amazon Bedrock using Zeabur's own AWS credentials. A separate OpenAI key belonging to a Zeabur user appeared to have been used for over a thousand image generation requests in the hours after the breach. Zeabur attributed the root cause to long-lived admin credentials stored in plaintext and reachable over the public internet.

What Railway Users Should Do Now
  • Rotate all environment variables stored on Railway, including API keys for every third-party service.
  • Rotate private container registry credentials.
  • Review service logs for any unusual or unauthorized activity since October 21.
  • After builds resume, redeploy the latest commit manually and recreate CLI and CI credentials issued during the outage window.
  • Reconnect GitHub integrations if they were created after the freeze began.
Railway has not disclosed how many accounts are confirmed affected, saying only that initial analysis suggests the exposure is limited to a subset of its user base. The company says it will contact confirmed affected customers directly by email once the investigation identifies them. The platform hosts more than 16 million services, which means even a small percentage of users represents a substantial number of potentially compromised deployments.

For developers whose work depends on Railway, the incident is a reminder that secrets stored with a third-party platform inherit that platform's threat surface. Security practitioners have long recommended treating any externally stored credential as potentially compromised following an incident involving its host, regardless of whether that specific account appears in confirmed breach data. Railway's recommendation to rotate everything, not just keys flagged as compromised - follows that same logic. The cost of rotating a key is low; the cost of leaving a live key in an attacker's hands is not.

Railway said it will publish further details about the incident as the investigation continues and updates are being posted to its status page under incident reference QH8L8B3E.