Search This Blog

Showing posts with label Microsoft Web Server. Show all posts

Microsoft Hit by Huge Service Outage

This week's 6-hour-long global outage of Microsoft 365 was caused by a flawed Enterprise Configuration Service (ECS) deployment, as per a preliminary post-incident review. This deployment caused cascade errors and availability effects across numerous locations.

ECS is an internal central configuration repository created to allow Microsoft services to make targeted updates, such as particular configurations per tenant or user, as well as broad-scope dynamic changes affecting many services and features.

According to Microsoft, a recent deployment that featured a "broken link to an internal storage service" was the most likely reason for an outage that prevented many customers from accessing or using a variety of Microsoft 365 products for several hours.

Access to several Microsoft services, including Microsoft Teams, Exchange Server, Microsoft 365 admin center, Microsoft Word, and other Office programs, was slowed down as a result of the service issues, which began on Wednesday, July 20 in the evening and persisted into Thursday morning. Microsoft Managed Desktop and other services were also not able to auto-patch due to the problem.

Overview of the outage

Through its public Twitter statements, Microsoft failed to mention the location of the disruptions. According to comments in Microsoft's Twitter statement, the Teams outage appears to have impacted users in Los Angeles, Dallas, New York City, Hong Kong, and Eastern Australia.

With its cloud computing, Microsoft does have a complex service level agreement. Accordingly, the sole form of compensation for any downtime that an organization can receive is a service-time credit. Additionally, since it is not automatically applied, they must ask for the service credit.

"Telemetry shows that this incident had an impact on about 300,000 calls. Due to business hours falling inside the effect timeframe, the Asia Pacific (APAC) region was the most impacted. Direct Routing and Skype MFA were also significantly affected," the company explained.

What sparked the outage?

In the end, the incident had an impact on users seeking to use one or more of the Microsoft 365 apps and services, according to Bleeping Computer.

The botched Enterprise Configuration Service (ECS) deployment was the initial root cause of this outage, as stated by Redmond in their incident report. "Backward compatibility with services that use ECS was impacted by a deployment of the ECS service that had a code flaw. The end result was that it would send inaccurate configurations to all of its partners for services using ECS " the firm stated.

As a result, downstream services received a status response with the code 200, suggesting that the pull was successful, but it just included a JSON object that was poorly formatted. How each Microsoft service used the flawed configuration supplied by ECS determined the impact's severity. Impact varied from services collapsing, like Teams, to low or no impact on other services.

Microsoft claims that as a result of this incident, they are working to strengthen the Microsoft Teams service's resilience so that it may fall back to a previous version of the ECS configuration in the case of a future ECS failure.

IISpy: Installs Backdoor on Microsoft’s Web Server Software


Cybersecurity investigators have detected malware that could deploy backdoor Internet Information Services (IIS) on Microsoft's Web server software. Labeled IISpy, the malware employs several tools to interfere with the logging and identification of the server so that it can undertake long-term spying. 

The backdoor has also been operational since July 2020, at least, and is employed as a privileged escalation mechanism with Juicy Potato (detected as Win64/HackTool.JuicyPotato by ESET security solutions). 

Threat actors first get initial access to the IIS server by exploiting a flaw and then employ Juicy Potato as a Native IIS extension to gain the administrative rights needed for IISpy to be installed. IISpy impacts a tiny percentage of the IIS platforms in Canada, the U.S., and Holland as per the telemetry. However, it might not be the whole picture, as administrators are still using no server security software and the IIS servers' sight is limited. 

Since IISpy is designed to be an IIS extension, each HTTP request received by the affected IIS server can be viewed and the server would respond. IISpy utilizes its C&C-Communication channel to act as passive implantation in the network. 

Whilst submitting an HTTP request to the affected server, attackers start a connection. The backdoor detects the request from the attacker, retrieves and performs the built-in backdoor commands, and changes the HTTP response to include the output of the command. 

The backdoor provides attackers with system information, file or shell commands, and much more. The malware does not include all valid HTTP visitor requests made to the infected IIS server, which are handled by the harmless server modules. However, the OnLogRequest event handler, executed shortly before the IIS server logs a finished HTTP request, is implemented with an anti-logging function. The backdoor employs this handler to change the system logs for requests from the attackers, as per the researchers. 

Researchers have suggested firms dealing with sensitive information should look for such malware on their systems. Companies who use Outlook on their Exchange email servers on the web (OWA) service specifically should be aware. 

Further researchers added, “OWA is implemented via IIS and makes an interesting target for espionage. In any case, the best way to keep IISpy out of your servers is to keep them up to date, and carefully consider which services are exposed to the internet, to reduce the risk of server exploitation.”