Flurry of reboots signal Exchange Server patching
15th March, 2021
Over 100,000 Outlook Web Access servers have been rebooted since Microsoft released security updates for the ProxyLogon remote code execution vulnerability. The subsequent flurry of reboot activity is likely indicative of many Microsoft Exchange servers being restarted after having security updates applied.
Around half of all servers running Outlook Web Access (a service included with Microsoft Exchange Server) were rebooted in the five days after the emergency patch was released. Some of these have since been rebooted again, so will appear later in the above graph. Rebooted machines are likely to have been updated, but the absence of a reboot after 2 March does not necessarily indicate vulnerability. Anecdotally, most servers have requested a reboot after being updated, but some may only require services to be restarted – although administrators may have opted to reboot the servers anyway.
Microsoft’s original fixes can only be applied to servers that already have the latest cumulative updates of Exchange Server already installed; however, amidst mass exploitation of the vulnerabilities, Microsoft also released a set of security updates that can be applied to older and unsupported Exchange servers that do not—or cannot—have the latest cumulative updates installed.
The alternative security update path is intended as a temporary measure to protect vulnerable machines. Crucially, installing a later cumulative update that does not include the March 2021 security fixes will make the server vulnerable again, and any machine that uses the alternative security update path must be rebooted even if not prompted. In these cases, the servers will certainly not be protected until after the reboot.
Some of the more recent reboots may have been prompted by Microsoft’s 9 March “Patch Tuesday” collection of software updates, which also includes fixes for the remote code execution vulnerabilities in Microsoft Exchange.
On 6 March, four days after the original security updates were released, Netcraft found more than 99,000 Outlook Web Access servers were still running versions flagged as definitely vulnerable by Kevin Beaumont. However, applying Microsoft’s updates even in a timely fashion could have been like shutting the barn door after the horse had bolted, as more than 10% of all visited Outlook Web Access installations were already compromised with attackers' web shells installed. These provide the criminal with continued administrative access to the compromised servers after the security updates had been applied.
The last reboot dates were correct as at 14 March. We were able to accurately determine the time since last reboot for 82% of the IP addresses running Outlook Web Access.