Over 100,000 Outlook Web Access servers have been rebooted since Microsoft released security updates for the ProxyLogon remote code execution vulnerability. The subsequent flurry of reboot activity is likely indicative of many Microsoft Exchange servers being restarted after having security updates applied.
Around half of all servers running Outlook Web Access (a service included with Microsoft Exchange Server) were rebooted in the five days after the emergency patch was released. Some of these have since been rebooted again, so will appear later in the above graph. Rebooted machines are likely to have been updated, but the absence of a reboot after 2 March does not necessarily indicate vulnerability. Anecdotally, most servers have requested a reboot after being updated, but some may only require services to be restarted – although administrators may have opted to reboot the servers anyway.
Microsoft’s original fixes can only be applied to servers that already have the latest cumulative updates of Exchange Server already installed; however, amidst mass exploitation of the vulnerabilities, Microsoft also released a set of security updates that can be applied to older and unsupported Exchange servers that do not—or cannot—have the latest cumulative updates installed.
The alternative security update path is intended as a temporary measure to protect vulnerable machines. Crucially, installing a later cumulative update that does not include the March 2021 security fixes will make the server vulnerable again, and any machine that uses the alternative security update path must be rebooted even if not prompted. In these cases, the servers will certainly not be protected until after the reboot.
Some of the more recent reboots may have been prompted by Microsoft’s 9 March “Patch Tuesday” collection of software updates, which also includes fixes for the remote code execution vulnerabilities in Microsoft Exchange.
On 6 March, four days after the original security updates were released, Netcraft found more than 99,000 Outlook Web Access servers were still running versions flagged as definitely vulnerable by Kevin Beaumont. However, applying Microsoft’s updates even in a timely fashion could have been like shutting the barn door after the horse had bolted, as more than 10% of all visited Outlook Web Access installations were already compromised with attackers’ web shells installed. These provide the criminal with continued administrative access to the compromised servers after the security updates had been applied.
Posted by Paul Mutton in Security
Around 3.6 million websites across 464,000 distinct domains were taken offline after the major fire at an OVHcloud datacenter site in Strasbourg overnight.
More than 18% of the IP addresses attributed to OVH in Netcraft’s most recent Web Server Survey — which took place two weeks ago — were no longer responding at 06:00-07:15 UTC this morning.
Thankfully, everybody is safe; but OVH said the fire in its SBG2 datacenter was not controllable and no data is likely to be recoverable. Part of its SBG1 datacenter has also been destroyed. Firefighters were protecting SBG3 throughout the night, and although there was no direct fire impact on SBG4, it was also unavailable due to the whole site being isolated. Consequently, all services in SGB1-4 have been offline.
Websites that went offline during the fire included online banks, webmail services, news sites, online shops selling PPE to protect against coronavirus, and several countries’ government websites.
Examples of the latter included websites used by the Polish Financial Ombudsman; the Ivorian DGE; the French Plate-forme des achats de l’Etat; the Welsh Government’s Export Hub; and the UK Government’s Vehicle Certification Agency website, which got a new SSL certificate by 10am and is now back online with a UK hosting company.
Unsurprisingly for a French hosting company, the most affected country code top-level domain (ccTLD) is
.fr, which had 184,000 knocked-out websites spread across 59,600 distinct domain names – these account for 1.9% of all
.fr domains in the world. In comparison, there were only 24,100
.uk websites hosted in the affected datacenters, across just 8,700 unique domains. Most of the affected websites use the generic
.com top-level domain, amounting to 880,000 websites across 180,000 domains.
This weekend, several days after Tuesday 2nd March when Microsoft released fixes for the ProxyLogon vulnerability, Netcraft found more than 99,000 unpatched Outlook Web Access servers accessible on the internet — of which several thousand have clear evidence of one or more web shells installed.
Outlook Web Access (OWA) provides remote access to on-premises Microsoft Exchange mailboxes. While a treasure trove of corporate email is a tempting enough target itself, it can also act as a jumping-off point for deeper network access. Vulnerable versions allow unfettered remote access to the mail server. Originally attributed to the Hafnium group, the variety of different web shells and file naming conventions found by Netcraft suggest that the shells belong to multiple groups who have been spurred into action since Microsoft’s announcement by the scale of the opportunity.
Netcraft has established that at least 10% of all visited OWA installations are now infested with web shell backdoors that do not use randomised filenames, and so could plausibly be guessed by anybody. These implants allow continued administrative access to the server, long after the underlying vulnerability has been patched.
All of the backdoors hide in plain sight on the web server’s file system but are disguised as benign scripts or information dumps in order to avoid detection. There are several different variants of the backdoor script, but all have the same common feature in that they pass the hacker’s commands to the JScript Eval command, allowing arbitrary code to be executed directly on the web server.
Most of the backdoor scripts accept the criminals’ arbitrary commands via a specially named GET or POST parameter, while others require the commands to be Base64 encoded first, and some only accept them via a POST parameter.
Netcraft has also seen several different variants of these backdoor scripts being uploaded to individual websites, likely in an attempt to preserve unauthorised access to the compromised web server. Unless all of the backdoor scripts are found and removed, the hackers will still be able to get in and create more.
While some of the backdoor variants are wildly different in appearance, they all function in a similar way and require the user to know a secret variable name before any commands can be executed on the server. The variable name effectively acts as a password and provides the only security mechanism to ensure that the backdoor can only be used by the person or persons responsible for uploading it.
However, some of the shells use easily guessable variable names like “o” and “orange”, which could plausibly allow them to be misused by other hackers if they can find the scripts and guess the correct variable names. This presents an even more dangerous situation where other fraudsters could then upload their own web shells to secure a foothold on the server. Such a situation could escalate quickly… new battlegrounds could erupt where rival fraudsters try to delete each others’ web shells and upload more of their own in a race to secure access and decide how best to monetize their exploits, all long after the initial OWA vulnerabilities have been resolved.
Posted by Paul Mutton in Security
|4||Hyve Managed Hosting||Linux||0:00:00||0.000||0.151||0.076||0.151||0.151|
|5||CWCS Managed Hosting||Linux||0:00:00||0.000||0.320||0.083||0.167||0.167|
|7||New York Internet (NYI)||FreeBSD||0:00:00||0.005||0.509||0.061||0.123||0.123|
Choopa.com took the top spot as the most reliable hosting company site in February. The top five hosting company sites all had no failed requests during the month, with the ranking decided by fastest average connection time. Choopa.com had the fastest connection time out of all of the top 10 hosting company websites, at just 5ms. In the past 12 months Choopa.com appeared in the top 10 nine times. Choopa.com offers a range of services including cloud hosting, dedicated hosting and colocation in its own primary facility in Piscataway, New Jersey as well as other facilities in Los Angeles, Amsterdam, and Tokyo.
Spots two and three in February go to Rackspace and Webair. Rackspace provide a wide variety of cloud services from its global network of over 50 locations in five continents, and Webair offer managed and private cloud services, storage and backup solutions from its eight facilities in New York, Chicago, Los Angeles, Montreal, London, Paris, Amsterdam and Singapore.
In the February 2021 survey we received responses from 1,204,252,411 sites across 263,042,054 unique domains and 10,766,606 web-facing computers. This reflects a gain of 6,270,052 sites, 92,829 domains, and 116,789 computers.
nginx is top of the charts when it comes to total count of sites as well as number of unique domains and web-facing computers. 34.5% of all sites run on nginx, 30.4% of domains, and 35.0% of web-facing computers. Apache comes in at seconds place in these metrics, with a 26.3% market share of sites, a similar 26.4% share of domains, and 32.7% of web-facing computers.
In terms of domains, OpenResty and Cloudflare come in at third and fourth place to make up an additional 14.4% and 7.1% of the market respectively. OpenResty is a web application server that is built upon the technology of nginx, but, strictly speaking, is not an nginx fork. Cloudflare historically based their server stack around nginx, but transitioned towards using more in-house developed technologies over time. As of this month, these web server vendors are tracked individually in the monthly Web Server Survey charts.
Although nginx leads the wider market, Apache still has a small lead when it comes to the top one million busiest sites, with a 25.6% market share – 2.4pp ahead of nginx. Apache increased its share of the top million by 0.54pp in February. Although OpenResty takes a sizable chunk of the wider market, it is not nearly as common amongst the top million, taking only a 1.6% share. This disparity can be explained through GoDaddy’s extensive use of OpenResty for domain parking.
Apache also holds a more significant lead in terms of Netcraft’s active sites metric, which favours sites with unique content. Apache serves 25.5% of active sites, whereas nginx serves 19.8%. Google accounts for a reasonably large 9.9% share of active sites, owing to its popular Blogger service.
Microsoft’s server software market share remains in decline. Microsoft’s figures took a significant drop in 2020 in favour of OpenResty, and Microsoft now only has 6.5% (-1.0pp) of the site market and 6.0% (-0.3pp) of domains as of February 2021. OpenResty also looks set to overtake Microsoft as the third largest vendor in terms of sites and active sites.
Other vendor and hosting news
Nginx has pushed out its first product updates for 2021 – nginx version 1.19.7 and NGINX Unit 1.22.0. Lighttpd also released version 1.4.59 of its web server, which now enables HTTP version 2 by default.
|Developer||January 2021||Percent||February 2021||Percent||Change|
Posted in Web Server Survey
|2||New York Internet (NYI)||FreeBSD||0:00:00||0.000||0.545||0.060||0.118||0.118|
|7||CWCS Managed Hosting||Linux||0:00:00||0.005||0.317||0.082||0.163||0.163|
|10||Hyve Managed Hosting||Linux||0:00:00||0.010||0.160||0.076||0.151||0.151|
Rackspace kicked off 2021 with the most reliable hosting company site in January. The top five hosting company sites each responded to all of Netcraft’s requests in January and were separated by average connection time. Rackspace offers a variety of cloud hosting solutions from 40 data centres across five different continents in the Americas, Europe, Asia and Australia.
The podium is completed by New York Internet (NYI) and EveryCity. NYI offers bare metal, cloud and colocation services from its four data centres in the US. UK-based EveryCity provides cloud hosting solutions and managed third-party services from its primary data centre located in the heart of London.
ServerStack and dinahosting also responded to all of Netcraft’s requests in January. ServerStack maintains its place in the top 10 and has now appeared 11 times in the past 12 months, more than any other hosting company site. ServerStack provides managed and dedicated solutions from its three data centres in North America and Europe. dinahosting offers its services from Interxion, in Madrid, and customers can choose from a range of cloud and managed solutions as well as register domain names.
FreeBSD appeared in second place in January with NYI and SmartOS appeared in third place with EveryCity. Linux was used by the other eight sites in the top 10.
Your link here? Advertising on the Netcraft Blog