April 2020 Web Server Survey
8th April, 2020
In the April 2020 survey we received responses from 1,246,121,153 sites across 260,089,947 unique domains and 9,669,267 web-facing computers. This reflects a gain of 10,000 computers and 2.90 million domains, but a loss of 16.9 million sites.
nginx and Microsoft lost the most sites this month — 13.4 million and 10.4 million each — but like all other major vendors, they both gained domains.
Since attaining the largest share of domains last month, nginx has extended its lead with net growth of 1.84 million domains and now has a 28.5% share of this market, compared with Apache's 27.8%.
Although Apache gained the largest number of sites this month — more than 2 million — it lost 598,000 active sites and its presence amongst the top million websites decreased by 4,230 sites, which took its top-sites count down by 1.43%. Nonetheless, Apache still has the largest share of the top million sites for now (29.1% compared with nginx's 25.5%), and also continues to lead in terms of active sites and web-facing computers.
Vendors respond to COVID-19
As the coronavirus pandemic continues to affect many people's lives in an unprecedented fashion, some web server vendors have offered to help in a variety of direct and indirect ways.
Microsoft has made an initial $1 billion donation to Puget Sound's COVID-19 Response Fund; published a map that tracks active, recovered and fatal cases; and has offered its Healthcare Bot service powered by Microsoft Azure to help frontline organisations screen patients for potential infection and care.
NGINX and F5 are offering free resources for websites impacted by the crisis. This includes free access to its core training for NGINX Open Source; providing additional help and one free year of NGINX Plus to the education, public government and non-profit sectors; and encouraging its employees to respond to NGINX related matters on Stack Overflow and Twitter.
Google has made its COVID-19 datasets free to access and query. Researchers can also use Google's BigQuery ML language to create and execute machine learning models for free. Google's COVID-19 public dataset program is to remain in effect until 15 September.
Google has already had a number of measures in place to ensure that its systems stay up and running during the coronavirus crisis. For more than ten years it has carried out regular disaster recovery testing to identify and address potential problems before they happen, and its engineers operate from multiple locations. With some businesses experiencing increased online sales while consumers stay at home, Google has also activated its enhanced support structure which was developed for peak demand situations like Black Friday.
Last month, Google announced availability of Game Servers beta, which is a managed service offering the Kubernetes-based, open source Agones game server hosting project cofounded by Google and Ubisoft. Agones automatically scales Kubernetes to meet unpredictable player demand, and so its launch is conveniently timed to help cope with the increased amount of online multiplayer gaming taking place while many people are either self-isolating or on lockdown during the global coronavirus crisis.
Online gaming is helping some companies to weather the pandemic, such as Chinese technology group Tencent, which expects revenues from its games business to hold up better than that of its main rival, Alibaba, whose Taobao Tengine web server currently powers 13.7 million websites. Alibaba's co-founder, Jack Ma, has donated coronavirus test kits and masks to Europe and the US despite the effect the pandemic has had on its Tmall and Taobao retail businesses.
Cloudflare has made its Cloudflare for Teams service free for small businesses during the outbreak, helping employees to work from home securely and effectively.
Finally, Netcraft has been protecting consumers and businesses from the despicable — yet inevitable — influx of coronavirus-themed cybercrime, which has recently scaled up a notch. The types of fraudulent activity that are purposely exploiting the pandemic include tax refund scams and other phishing attacks that have been modified to make use of coronavirus-themed emails, as well as smishing, password-stealing malware, advance fee scams, and masses of fake online stores purportedly selling COVID-19 vaccines, cures and related protective equipment.
| Developer | March 2020 | Percent | April 2020 | Percent | Change |
|---|---|---|---|---|---|
| nginx | 473,308,955 | 37.47% | 459,886,788 | 36.91% | -0.57 |
| Apache | 306,114,673 | 24.24% | 308,143,708 | 24.73% | 0.49 |
| Microsoft | 170,567,386 | 13.50% | 160,121,865 | 12.85% | -0.66 |
| 41,227,959 | 3.26% | 42,648,748 | 3.42% | 0.16 |
Most Reliable Hosting Company Sites in March 2020
3rd April, 2020
| Rank | Performance Graph | OS | Outage hh:mm:ss |
Failed Req% |
DNS | Connect | First byte |
Total |
|---|---|---|---|---|---|---|---|---|
| 1 | GoDaddy.com Inc | Linux | 0:00:00 | 0.000 | 0.412 | 0.009 | 0.034 | 0.035 |
| 2 | EveryCity | SmartOS | 0:00:00 | 0.000 | 0.233 | 0.076 | 0.152 | 0.152 |
| 3 | CWCS Managed Hosting | Linux | 0:00:00 | 0.000 | 0.338 | 0.079 | 0.160 | 0.161 |
| 4 | krystal.uk | Linux | 0:00:00 | 0.000 | 0.341 | 0.091 | 0.181 | 0.181 |
| 5 | www.dinahosting.com | Linux | 0:00:00 | 0.000 | 0.307 | 0.098 | 0.196 | 0.196 |
| 6 | Pair Networks | Linux | 0:00:00 | 0.000 | 0.353 | 0.098 | 0.197 | 0.197 |
| 7 | Rackspace | Linux | 0:00:00 | 0.005 | 0.477 | 0.009 | 0.020 | 0.020 |
| 8 | Bigstep | Linux | 0:00:00 | 0.005 | 0.254 | 0.076 | 0.153 | 0.153 |
| 9 | Swishmail | FreeBSD | 0:00:00 | 0.005 | 0.239 | 0.082 | 0.164 | 0.164 |
| 10 | ServerStack | Linux | 0:00:00 | 0.005 | 0.273 | 0.084 | 0.169 | 0.169 |
In March 2020 GoDaddy had the most reliable hosting company site, with no failed requests and an average connection time of 9ms. GoDaddy has now topped the table for two out of the three months of 2020 so far. GoDaddy provides services that allow customers to build their own web presence, which include hosting solutions, domain registration, and a popular website builder focused on ease of use.
Coronavirus Cybercrime Scaling Up
2nd April, 2020
Just like Coronavirus itself, the Coronavirus-themed cybercrime it has spawned is quickly becoming a pandemic of its own. Cybercriminals have been quick to take advantage of the media attention on the story, using lures with a Coronavirus theme. Many of the attacks Netcraft has observed have used the fear and uncertainty surrounding the situation to trigger a response from their victims.
Netcraft has tracked Coronavirus-themed cybercrime since 16th March, shortly after it was declared a pandemic by the WHO. This post covers some of the trends Netcraft has observed since our previous post on the topic.
Coronavirus certificates
Analysis of certificate transparency logs for new certificates covering hostnames containing keywords “COVID” and “Coronavirus” shows increasing numbers of certificates are being issued for Coronavirus-themed hostnames.
Whilst some of the certificates included in the graph will be being used for legitimate purposes, many certificates – particularly those which have been registered since the outbreak started – are being used to spread disinformation, host fake shops and pharmacies, serve phishing websites and to disseminate malware.
Coronavirus Cybercrime
27th March, 2020
Netcraft has tracked Coronavirus-themed cybercrime since 16th March, shortly after it was declared a pandemic by the WHO. Scammers have been quick to take advantage of the massive worldwide attention to Coronavirus (COVID-19), and are increasingly making use of it as a theme for online fraud.
Netcraft is the largest provider of anti-phishing takedowns in the world and provides countermeasures against some 75 other types of cybercrime for governments, internet infrastructure and many of the world’s largest banks and enterprises. Coronavirus-themed cybercrime accounts for around 5% of all the attacks we perform countermeasures against, even without accounting for attacks that may otherwise be attributed to existing phishing targets.
Coronavirus Phishing attack impersonating the World Health Organisation
March 2020 Web Server Survey
20th March, 2020
In the March 2020 survey we received responses from 1,263,025,546 sites across 257,194,796 unique domains and 9,659,223 web-facing computers. This reflects a gain of 94,300 computers, 2.12 million sites and 3.00 million domains.
Microsoft and nginx both saw increases in the total number of domains in March 2020, with nginx gaining 4.84 million domains (+7.2%) and increasing its market share by 1.6 percentage points to 28.1%. Microsoft gained 215,000 domains, though this was not substantial enough to avoid losing market share to nginx.
nginx’s sharp increase saw it overtake Apache in terms of domain market share for the first time, with a marginal lead of 136,000 domains. However Apache continues to lead nginx by a considerable amount in terms of active sites —despite losing 225,000 active sites this month, Apache maintains an 8.21 percentage point lead in market share over nginx. Apache also leads in terms of web-facing computers, though with only 3.17 percentage points separating them from nginx.
Several server vendors which hold a lower market share saw mixed results this month. Google lost 115,000 domains but gained 510,000 active sites, while Oracle lost 27,800 domains and 22,200 active sites. Both hold less than one percent of domain market share, with Google claiming 0.87% (-0.06 percentage points), and Oracle holding 0.22% (-0.01 percentage points).
After having gained almost 2 million domains every month since December, Cloudflare’s rapid growth slowed this month with a gain of only 714,929 domains. Cloudflare power their content delivery network with their own server software, originally based on nginx , which accounted for 9.31% of observed domains.
Vendor News
NGINX released several new versions of its products this month. The nginx web server was updated to 1.17.9 with several small changes and bug fixes, one of which is related to HTTP/2 support. The company’s dynamic application server NGINX Unit was updated to 1.16.0, adding functionality which allows more configurable round-robin load balancing.
LiteSpeed Technologies released version 5.4.6 of their LiteSpeed Web Server . This release adds support for the latest draft specification of HTTP/3, which itself was published in mid-February. The release also hardens the server’s default TLS configuration by disabling support for TLS 1.1 unless enabled by the user.
Apache also released versions 8.5.53, 9.0.33, and 10.0.0-M3 of Apache Tomcat, which include several small feature updates and bug fixes.
| Developer | February 2020 | Percent | March 2020 | Percent | Change |
|---|---|---|---|---|---|
| nginx | 459,966,569 | 36.48% | 473,308,955 | 37.47% | 1.00 |
| Apache | 309,061,300 | 24.51% | 306,114,673 | 24.24% | -0.27 |
| Microsoft | 179,225,073 | 14.21% | 170,567,386 | 13.50% | -0.71 |
| 40,120,733 | 3.18% | 41,227,959 | 3.26% | 0.08 |
Browsers on track to block 850,000 TLS 1.0 sites
3rd March, 2020
More than 850,000 websites still rely on the outdated TLS 1.0 and TLS 1.1 protocols that are scheduled to be blocked by the majority of web browsers this month. These older versions of the Transport Layer Security protocol, which date back to 1999 and 2006, are vulnerable to numerous practical attacks that have been resolved in later versions. Among the sites still using these outdated setups are major banks , governments , news , and telecoms companies. Big and small alike, such websites are about to be derailed by full-page browser warnings, with the added prospect of getting blocked entirely later on.
This all comes despite more than a year’s notice. Back in late 2018, the four largest browser vendors — Mozilla , Google , Apple , and Microsoft — jointly announced the deprecation of TLS 1.0 and 1.1, with support to be removed from their browsers in March 2020 or shortly thereafter. But a number of notable sites have not heeded these warnings, and have so far failed to switch to a version of TLS more modern than 1.0.
Firefox Nightly and Beta have already removed support for TLS 1.0 and TLS 1.1 and show a full-page interstitial warning, although they currently allow users to bypass it and to continue accessing the affected website.
Included in the list is Huawei , which is already under fire for its less than reassuring security practices. But it’s not just Huawei that’s letting TLS 1.0-only servers slip through the cracks — the UK’s largest mobile network, O2, uses a TLS 1.0-based redirect services on https://o2.co.uk . Governmental websites are also no exception, including the South Africa Justice department, justice.gov.za , and the California Tax Service Center, taxes.ca.gov . Usage of TLS 1.0 is also particularly prevalent on less popular sites or internal services — places where browser security warnings may go unnoticed for some time.