May 2020 Web Server Survey
26th May, 2020
In the May 2020 survey we received responses from 1,238,024,212 sites across 261,192,350 unique domains and 9,892,834 web-facing computers. This reflects a gain of 224,000 computers and 1.10 million domains, but a loss of 8.10 million sites.
nginx lost the greatest number of sites, with 14.2 million fewer than in April, but conversely had by far the greatest increases in unique domain names (+1.50 million, +2.02%), and web-facing computers (+137,000, +4.43%) this month. OpenResty, which is based on nginx, also saw a large 237,000 domain count increase to reach 7.15 million – the second largest increase in domains for any vendor this month.
Apache had losses in most metrics, dropping 244,000 domains. It did, however, come away with 6.88 million more sites and 45,000 more computers this month than last. Apache still leads in the active sites, computers, and top one million sites metrics.
Microsoft lost out on all metrics this month, dropping by 5.08 million sites and 175,000 unique domains. Both Apache and Microsoft have been on slow long-term downward trends in most metrics. Although they have both increased their count of web-facing computers over time, nginx has seen much stronger growth in comparison. Despite running on a 17.9% share of domain names and 16.4% share of computers, Microsoft holds a much smaller 4.72% share of active sites.
LiteSpeed currently serves 4.20 million domains, giving it a 1.61% market share. It has a slightly higher 1.88% share amongst the top one million sites. LiteSpeed has seen consistent growth, and has had a 23.3% domain count growth over the last 12 months.
Vendor News
Nginx released a new stable version of the nginx web server. nginx version 1.18.0 incorporates additional features which have been introduced in the mainline 1.17.x nginx versions over time. Nginx also released version 1.17.0 of the Nginx Unit application server.
LiteSpeed released new 1.6.13 and 1.7.1 versions of their OpenLiteSpeed web server, introducing bug fixes, security features, updates from their LSQUIC library, CentOS 8 support, and more.
| Developer | April 2020 | Percent | May 2020 | Percent | Change |
|---|---|---|---|---|---|
| nginx | 459,886,788 | 36.91% | 445,724,550 | 36.00% | -0.90 |
| Apache | 308,143,708 | 24.73% | 315,019,262 | 25.45% | 0.72 |
| Microsoft | 160,121,865 | 12.85% | 155,042,311 | 12.52% | -0.33 |
| 42,648,748 | 3.42% | 44,304,867 | 3.58% | 0.16 |
Most Reliable Hosting Company Sites in April 2020
1st May, 2020
| Rank | Performance Graph | OS | Outage hh:mm:ss |
Failed Req% |
DNS | Connect | First byte |
Total |
|---|---|---|---|---|---|---|---|---|
| 1 | GoDaddy.com Inc | Linux | 0:00:00 | 0.000 | 0.378 | 0.008 | 0.033 | 0.035 |
| 2 | New York Internet (NYI) | FreeBSD | 0:00:00 | 0.000 | 0.519 | 0.054 | 0.109 | 0.109 |
| 3 | Webair | Linux | 0:00:00 | 0.000 | 0.286 | 0.071 | 0.142 | 0.143 |
| 4 | Swishmail | FreeBSD | 0:00:00 | 0.000 | 0.207 | 0.083 | 0.165 | 0.165 |
| 5 | Pair Networks | Linux | 0:00:00 | 0.000 | 0.322 | 0.098 | 0.195 | 0.195 |
| 6 | Hyve Managed Hosting | Linux | 0:00:00 | 0.005 | 0.144 | 0.077 | 0.154 | 0.154 |
| 7 | EveryCity | SmartOS | 0:00:00 | 0.005 | 0.208 | 0.079 | 0.157 | 0.157 |
| 8 | CWCS Managed Hosting | Linux | 0:00:00 | 0.005 | 0.286 | 0.080 | 0.162 | 0.162 |
| 9 | ServerStack | Linux | 0:00:00 | 0.005 | 0.234 | 0.084 | 0.167 | 0.168 |
| 10 | krystal.uk | Linux | 0:00:00 | 0.005 | 0.314 | 0.095 | 0.189 | 0.189 |
GoDaddy had the most reliable hosting company site in April 2020. This is the second consecutive month that GoDaddy has had the most reliable hosting company site and GoDaddy has not appeared outside the top two so far in 2020. GoDaddy provides tools that help businesses create a web presence and has provided ideas for small businesses to cope through the COVID-19 pandemic.
All of the monitored hosting providers are continuing to provide their services despite the challenges introduced by the COVID-19 pandemic. The next four hosting company sites in the top 10 also responded to all of Netcraft's requests in April: New York Internet, Webair, Swishmail, and Pair Networks. The top five hosting company sites were separated by average connection time.
New York Internet has published a COVID-19 Preparedness Plan to demonstrate how it is helping provide services to its customers throughout these uncertain times. In New York, Webair has donated meals and personal protective equipment to frontline workers.
Netcraft wins 2020 Queen's Award for Enterprise
21st April, 2020
Netcraft has today received a Double Queen’s Award for Enterprise.
A Queen’s Award is the highest UK Government award for a British business. It is awarded on the Queen’s Birthday each year, and, in different times, it would include an invitation to a mass gathering at Buckingham Palace. The criteria set by our Government searches for considerable progress sustained over a six year period. This year, 128 companies received a Queen’s Award for International Trade and 66 companies a Queen’s Award for Innovation.
Netcraft is one of three companies to receive a Queen’s Award in both categories. The full list of winners is listed in the Queen’s Awards Press Book.
Governments Introduce Coronavirus-specific Cybercrime Legislation
13th April, 2020
Governments and organisations globally have been making announcements that just a few weeks prior would have been unprecedented. As more of our lives are moving online in an attempt to adapt to changes brought about by the Coronavirus pandemic, many are trying out services they were previously unfamiliar with, such as video conferencing or online grocery shopping. While others are finding themselves with more time to pursue online hobbies such as gaming.
The combined effect of information overload and a mass of people using unfamiliar software and services has created an environment ripe for exploitation by cybercriminals.
Netcraft has tracked Coronavirus-themed cybercrime since 16th March, shortly after it was declared a pandemic by the World Health Organisation. While Netcraft continues to see high volumes of Coronavirus-inspired fake shops, advance fee fraud, phishing and malware lures, this post covers some of the trends Netcraft has observed since our previous posts on the topic.
Recently observed Coronavirus-themed threats
Fake Government information sites and mobile malware
Many governments have set up dedicated websites offering advice and services to support their citizens through the pandemic. Cybercriminals are taking advantage of this by providing copy-cat sites with a malicious twist.
In one recent campaign, the cybercriminals deployed a site that poses as the UK Government and offers “credit card refunds” for “COVID-19 support”. The fraudulent site uses UK Government branding and collects the victim’s personal information – including their credit card number, date of birth and telephone number.
Netcraft Adds Coronavirus-related Cybercrime Protection to its Apps and Browser Extensions
10th April, 2020
Netcraft has added protection from Coronavirus-related cybercrime to its mobile apps for Android and iOS, and to its browser extensions for Chrome, Firefox, Opera, and Microsoft Edge. Websites containing these attacks will be blocked for those who have the app or extension installed. The iOS app — currently available in the UK and Canada — blocks Coronavirus-themed attacks impersonating Canadian and UK businesses as well as providing global coverage of fake shops purporting to sell Coronavirus-related goods.
Any Coronavirus-related cybercrime can easily be reported through the extension or app, by emailing scam@netcraft.com, or at report.netcraft.com, protecting other users from these attacks.
The Netcraft Browser Extension now blocks Coronavirus-related cybercrime
Since 16 March Netcraft has been monitoring and disrupting Coronavirus-themed cybercrime, which accounts for five percent of the attacks we perform countermeasures against and is becoming more prevalent on the internet.
The Netcraft App can be downloaded from any of the major three major app stores:
The Netcraft Extension can be downloaded for any of the four major browsers:
April 2020 Web Server Survey
8th April, 2020
In the April 2020 survey we received responses from 1,246,121,153 sites across 260,089,947 unique domains and 9,669,267 web-facing computers. This reflects a gain of 10,000 computers and 2.90 million domains, but a loss of 16.9 million sites.
nginx and Microsoft lost the most sites this month — 13.4 million and 10.4 million each — but like all other major vendors, they both gained domains.
Since attaining the largest share of domains last month, nginx has extended its lead with net growth of 1.84 million domains and now has a 28.5% share of this market, compared with Apache's 27.8%.
Although Apache gained the largest number of sites this month — more than 2 million — it lost 598,000 active sites and its presence amongst the top million websites decreased by 4,230 sites, which took its top-sites count down by 1.43%. Nonetheless, Apache still has the largest share of the top million sites for now (29.1% compared with nginx's 25.5%), and also continues to lead in terms of active sites and web-facing computers.
Vendors respond to COVID-19
As the coronavirus pandemic continues to affect many people's lives in an unprecedented fashion, some web server vendors have offered to help in a variety of direct and indirect ways.
Microsoft has made an initial $1 billion donation to Puget Sound's COVID-19 Response Fund; published a map that tracks active, recovered and fatal cases; and has offered its Healthcare Bot service powered by Microsoft Azure to help frontline organisations screen patients for potential infection and care.
NGINX and F5 are offering free resources for websites impacted by the crisis. This includes free access to its core training for NGINX Open Source; providing additional help and one free year of NGINX Plus to the education, public government and non-profit sectors; and encouraging its employees to respond to NGINX related matters on Stack Overflow and Twitter.
Google has made its COVID-19 datasets free to access and query. Researchers can also use Google's BigQuery ML language to create and execute machine learning models for free. Google's COVID-19 public dataset program is to remain in effect until 15 September.
Google has already had a number of measures in place to ensure that its systems stay up and running during the coronavirus crisis. For more than ten years it has carried out regular disaster recovery testing to identify and address potential problems before they happen, and its engineers operate from multiple locations. With some businesses experiencing increased online sales while consumers stay at home, Google has also activated its enhanced support structure which was developed for peak demand situations like Black Friday.
Last month, Google announced availability of Game Servers beta, which is a managed service offering the Kubernetes-based, open source Agones game server hosting project cofounded by Google and Ubisoft. Agones automatically scales Kubernetes to meet unpredictable player demand, and so its launch is conveniently timed to help cope with the increased amount of online multiplayer gaming taking place while many people are either self-isolating or on lockdown during the global coronavirus crisis.
Online gaming is helping some companies to weather the pandemic, such as Chinese technology group Tencent, which expects revenues from its games business to hold up better than that of its main rival, Alibaba, whose Taobao Tengine web server currently powers 13.7 million websites. Alibaba's co-founder, Jack Ma, has donated coronavirus test kits and masks to Europe and the US despite the effect the pandemic has had on its Tmall and Taobao retail businesses.
Cloudflare has made its Cloudflare for Teams service free for small businesses during the outbreak, helping employees to work from home securely and effectively.
Finally, Netcraft has been protecting consumers and businesses from the despicable — yet inevitable — influx of coronavirus-themed cybercrime, which has recently scaled up a notch. The types of fraudulent activity that are purposely exploiting the pandemic include tax refund scams and other phishing attacks that have been modified to make use of coronavirus-themed emails, as well as smishing, password-stealing malware, advance fee scams, and masses of fake online stores purportedly selling COVID-19 vaccines, cures and related protective equipment.
| Developer | March 2020 | Percent | April 2020 | Percent | Change |
|---|---|---|---|---|---|
| nginx | 473,308,955 | 37.47% | 459,886,788 | 36.91% | -0.57 |
| Apache | 306,114,673 | 24.24% | 308,143,708 | 24.73% | 0.49 |
| Microsoft | 170,567,386 | 13.50% | 160,121,865 | 12.85% | -0.66 |
| 41,227,959 | 3.26% | 42,648,748 | 3.42% | 0.16 |