In the January 2020 survey we received responses from 1,295,973,827 sites across 249,618,033 unique domain names and 9,576,845 web-facing computers. This reflects a gain of 27.7 million sites, 5.86 million domains, and 146,000 computers.

Apache, nginx, and Microsoft all saw increases in their totals for number of domains in January 2020, although nginx demonstrated substantially the largest growth (+2.53 million), bringing its market share up to 25.8% and its total to 64,391,621 domains. The growths this month for Apache (+80,900) and Microsoft (+66,300) were much smaller in comparison. An additional 2.18 million domains identified themselves as Cloudflare servers, an 11.3% increase since December, bringing the Cloudflare-exclusive server platform up to 21.4 million domains. LiteSpeed usage grew by 96,500 domains — a fairly consistent and strong 2.5% growth — giving it a new total of 3.97 million.

Looking instead over the span of the past year, Microsoft's domain count decreased by 12.1 million (-20.8%), whilst nginx grew by 12.5 million (+24.0%), partly due to a swing from Microsoft to nginx at GoDaddy seen in March. Apache's count of domains is largely the same as it was a year ago, only having shrunk slightly by 1.79 million domains (-2.4%).

The gains and losses in domains were also met with similar trends in active sites. The number of active sites seen by Netcraft in January 2020 increased to 189 million, up from 183 million since the previous month. Around 1.99 million of the increase came from nginx, and 1.12 million from Cloudflare, increasing their respective totals by +5.7% and +6.6%. Apache and Microsoft, on the other hand, remained fairly stable this month, but overall lost out over the course of a year. LiteSpeed's active site count also continues to grow steadily, increasing by 149,000 (+3.1%) this month to reach 4.95 million.

By count of web-facing computers, Apache, nginx, and Microsoft all demonstrated gains, with nginx showing the most significant gain of 83,700 additional computers since December. Over the past year, nginx has grown by 771,000 computers (+33.2%), far in excess of Apache's 192,000 (+6.1%) and Microsoft's 105,000 (+6.8%) growths.

Windows Server 2008 End-of-life

On January 14, 2020 Microsoft ended support for Windows Server 2008 and Server 2008 R2. As a component of the operating system, versions of Microsoft's IIS (Internet Information Services) are tightly bound to the Windows versions they run on, with IIS/7.5 being the version integrated into Windows Server 2008 R2. As such, IIS/7.5 is similarly end-of-life, receiving no further security fixes. Despite forward notice, and the availability of more modern versions, there is often a great deal of inertia preventing companies from upgrading operating system software before it falls out of support. As of January 2020, Netcraft counted 887,000 web-facing computers running Windows Server 2008 and 2008 R2, making them the most popular versions of Windows employed in the webserver market. Furthermore, over half of all counted Windows computers ran some end-of-life version of the Windows family of operating systems.

Turning to the use of IIS specifically, Netcraft found almost 2.85 million active sites running on IIS/7.5 - 32.5% of all active sites running on some version of IIS. Approximately 940,000 active sites run on even older versions of IIS. Within the top one million sites, 25,700 of the 77,800 sites running on IIS use an outdated version.

Vendor news

LiteSpeed Technologies released versions 1.6.5 and 1.5.11 of their open source OpenLiteSpeed web server. The updates introduce improvements to caching performance, security, and stability. The updates also see OpenLiteSpeed move to version 2.8.3 of lsquic, LiteSpeed's C implementation of the experimental QUIC and HTTP/3 transport protocols.

Nginx released version 1.14.0 of their Nginx Unit dynamic application server, providing additional features and bug fixes. Nginx also released a minor bugfix update for the open source edition of the main nginx web server product.

The Apache Tomcat project has released updated versions for each supported release of its Java HTTP server and Servlet container software. Versions 9.0.30, 8.5.50 and 7.0.99 include various fixes and updates.

Continue reading

In the December 2019 survey we received responses from 1,268,289,402 sites across 243,753,534 unique domain names and 9,430,985 web-facing computers. This reflects a loss of 40.1 million sites, but a gain of 19,200 domains and 156,000 computers.

nginx gained the largest number of sites (+41.2 million), while Apache lost 50.1 million and Microsoft lost 17.5 million; but in terms of unique domains, all major vendors suffered losses. Microsoft lost the most domains (-2.67 million), followed by Apache (-572,000); and, despite gaining so many sites, nginx lost 422,000 domains.

nginx has continued its strong and steady growth in web-facing computers, this month gaining 58,500 computers compared with Apache's gain of 42,900. nginx is edging ever-closer to Apache's leading market share, which fell by 0.13 percentage points to 35.3%, leaving it only 3.4 points ahead of nginx.

If current trends continue, nginx looks set to overtake Apache at some point in late 2020, which would make it the largest vendor in terms of web-facing computers. nginx has already been in the lead by sites since April 2019, and is also closing in on domains, but is still some way behind Apache in active sites and the top million sites.

Vendor news

OpenLiteSpeed 1.6.4 was released on 18 November. This is the open source version of the LiteSpeed web server, which was first to ship HTTP/3 support in July, and boldly claims to outperform nginx when handling HTTP/3 traffic. More than 20 million sites across 3.9 million domains are running either LiteSpeed or OpenLiteSpeed - both variants exhibit the same "LiteSpeed" server header.

QUIC and HTTP/3 have been supported by Cloudflare's partly nginx-based edge network since September; and in October, Cloudflare released a patch that allows its savoury implementation of these protocols (quiche) to be integrated into the upstream nginx codebase. LiteSpeed's HTTP/3 benchmarks were carried out against this patch.

On 19 November, Microsoft announced the availability of Microsoft Azure from its new cloud datacenter regions in Norway. Norway's largest financial services group, DNB Group, has already chosen the new region to migrate services into the cloud in accordance with Norwegian data handling regulations.

Finally, NGINX Plus Release 20 (R20) was released on 3 December. Based on the open source nginx web server, NGINX Plus is a commercial offering, and R20 includes real-time monitoring and logging of rate-limited traffic, enhancements to connection limiting, and some security improvements for HTTP/2.

Continue reading

In the November 2019 survey we received responses from 1,308,343,327 sites across 243,734,379 unique domain names and 9,274,744 web-facing computers. This reflects a gain of 7.46 million sites, 2.18 million domains and 230,000 computers.

Amongst the largest web server vendors, Microsoft, nginx, and Google all gained sites this month, increasing by 19.41 million (+10.59%), 10.17 million (+2.38%), and 1.33 million (+3.83%) sites respectively, whilst Apache lost 13.56 million (-3.64%) sites.

While Microsoft gained substantially the most sites overall, this translated to a relatively small increase of domains (+13,000). As a result, it saw the largest loss of domain market share, dropping by 0.18pp. Apache similarly lost market share, with a drop of 0.16pp, despite a large increase in domains (+268,000).

nginx fared better, with an increase in domains (+522,000) almost double that of Apache’s leaving its market share largely unchanged. nginx also saw large gains in web-facing computers (+113,000) and amongst the top million busiest sites (+1,000) giving it the largest market share increases in those metrics.

All of the largest web server vendors were however outpaced this month in domain gains by both Cloudflare and OpenResty, which increased by 1.06 million and 558,000 domains. While the survey tracks these products separately, OpenResty makes uses of the nginx core, integrating it with additional Lua-based modules, while Cloudflare’s own server software was originally based on nginx. The survey now finds a total of 18.22 million domains for the Cloudflare web server software, and 7.10 million for OpenResty.

Continue reading

In the October 2019 survey we received responses from 1,300,884,420 sites across 241,553,033 unique domain names and 9,045,029 web-facing computers. This reflects a gain of 9.71 million sites, 421,000 domains and a loss of 23,000 computers.

Amongst the largest web server vendors, only nginx and Google gained sites this month with the two vendors increasing by 5.67 million (+1.34%) and 1.80 million (+5.45%) sites respectively, whilst Apache and Microsoft lost 2.14 million (-0.57%) and 6.77 million (-3.56%) sites.

nginx was in fact the only major web server vendor to see increases in most metrics this month, with gains in domains (+480,000), web-facing computers (+12,000), and top million websites (+472).

Apache saw substantially the largest loss of domains, dropping by 762,000 (-1.04%), with Microsoft following with a smaller drop of 125,000 (-0.26%). These losses were largely offset by increases in domains using other smaller vendors, such as Cloudflare and Tengine, both of which are based on nginx.

In terms of web-facing computers, Microsoft saw the largest loss of 56,000 (-3.46%), followed by Apache which dropped 15,000 (-0.46%). A large proportion of Microsoft's loss came at hosting provider Cloud Innovation which appeared to switch many sites to Tengine.

Continue reading

In the September 2019 survey we received responses from 1,291,178,101 sites across 241,131,705 unique domain names and 9,068,313 web-facing computers. This reflects a gain of 19 million sites, 1.69 million domains and 119,000 computers.

All major vendors gained sites this month. The largest gain was for nginx with an increase of 20.6 million sites, followed by Microsoft (+2.9 million), Google (+2.1 million) and Apache (+462,000). This extends nginx's lead as the largest web server vendor by number of sites; it gained 1.12 percentage points taking it to a 32.7% market share. nginx also showed the largest gains in number of unique domains and web-facing computers.

The largest gain within the top million sites this month was by LiteSpeed, which also saw gains in hostnames, domains, and web-facing computers. The September survey saw 1,422 more sites within the top million using this light-weight Apache alternative, an 8.0% increase. This was accompanied by increases of 480,000 sites (+2.6%), 326,000 domains (+9.4%) and 1,665 web-facing computers (+8.1%).

There are losses in market share for both Apache and nginx as the largest server vendors by number of active sites. Apache lost 22,000 active sites while nginx gained 915,000; due to large gains elsewhere this amounted to Apache losing 0.94pp and nginx losing 0.11pp. Google gained 800,000 active sites and 0.16pp of market share to retake third place from Cloudflare; Cloudflare gained 591,000 sites. The largest increase of active sites was in sites running openresty with an increase of 1.04 million.

Web Server Releases

Apache 2.4.41 was released on August 14th bringing several security fixes. This is the first release of Apache 2.4 since 2.4.39 was released on April 1st.

OpenLiteSpeed released a major new feature in version 1.6.0 on September 10th adding support for QUIC and HTTP/3 as well as a new one-click build tool and support for more platforms.

Both OpenResty and Tengine released versions incorporating the nginx patches that fix the HTTP/2 related security issues discussed in last month's blog. OpenResty version 1.15.8.2 was released on September 8th and Tengine 2.3.2 released on August 20th.

Continue reading

In the August 2019 survey we received responses from 1,271,920,923 sites across 239,441,736 unique domain names and 8,948,887 web-facing computers. This reflects a large loss of 124 million sites, but a gain of 1.30 million domains and 10,700 computers.

All major vendors lost active sites this month, and of those, only Google made a gain in sites (+1.58 million). Microsoft lost the largest number of active sites (-2.03 million), while nginx lost the most sites (-81.4 million, -16.9%) but remains in the lead with a 31.6% share of all sites.

Despite losing so many sites, nginx showed the strongest growth in unique domains, web-facing computers, and among the top million sites. This bears more significance than the more unpredictable changes in the site counts, which are prone to fluctuations month-on-month as link farms, spam networks and other low-value web content comes and goes.

With a gain of 58,500 web-facing computers, nginx now has more than 31% of the computer market share – just 5.39 percentage points behind Apache – while Microsoft has lost 65,000 computers. As is evident in the graphs, counting web-facing computers provides the most stable metric and makes long term trends easy to spot. In particular, the clear and consistent rise in nginx's market share and the steady decline of Apache makes it hard not to imagine nginx taking the market lead from Apache by early next year.

The number of top-million websites powered by nginx has increased by 1,292, while Apache's count fell by 3,101. Apache maintains the lead in this market, but is now only 5.92 percentage points ahead of nginx. Apache also continues to lead in terms of unique domains, despite losing 784,000 this month. It has a similar lead over nginx, which is now only 5.32 percentage points behind Apache after gaining 753,000 domains.

Microsoft lost counts in almost all metrics this month, apart from where it gained 166,000 domains, although this still resulted in a small drop in its domain market share. The sites market is the only one where its share did not fall, despite losing 16.6 million sites.

Netflix finds nginx vulnerabilities

nginx 1.61.1 stable and nginx 1.17.3 mainline were released on 13th August, in order to address three HTTP/2 security issues that could cause excessive memory consumption and CPU usage. All versions between 1.9.5 – 1.17.2 are affected, but only if HTTP/2 is enabled. These security issues were discovered by Jonathan Looney at Netflix, which chose to use nginx when developing its own globally distributed content delivery network, known as Netflix Open Connect.

The content delivery network consists of Open Connect Appliances, which run the FreeBSD operating system and use nginx to stream audio and video directly to Netflix customers. Most of this content is served from appliances hosted by ISPs, rather than across the internet, which leads to better performance whilst vastly reducing the amount of peered traffic when huge numbers of customers worldwide stream a popular show at the same time. Thousands of ISPs have enthusiastically participated in this program because it is free to connect to the Open Connect network, and it prevents Netflix traffic from taking up a significant amount of an ISP's internet capacity.

FreeBSD is dying?!

Netflix chose FreeBSD for its balance of stability and features (as did Netcraft once upon a time), but it is becoming an increasingly less common frontend operating system on the web as a whole. Only 60,200 (0.67%) web-facing computers are running FreeBSD today. To put this into perspective, more than twice as many servers are still running Windows Server 2003, even though it has not been supported for several years.

Linux is by far the most commonly used operating system for web-facing computers. It is installed on 6.64 million (74.2%) servers, and at least 1.05 million of these can be positively identified as running the Ubuntu distribution.

Naturally, the choice of operating system depends to some extent on what type of web server will be running on it, and vice versa. For example, it is no surprise that most instances of Microsoft IIS can be found running on Windows Server, and most instances of Windows Server are used to run Microsoft IIS; but it is clear that the Linux operating system is especially favoured for some web servers. Between 92% and 96% of all web-facing computers that use each of nginx, Apache, Litespeed and lighttpd can be found running Linux.

AWS ELB overtakes Beaver

The awselb (Amazon Web Services Elastic Load Balancing) web server was found on 69,800 web-facing computers this month, overtaking Beaver to become the fourth most commonly used frontend server by computers. Practically all of these machines appear to be running Linux, and are responsible for hosting 464,000 sites across 48,500 unique domains.

ELB achieves fault tolerance and scalability by automatically distributing incoming application traffic across multiple targets – and can even spread it across multiple AWS Availability Zones – so the 69,800 AWS ELB servers exposed to the internet are likely to be only the tip of the iceberg in terms of the AWS infrastructure used by each website.

Continue reading