In the July 2019 survey we received responses from 1,395,897,118 sites across 238,145,990 unique domain names and 8,938,144 web-facing computers. This reflects gains of 62.5 million sites, 1.15 million domains, and 98,000 computers.

nginx continues to extend its lead at the top of the list of web server vendors by total number of sites: it has gained 43.3 million sites since the June 2019 survey, bringing its total to 483 million. It now leads second-place Apache by 6.84 percentage points.

nginx has achieved the milestone of serving more than a third of sites in the survey. It becomes the fourth web server to achieve this in the 24 years since Netcraft's Web Server Survey began in August 1995. At that time NCSA [archive.org] - one of the very earliest web servers - served 53% of all sites. NCSA was quickly replaced by Apache, which passed the one-third milestone in June 1996 and continued to serve more than a third of sites until February 2016. Microsoft have served more than a third of sites for four separate periods most recently falling below a third in January 2019.

Unusually, nginx did not fare as well in any of the other metrics this month, losing out in both absolute numbers and market share in terms of domains (-4.0 million, -1.81pp), active sites (-1.2 million, -1.00pp) and in the top million busiest sites (-9,300, -0.93pp). The drops in domains and active sites are accounted for by large changes at two hosting providers; 2.8 million domains hosted by Endurance International Group moved from using nginx to Apache, and 1.5 million domains and 1.4 million active sites hosted by ecommerce provider Shopify now identify as using cloudflare. nginx also lost a small amount of market share of web-facing computers (-0.10pp) despite gaining 21,300 in absolute terms. These losses buck the trend of recent months that has seen nginx gaining market share from Apache and Microsoft.

Apache gained both the largest number of domains and the largest number of active sites since the June survey with increases of 2.2 million and 2.1 million respectively. Microsoft gained the largest number of web-facing computers with an increase of 43,200.

Cloudflare's web server moves up another place into fourth this month after gaining 4.7 million sites to bring its total to 39.9 million. It serves 9.5 million more sites than fifth placed openresty, but stands well behind the 190 million sites served by Microsoft's IIS. The 39.9 million sites served using Cloudflare are spread across 15.2 million unique domains, 2.6 sites per domain, a significantly smaller number of sites per domain than the top three placed web servers. Nginx has 8.2 sites per domain, Apache has 5.3, and Microsoft IIS has 9.3; the total for all sites in the survey is 5.8 sites per domain.

Envoy - the open source edge and service proxy for cloud-native applications, which leapt up to the tenth largest web server by sites in the May survey - has dropped over 200 places and is only seen hosting 13,000 sites in the July survey. This comes as a result of Squarespace sites no longer identifying themselves as using Envoy, but rather announcing "Squarespace" as the web server. Squarespace is the eleventh largest web server by number of sites with 5.2 million sites on 2.8 million unique domains; the seventh largest number of domains.

Continue reading

In the June 2019 survey we received responses from 1,333,421,275 sites across 236,991,999 unique domain names and 8,840,331 web-facing computers. This reflects gains of 6.76 million sites, 1.98 million domains, and 113,000 computers.

nginx has further extended its lead in the hostnames metric, with 52.2 million additional sites pushing its total up to 440 million. Since overtaking Apache in April, nginx has already built up a 4.89 percentage point lead over Apache's market share and is not far off accounting for more than one third of all sites.

Apache retains the lead in all other metrics, but for how long? The proportion of unique domains that are served by Apache has fallen to 30.1%, while nginx snapped up the majority of this month's net domain growth, bringing its share to within 3.4 percentage points of Apache's. There has been a clear trend of Apache's domain market share decreasing, while nginx's increases and looks likely to overtake Apache within a year. A similar trend is evident in web-facing computers and among the top million sites, so it may not be long before nginx steals more of the limelight.

Reports of a remote code execution vulnerability in most recent versions of nginx were downplayed by its developers, who claimed that neither of the two overflow vulnerabilities appeared to be generally exploitable. Both bugs affected the njs scripting language implementation that allows nginx's functionality to be extended; one bug was fixed in njs 0.3.2, which was released on 21 May along with nginx 1.17.0, while the other bug is expected to be fixed in the future release of njs 0.3.3.

Several other product updates were also released during May, including NGINX Unit 1.9.0, lighttpd 1.4.54, LiteSpeed Web Server 5.3.8 (Stable), OpenLiteSpeed 1.5.1, and cPanel & WHM 80. The latest version of cPanel & WHM adds the ability to run Node.js applications, manage API tokens to allow resellers and third-party developers to run API functions, and ensure visitors are redirected to the secure HTTPS versions of sites.

Envoy – the open source edge and service proxy for cloud-native applications, which leapt to prominence in last month's survey – remains in tenth place with around 5 million sites, which are spread across 2.7 million unique domain names.

Cloudflare's web server has moved up into fifth place after gaining more than 5 million websites, giving it a total of 35.1 million sites across 12.9 million domains. The Cloudflare server is used by the edge nodes of Cloudflare's content delivery network, which now serves 2.6% of all sites in this month's survey.

Microsoft saw losses in every metric this month, apart from web-facing computers, where it made a gain of 13,600; however, this was not enough to stop its market share falling slightly, while Apache gained 17,300 computers and nginx gained 64,000.

Continue reading

In the May 2019 survey we received responses from 1,326,664,693 sites across 235,011,143 unique domain names and 8,726,985 web-facing computers. Although this reflects a gain of 1.12 million domains and 113,000 computers, there has been a loss of 119 million sites.

This month's relatively large drop in sites (-8.2%) includes a 10.3 million reduction in the number of websites that are served by nginx, just a month after it became the first vendor other than Microsoft and Apache to serve the largest number of websites over the past 23 years. As Apache lost only 696,000 sites this month, nginx is now only 1.73 million sites ahead, with a market share of 29.20% compared with Apache's 29.07%.

The sites metric has been particularly volatile for Microsoft, which was within two percentage points of Apache's share last month; but this month, it suffered a significant loss of 112 million (-30.8%) sites, leaving it more than 10 points behind with a market share of 18.9%.

Despite losing more than 10 million sites, nginx has outperformed every vendor in all other headline metrics this month – this includes a gain of 939,000 active sites, 1.06 million domains, 63,800 web-facing computers, and an additional 2,120 sites within the top-million websites. Apache continues to lead in all of these metrics, while nginx is in second place and closing, increasing its market share while Apache's declines.

Apache lost 4,330 entries from the top million sites this month, decreasing its share of that market to 31.8%, but leaving it still more than 5 percentage points ahead of nginx. Some of the highest-traffic users of Apache include FedEx, Orange, Slack, Adobe and Ubuntu, while prominent users of nginx include the likes of DuckDuckGo, the BBC, GitLab, and the bit.ly short link service.

The highest-traffic site, www.google.com, uses the in-house gws (Google Web Server), which is also used by many other top-million Google websites, including Google Maps and dozens of country-specific variants of the main search site – like google.de and google.nl.

Envoy

A relatively unfamiliar server called envoy has suddenly leapt into 10th place by sites after experiencing 500-fold growth, increasing its website count from just 10,300 to 5.10 million sites across 2.71 million distinct domains. The majority of these sites are hosted by SquareSpace, which provides easy-to-use website and online store building services that feature drag-and-drop layouts with ready-made templates.

Envoy is an open source edge and service proxy designed for cloud-native applications. It was originally built by the transportation network company Lyft, but Squarespace is now by far the most visible user of the product. Squarespace engineers pursued a self-service infrastructure with Kubernetes to handle the complexity of their software, but to keep up with growth and demand, they started integrating the Envoy proxy into their system more than a year ago to build a service mesh control plane – a policy that turns a set of isolated stateless sidecar proxies into a distributed system.

While this is the first survey in which Envoy has been seen en masse at Squarespace, it is possible that they have already been using it for a number of months without revealing the envoy server header.

cPanel

This month's survey also saw cPanel advance to 9th place with a total of 7.35 million sites. cPanel is a Linux-based web hosting control panel that provides a graphical interface for administering websites via a secure service on port 2083. Most cPanel-administered websites exhibit the bare cPanel server banner on HTTP port 80, but there are also 5.77 million Apache-powered sites that reveal the use of cPanel via Server headers similar to Apache/2.4.39 (cPanel) OpenSSL/1.0.2r mod_bwlimited/1.4.

While the cPanel interface is aimed at individual end users or hosting customers, the associated WHM (WebHost Manager) interface allows hosting providers to manage large numbers of cPanel user accounts and add custom branding to their customers' dashboards. A slightly cheaper cPanel Solo product also provides most cPanel and WHM features to hosting providers or individuals with only a single server account to manage. These products have led to cPanel being found across a variety of hosting locations in more than 150 countries, ranging from more than 400,000 cPanel-powered sites operated by a single French hosting company, to a single cPanel-powered website in the whole of Zambia.

New web server releases

• nginx 1.16.0 was released on 23 April 2019. In keeping with nginx tradition, this first release in the 1.16.x stable branch includes all of the new features and bug fixes that were introduced in the 1.15.x mainline branch – this means it has improved UDP proxying, support for TLS 1.3 early data and dynamic loading of TLS certificates amongst other features.

• Envoy 1.10.0 was released on 5 April 2019, enabling TLS 1.3 on the server-side, and making a multitude of other changes and additions. The previous release, 1.9.1, addressed two vulnerabilities – CVE-2019-9900 and CVE-2019-9901– both of which could have allowed remote attackers to bypass access control rules.

• GoDaddy is currently using DPS 1.6.0 to serve customer sites that have been created with its Website Builder tool. This server software continues to be updated fairly frequently: the sites were using DPS 1.5.11 when this month's survey data was originally collected, and DPS 1.5.7 during last month's survey.

Continue reading

In the April 2019 survey we received responses from 1,445,266,139 sites across 233,886,577 unique domain names and 8,613,630 web-facing computers. This reflects a loss of 16.8 million sites, but a gain of 1.72 million domains and 87,000 computers.

Most websites now use nginx!

Despite the overall loss of sites this month, nginx gained 22.3 million websites and 2.03 million additional active sites. nginx also gained the largest number of web-facing computers, increasing its total by 63,000 to 2.57 million (+2.52%). nginx's market share of web-facing computers is now nearly 30%, and this is continuing to grow steadily closer to Apache's leading share of 37.3%.

Microsoft and Apache lost shares in every headline metric this month, with both vendors contributing significantly to this month's overall loss of sites. Microsoft lost 18.9 million sites, while Apache lost 17.2 million, causing their shares to decrease by 1.01 and 0.87 percentage points.

These changes have pushed nginx into the lead, giving it a 27.5% share of all sites in Netcraft's April 2019 Web Server Survey. Significantly, this is the first time since 1996 that a vendor other than Microsoft or Apache has served the largest number of websites.

The reign of "a patchy" web server

If we cast our minds back to early 1996 (not long before the arrest of the Unabomber and the release of the Spice Girls' first cassette tape single), the most commonly used web server software was still NCSA HTTPd. This long-since discontinued web server was originally developed three years earlier by Rob McCool at the National Center for Supercomputing Applications; but when he left the NCSA in mid-1994, development stalled, prompting many webmasters to develop their own extensions and bug fixes.

A small group of these developers coordinated their changes in the form of patches, eventually leading to the foundation of the independent Apache Group. By April 1995, the group had made its first official public release of the Apache web server (version 0.6.2), which was based on NCSA HTTPd 1.3.

Although the NCSA resumed development of its own web server around the same time, Apache quickly took hold of the market. Exactly a year after Apache's first public release, Netcraft's April 1996 Web Server Survey showed that Apache had succeeded in overtaking NCSA's lead, marking the start of a remarkable uninterrupted 18-year reign.

The non-profit Apache Software Foundation, was later formed in 1999, and today is home to hundreds of other projects in addition to the Apache HTTP Server. Microsoft briefly took the lead from Apache in 2014, and retook the lead from 2016 until being beaten by nginx this month.

However, for now, Apache remains on top of every other headline metric: It leads with a 30.3% share of active sites, 37.3% of all web-facing computers, 31.5% of all domains, and it serves 32.2% of the top million websites. Nonetheless, nginx's strong and consistent growth makes it look set to challenge Apache's lead within a year or two. Most noticeably, it is rapidly catching up with Apache's declining share of web-facing computers, and may also soon threaten Apache's share of the top million websites.

New web server releases

There have been several new releases over the past month:

• Apache 2.4.39 was released on 1 April. As usual, this latest release in the 2.4.x stable branch is regarded by the Apache Software Foundation to be the best available version; but more importantly, it resolves several security issues including an access control bypass and privilege escalation vulnerability.
• nginx 1.15.10 mainline was released on 26 March, adding some new directives and certificate features. This was followed by two bugfix releases: nginx 1.15.11 on 9 April, and nginx 1.15.12 on 16 April.
• njs 0.3.1 (the JavaScript-like scripting language that allows nginx functionality to be extended) was released on 16 April, adding several new features, including support for arrow functions.
• OpenLiteSpeed 1.4.46 was released on 5 April, adding support for PHP7 and app servers that use nodeJS, Python and Ruby.
• Taobao's Tengine 2.3.0 was released on 25 March. This development version of the open source nginx fork inherits all features from nginx 1.15.9 and includes several new features, changes and bugfixes.

Tengine Aserver

Visible uptake of the latest Tengine 2.3.0 development version is likely to be slow. The most commonly used numbered version is still 2.2.0, despite there being four newer development versions since its release more than three years ago; and development version 1.4.2, which was released in 2012, is not far behind.

Tengine 2.2.0 is currently used by 13.9% of all Tengine-powered websites, but the majority – 42.4% – do not reveal a version number at all, and a further 29.8% respond with the Tengine/Aserver server header. Nearly all of the websites that use Tengine Aserver are online stores hosted on subdomains of taobao.com and the domain of its parent company, alibaba.com, suggesting that it is a custom version designed specifically for these applications.

Development versions of Tengine appear to be far more popular than stable releases. The most recent stable version, Tengine 2.1.2, was released in December 2015 and is used by only 0.26% of all Tengine-powered websites.

Continue reading

In the March 2019 survey we received responses from 1,462,021,378 sites, 232,162,099 unique domains, and 8,526,624 web-facing computers. This reflects a loss of 15.8 million sites, but a gain of 2.57 million domains and 160k web-facing computers.

The March survey brings major upheaval in the domains metric. A large 8 million domain switch from Microsoft to nginx at GoDaddy has allowed nginx to overtake Microsoft to gain second place. Many of these freshly-switched domains do not host distinct content, instead redirecting elsewhere. Accordingly, nginx's share of active sites increased by a more moderate 3.3%. The total number of domains using nginx grew by 15.4%, a 3.24 percentage point increase of market share. Microsoft lost 3.95 p.p. Both remain some way behind Apache in the domains metric, which is 13.8 million domains ahead of nginx.

The largest three vendors - Apache, Microsoft, and nginx - showed at least minor increases in the number of active sites using their respective platforms this month. However, there has been an overall decrease in active site numbers over the past year. Apache in particular has seen a significant drop in active sites of around 20 million. In contrast, the LiteSpeed web server has seen more stable growth from month to month, having increased 960k (+31.7%) over the past year. Cloudflare’s server software, which had previously identified itself as a flavour of nginx prior to 2018, also demonstrated significant growth, serving content for 4.54 million more active sites (+65.6%) than it did a year ago.

Within the top million busiest sites, nginx continues to increase its share of the market, with a further small increase of 0.078 p.p. this month. Apache had its first monthly increase since September 2013, gaining back 0.051 p.p. Apache has been on a downward trend since late 2011 when it held around a 66% majority. Despite this, it has still remained the largest vendor amongst the million busiest sites, despite its market share halving to 32.5% as of March 2019. Microsoft servers have also seen a general decrease of market share over time, and its share has fallen to just under 9% this month.

F5 Acquires NGINX

On March 11th, F5 Networks, Inc. announced its acquisition of NGINX, Inc., the developer of the open source nginx web server. A similar announcement was made by NGINX on their website the same day. The acquisition comes with a $670 million price tag.

F5 and NGINX both provide rival sets of products, although the two have traditionally targeted different communities. F5’s BIG-IP line-up provides a variety of specialised products and service-based solutions for application delivery networking. Much of the key functionality provided by BIG-IP products, such as reverse proxying and load-balancing, overlap with the feature set of NGINX Plus, an exclusively software-based product providing a load balancer, web server, and content cache, along with exclusive features not included in the open-source edition of nginx. NGINX has made numerous blog posts and guides promoting its NGINX Plus product over F5’s BIG-IP, with competitive pricing and greater developer agility being common points of discussion.

In March 2019, Netcraft identified 12.6 million domains being served by F5 BIG-IP systems. Most of these BIG-IP systems are used in conjunction with other web server software.

Continue reading

In the February 2019 survey we received responses from 1,477,803,927 sites, 229,586,773 unique domains, and 8,366,753 web-facing computers. This reflects a loss of 40.4 million sites, and gains of 979k domains, and 157k web-facing computers.

Microsoft experienced the largest gain in domains this month, with a net increase of just over one million. Despite several months of relatively small fluctuating gains and losses, the number of domains running Microsoft server software is on a general upward trend. Its total now stands at 59.3 million domains, up by just under 2.6 million (+4.6%) since this time last year. However, this strong domain growth was not reflected in any other metric this month – Microsoft gained only 478 web-facing computers, made losses both in active sites and within the top million sites, and suffered a sizable loss of 65 million hostnames.

On 20 February, Microsoft published a security advisory regarding a potential denial of service vulnerability in IIS. It can be exploited by sending specially crafted HTTP/2 requests to a Microsoft IIS web server, causing CPU usage to spike to 100% until the malicious connections are killed by IIS. Microsoft has addressed this issue in February's "non-security" update by providing the ability to define limits on the number of HTTP/2 settings parameters allowed over a connection.

nginx's growth in the domains metric was some way behind Microsoft's this month, with an increase of 622k. nginx's current total of 52.6 million domains represents a growth of over 8.6 million domains (+19.1%) since February 2018 – over three times that of Microsoft – however, its market share has remained just under 3 percentage points behind Microsoft's since April 2018. In terms of web-facing computers, nginx experienced the largest increase (+102k), continuing its steady gains in market share. It now holds a 29.0% share of the web-facing computer market with a total of 2.4 million.

The latest version of nginx (1.15.9 mainline) was released on 26 February, with some small changes including two new features and two bug fixes. Another product in the nginx family, NGINX Unit 1.7.1, was also released in February to address a security vulnerability in its router process. NGINX Unit is a lightweight web application server that can serve sandboxed Go, Perl, PHP, Python, Ruby and – soon – Java applications.

Apache is still losing domains, with a decrease of 216k this month, and 7.3 million domains over the past year. Apache is also gradually losing market share in terms of web-facing computers, despite an overall increase in the number of public web servers using it: this month, Apache gained 32.9k web-facing computers, taking its total up to 3.2 million, while competitor growth caused its market share to fall to 38.04%. The latest version, Apache 2.4.38, was released on 22 January. This release in the 2.4.x stable branch is regarded as the best available version of Apache, and includes three security fixes and multiple bug fixes.

Continue reading