In the October 2017 survey we received responses from 1,815,237,491 sites and 6,886,362 web-facing computers, reflecting a gain of 10.2 million sites and 88,300 computers.

Web-facing computers: nginx takes second place from Microsoft

nginx made the largest gains in websites, active sites, and web-facing computers this month, as well as increasing its presence among the top million sites. Most notably, the additional 42,100 web-facing computers it gained has taken its total up to 1.55 million computers, putting it ahead of Microsoft for the first time.

Overtaking Microsoft means that nginx is now the second largest server vendor in terms of web-facing computers. With its remarkably consistent growth, nginx is likely to retain this newfound position for some time – not least because Microsoft's web-facing computer share has been on a general decline since 2010.

In the other metrics, nginx gained 18.4 million sites, 941,000 active sites, and slightly increased its share of the top million sites to 29.43%. It stays ranked in 2nd place within the top million sites and active sites, but 3rd in all sites.

While Microsoft's loss of 3,470 web-facing computers helped propel nginx into second place, it made more significant losses in other metrics – it lost 30.1 million sites this month, although this corresponds to a loss of only 85,500 active sites.

New releases

Apache 2.4.29 was released on 23 October. This security, feature and bug fix release represents the latest version of the current 2.4.x branch. As usual, it is recommended over all previous releases, but it is difficult to track how many website administrators take heed of this advice.

For instance, many Apache servers do not reveal via their Server headers or error pages which version has been installed, while others may have been updated with backported patches that do not affect the displayed version number. Consequently, a large number of Apache servers claim to be running older versions than they really are. Only 12.6% of the 341 million sites running Apache claim to be running a 2.4.x release, whereas the true proportion is likely to be much higher, given that almost two-thirds of Apache-using sites do not disclose any version number.

Apache continues to lead the market in terms of active sites and web-facing computers, where it has market shares of 44.5% and 42.3%. It also has the largest presence among the top million sites, with 386,000 of these using Apache.

Another new release this month was LiteSpeed Web Server 5.2.2 (stable), which was released on 17 October. This addresses a couple of bug fixes and improves compatibility with the latest version of the popular cPanel web-based control panel.

As well as its commercially supported LiteSpeed Web Server, LiteSpeed Technologies Inc also provides OpenLiteSpeed, which is freely available under the GPL version 3 licence. LiteSpeed is currently the 7th largest vendor in terms of hostnames and active sites: Nearly 11.5 million sites in the survey are powered by LiteSpeed, and 2.7 million (24%) of these are deemed to be active sites.

One of LiteSpeed's most prominent gains was made in November last year, when a large number of hostnames under the .science top-level domain switched to it from Taobao's Tengine web server. This caused LiteSpeed's market share of sites to leap from 0.39% to 3.29%, although it has since settled back down to 0.63%. Nonetheless, this is still noticeably larger than its share of web-facing computers, which currently stands at 0.17%.

nginx 1.13.6 (mainline) and nginx 1.12.2 (stable) were also released in October. Both releases consist solely of bug fixes.

NGINX Unit

Alongside the new releases of nginx, the nginx.org homepage unusually announced the release of a different product this month: NGINX Unit 0.2 Beta.

NGINX Unit is a lightweight dynamic web application server designed to run applications written in Python, PHP, Go, JavaScript, Java and Ruby, although the current beta release does not support all of these languages, nor does it support TLS, routing or proxying yet.

Precompiled binaries are available for CentOS 7 and Ubuntu 16.04, but as it is a beta release, it is not recommended for use in a production environment. Consequently, it is unlikely to have much of a presence on the web in the near future; also, for performance reasons, it is likely that NGINX Unit would be installed behind a regular nginx web server acting as a reverse proxy.

In the September 2017 survey we received responses from 1,805,060,730 sites and 6,798,101 web-facing computers, reflecting a gain of 4.49 million sites and 84,800 computers. This is the largest number of web-facing computers in history, and cloud computing has helped fuel this growth considerably. The top three cloud hosting providers—Amazon, Alibaba and DigitalOcean—now account for nearly 22% of all web-facing computers, as well as more than a third of this month's net computer growth.

nginx has dominated this month's computer growth once again, with an additional 49,900 computers taking its market share up to 22.2%. This puts it within 20,000 computers of Microsoft's 22.5% share, giving nginx a realistic chance of taking second place by next month. Meanwhile, Apache remains far ahead in the lead with 2.88 million web-facing computers giving it a 42.3% share of the market.

While more than half of the websites in the survey are using Microsoft web server software, relatively few of these are active sites. Discounting link farms, domain holding pages and other automatically generated content, Microsoft accounts for only 7.3% of all active sites, while Apache leads with 44.9%, and nginx follows with 20.7%. Microsoft's active sites share has never exceeded Apache's, and ever since it peaked at 38% in early 2009, it has experienced a general decline.

Microsoft's presence within the top million websites has also been steadily declining – in 2008, it had a 20% share, but this has since dwindled to less than 10%. Today, 9.73% of the top million sites use Microsoft web server software (mostly IIS – Internet Information Services), while Apache leads with 38.9%. However, Apache has also been experiencing a general decline since 2011, while nginx has continued to muscle its way in. A further 1,299 nginx-powered sites made their way into the top million this month, lifting nginx's share to 29.4%.

Web-facing operating systems

Apache has been the most commonly installed web server for many years, but which underlying operating systems are used by the world's web facing computers?

By far the most commonly used operating system is Linux, which runs on more than two-thirds of all web-facing computers. This month alone, the number of Linux computers increased by more than 91,000; and again, this strong growth can largely be attributed to cloud hosting providers, where Linux-based instances are typically the cheapest and most commonly available.

It is difficult to distinguish different Linux distributions by examining their TCP/IP characteristics, but inferences can be made from other clues, such as Server headers. The Linux distribution can be positively identified in around 30% of cases, and of these 1.39 million Linux computers, just over half are running Ubuntu Linux, nearly a quarter are running CentOS, and around fifth are running Debian Linux.

Microsoft is next up, with 26.8% of all web-facing computers running various versions of Windows Server. Interestingly, this is higher than Microsoft's web server share (22.5%), and can be explained by around 20% of Windows computers being used to run other web server software – such as Apache and nginx – rather than its native Microsoft IIS web server. Conversely, around 51,000 Linux computers expose Microsoft IIS web servers to the internet – in these cases, the Linux frontends are typically acting as load balancers or reverse proxies to backend Windows computers running IIS.

Windows Server 2008 is the most commonly used version of the Windows Server family of operating systems – it currently runs on 971,000 web-facing computers, and this is followed by Windows Server 2012, which is installed on 505,000.

Windows and Linux are essentially the "big two" when it comes to web-facing operating systems. FreeBSD was once notorious for its reliability and impressively large uptimes when used as a server platform; indeed, Netcraft's infrastructure made extensive use of it in the past, but it is now a relatively niche operating system compared with its heyday. Today it is used by only 1.3% of web-facing computers, more than half of which are being used to run the Apache web server.

In the August 2017 survey we received responses from 1,800,566,882 sites and 6,713,340 web-facing computers. This represents a gain of 32.6 million sites (+1.84%) and 120,000 computers (+1.82%).

Continuing its consistently strong performance, nginx once more gained the most web-facing computers, increasing its total by 50,800 (+3.6%). Second-place Microsoft also gained computers (+5,000, +0.3%), albeit much more modestly. As a result, its market share dipped 0.34 percentage points. Apache gained 32,200 web-facing computers (+1.1%), but also saw its share fall, dropping 0.28 percentage points to 42.5%.

The gap in the market share of web-facing computers between nginx and Microsoft has narrowed to 1.13 percentage points. If current trends continue, the number 2 position should be well within nginx's reach before the end of 2017. However, with an almost 20-percentage-point lead over both nginx and Microsoft, market-leader Apache is unlikely to be dethroned anytime soon.

nginx also continues to gain market share amongst the million busiest sites, with 2,200 more top sites seen using the web server this month. Apache's dominant position continues to gradually erode, dropping by 1,700 sites in August. nginx is still a distant second, but for the first time is fewer than 100,000 sites behind. Third-place Microsoft also continues a gradual decline, and after losing a further 980 sites from the top million this month its market share now stands at 9.80%.

Turbulent times for .xyz

.xyz experienced the largest growth among all TLDs this month, gaining websites on over 761k unique domains (+22%). This brought the 7th largest TLD up to over 4 million domains hosting web content. However, it is unlikely to maintain this position for long, with the .xyz zone file losing over 44% of its registered domains shortly after the August survey ran.

The .xyz top-level domain was launched to the public in June 2014 and had reached almost 100,000 domains within the first 10 days after a Network Solutions promotion which gave free matching .xyz domains with each .com domain purchased. This was done on an opt-out basis, with the domains only being free for the first year, causing some confusion the following year when each domain became due for renewal at a cost of $38.

gen.xyz ran another promotion in June 2016 to celebrate its 2 year anniversary. For 2 days .xyz offered domains for just 2¢ (including the ICANN fee), leading to the TLD more than doubling in size after nearly 2.7 million domains were registered. Unsurprisingly, just over one year on, this has led to a dramatic drop in the number of registered .xyz domains; fewer than 1% of the domains created during the event are still registered.

In the July 2017 survey we received responses from 1,767,964,429 sites and 6,593,508 web-facing computers. This represents a small gain of 1.0 million sites (+0.06%) and 71,000 computers (+1.1%).

nginx growth unfaltering

A further 52,000 (+3.84%) web-facing computers were found running nginx this month, which has brought its market share up to 21.4%. It is currently the third-largest server vendor in terms of web-facing computers, but it is now only 122,000 computers away from Microsoft. With no reason to suspect that its consistently strong growth could falter soon, it is likely to take second place from Microsoft later this year.

Originally developed to solve the C10k problem, nginx has seen phenomenal growth in web-facing computers.

nginx's market share growth was also assisted by Microsoft's loss of 6,400 computers, while Apache's gain of 7,500 computers was not enough to stop its own share falling by 0.35 percentage points. Apache is still far in the lead, though – more than 2.8 million web-facing computers currently run various versions and derivatives of the Apache httpd, giving it a 42.8% share of all web-facing computers.

Microsoft now serves more than half of all sites

In terms of hostnames, Microsoft gained 78 million sites, while Apache lost 56 million. This large shift has given Microsoft more than half of the market for the first time ever – 53.2% of all hostnames – with nearly three times as many sites as Apache has.

This marks a complete role reversal from four years ago, when Apache held 52% of the market while Microsoft had just 19.7%. That was the last time more than half of the world's websites used Apache. However, the hostnames metric is volatile, being susceptible to large swathes of automatically generated sites served from relatively few computers. These types of site are not counted in Netcraft's active sites survey, which paints a very different picture: Apache has more than six times as many active sites as Microsoft, and more than twice as many as nginx.

Notably, Apache has always held the largest share of the active sites market ever since the metric was included in our surveys in 2000. While Microsoft came within 10 percentage points of Apache's share on a few occasions last decade, it is now a long way off with only a 7.48% share, compared with Apache's share of 45.2%.

Apache 2.2 reaches end of life

Apache 2.4.27 was released on 11 July, along with Apache 2.2.34, which will be the final release in the 2.2.x legacy branch. Security patches for Apache 2.2.34 may be made available until December 2017, but no further maintenance patches or releases are anticipated.

To remain secure, sites still using Apache 2.2 will need to migrate to Apache 2.4 fairly swiftly. While it is difficult to tell exactly how many sites are running soon-to-be unsupported versions of Apache 2.2, at least 72 million sites claim to be using Apache 2.2 in their Server headers. The majority of these sites are hosted in the United States.

On the same day as the Apache releases, nginx 1.12.1 stable and nginx 1.13.3 mainline were released, with both including a simple fix for an integer overflow vulnerability in nginx's range filter module.

Not to be outdone, version 2.0.0 of Microsoft's IIS Administration API was also released in July, little more than two months after 1.1.0 hit general availability. The API is intended to make it easier to manage Microsoft IIS web servers, and the new version includes a range of "under the hood" improvements that make it easier to install and configure. The Microsoft IIS team also released a new version of URL Rewrite and made several improvements to its browser-based management portal at manage.iis.net during June.

In the June 2017 survey we received responses from 1,766,926,408 sites and 6,522,692 web-facing computers. This represents a loss of 48 million websites, although the total number of computers has grown by 118,000 (+1.8%).

All of the top three vendors lost sites in this month's survey, but all showed gains in web-facing computers. Many of the sites that disappeared were spam sites that used the .cn (China) top-level domain. Microsoft lost the largest number of sites – nearly 28 million – while Apache lost 8.9 million and nginx lost 5.4 million.

Apache still leads the market in terms of web-facing computers, but its share fell by 0.28 points to 43.1% despite gaining 33,200 additional computers. Apache also continues to lead the active sites market, and with net growth of 1.2 million active sites this month, its share has increased to 45.9%.

nginx gained the largest number of web-facing computers, increasing its total by 54,200 to 1.36 million (+4.2%), and taking its market share of computers up by 0.46 percentage points to 20.8%. It is now less than 3 percentage points behind Microsoft's share.

nginx is also still increasing its presence amongst the top million websites. This month it gained 939 top sites, in contrast to the losses felt by Apache and Microsoft, which saw 2,970 and 692 sites depart from the top million. Some of the lesser-used servers that also increased their presence in the top million included openresty, Varnish and Tengine.

Google overtakes Microsoft in active sites

Notably, Google has overtaken Microsoft in active sites – it now has 13.3 million, compared with Microsoft's 13.2 million. This gives Google a 7.8% share of the active sites market, although its share of all sites is only 1.1%.

Netcraft first started tracking Google's custom web server software as a major vendor group 10 years ago, when it was used by 2.7 million sites. Google's servers were originally grouped under Apache, as they were based on open source Apache code, but the amount of customisation warranted making a new group – and no doubt even more changes have taken place over the past 10 years. Today, there are over 20 million Google sites, around two-thirds of which are considered active. This is a much higher ratio than most other vendors see – for example, only 1.5% of the 862 million sites using Microsoft's web server software are deemed active.

The most commonly seen Google web server is GSE (Google Servlet Engine), which is used by millions of Blogger sites that use blogspot domains (e.g. funny-cats.blogspot.com and catversushuman.blogspot.ca), and also by many Blogger-powered sites that use custom domain names. GSE is also used by sites under the googledrive.com and googlegroups.com domains, along with some other Google services such as Gmail, although none has the volume of hostnames seen at Blogger.

Another Google web server is Google Frontend, which is used by hundreds of thousands of sites, including App Engine sites hosted under the appspot.com domain. This server was much more prominent in the past, as it was also used by Blogger sites before they switched to GSE. Back then, Google Frontend sites also used an acronym in their HTTP response headers (Server: GFE), but Google Frontend sites now return the full name of the server software, i.e. Server: Google Frontend.

Google Frontend is also used to serve some of Google's legacy sites and deprecated services, such as the former social networking site at jaiku.com. This was shut down by Google in 2012, and all pages on the site now use Google Frontend to serve error pages.

Another Google server – ghs – is responsible for redirecting traffic from googlepages.com sites that were created with Google Page Creator. This website creation service was shut down in 2009, but existing pages were migrated to Google Sites, which hosts user content in subdirectories under the sites.google.com hostname. When a browser visits a legacy hostname like sunsetpizza.googlepages.com, ghs will redirect the user to its new location at https://sites.google.com/site/sunsetpizza/.

Other Server headers used more sparingly by Google-hosted services include UploadServer, sffe, ESF (used by Google Docs), and gws.

In the May 2017 survey we received responses from 1,814,996,345 sites and 6,404,290 web-facing computers. Although the total number of sites has fallen slightly, by 1.4 million since April, the number of computers has grown by 83,000.

Large shifts in site counts

Although it fared well in other metrics (see below), nginx suffered a massive 30% loss of hostnames this month. With a net loss of more than 100 million sites, its market share has fallen by 5.71 percentage points to 13.5%. The majority of the sites that disappeared were Chinese-language spam sites hosted at Amazon Web Services in the United States and Japan.

Meanwhile, Microsoft gained 79 million sites, which has taken its market share up to 49.1%. This is Microsoft's highest market share in the 22-year history of the Web Server Survey, and also reflects a significant change in fortunes over the past year: Last June, Apache had the largest market share of websites, but now Microsoft's share is more than twice as large as Apache's, which fell by a further 1.73 percentage points this month to 20.95%.

nginx leads active site growth

Despite the large loss of hostnames, nginx gained nearly a million active sites, which has taken its active sites share back above 20%. This suggests most of the 104 million hostnames it lost did not have distinct content, and therefore were of little interest to ordinary web users, despite the apparent size of the change.

Apache continues to lead the active sites market quite comfortably with a 45.6% share, although a loss of 1.4 million active sites has brought this down by 0.67 percentage points. nginx stays firmly in second place, with more than twice as many active sites as Microsoft.

nginx's computer growth continues unabated

nginx's consistent computer growth has continued, making it look ever more likely to overtake Microsoft later in 2017. Its gain of 48,500 computers – combined with Microsoft's loss of 6,000 – has reduced the difference in their market shares by nearly a whole percentage point.

nginx is now within 3.46 percentage points of Microsoft's share of 23.8%; but Apache also maintains a comfortable lead in this market – it increased its web-facing computer count by 28,800 this month, keeping Apache's share above 43%.

nginx pushing others out of the top million

nginx is also continuing to make strong progress amongst the top million websites, where it has been ahead of Microsoft for the past few years. It was the only major vendor to increase its presence this month, resulting in thousands of competing vendors' websites being pushed out of the top million. Apache suffered most, with 3,200 Apache-powered sites departing the top million, but it still leads this market with a 40.5% presence.

Apache has exhibited a slow and steady decline over the past several years. Coupled with nginx's consistent growth within the top million sites, the gap between the two is ever decreasing; however, it looks like it will be a good year or two until nginx seriously starts to threaten Apache's lead.

New nginx 1.13 mainline branch

nginx 1.13.0 was released on 25th April. This is the first release on the new, actively-developed 1.13.x mainline branch, adding bugfixes and new features to what was essentially the most recent stable version of nginx (1.12.0).

The most notable new feature in nginx 1.13.0 is its support for TLS 1.3, which aims to be the latest and most secure version of the Transport Layer Security protocol – although not many underlying crypto libraries actually offer TLS 1.3 yet. The TLS 1.3 specification has not yet been finalised, although the working draft has been sufficient for some servers and clients to implement it. For example, Mozilla's NSS cryptographic library – which is used by Firefox – enables TLS 1.3 support by default.

Microsoft IIS Administration API enters general availability

The Microsoft IIS Administration API that we mentioned in February is now Generally Available. The newest 1.1.0 release also facilitates management of the IIS Central Certificate Store, and includes an improved certificate API. These features are intended to make it easier to manage certificates across entire farms of web servers.

LiteSpeed enters the Amazon cloud

LiteSpeed Technologies is now a Technology Partner in the Amazon Web Services Partner Network (APN), coinciding with the release of the LiteSpeed Web Server AMI. This Amazon Machine Image is based on CentOS 7 with the LiteSpeed Web Server pre-installed. AWS users can now use this AMI to quickly deploy ready-to-use virtual machines that run the LiteSpeed Web Server in the cloud.

This month's Web Server Survey found 6.4 million websites running LiteSpeed, which encompasses 2.3 million active sites, more than 2 million unique domain names, and nearly 11,000 web-facing computers. One of the busiest sites currently using LiteSpeed is FanFiction.Net, which is an automated fan fiction archive.

Enter the Beaver

China saw a large number of new sites being served by the relatively unknown "Beaver" web server. Just over a million sites now exhibit the Beaver Server header, and these make use of more than 110,000 unique domain names – mostly under the .cn top-level domain. Most of the sites are hosted by Aliyun, which is China's largest cloud hosting provider, while the majority of the rest are hosted by other Chinese companies. Only a single Beaver site is hosted outside of China – this solitary instance is hosted in Japan, at Amazon's Tokyo AWS region.

The behaviour of these sites suggests that Beaver might not be an entirely new web server, but possibly an application based on Microsoft's HTTP Server API (HTTPAPI 2.0). This API lets C/C++ programmers receive HTTP requests and send responses without using Microsoft IIS. At least 38 million other websites also use HTTPAPI 2.0.

But most of the Beaver sites are currently inaccessible and display the following message: "According to the filing requirements of China's Ministry of Industry and Information Technology (MIIT), the website is accessible only if the ICP information is accurate and the ICP license is filed". An ICP Filing ("Bei'an") is required by all content providers in China before they can use hosting and CDN products, but this only allows them to be used for informational purposes. A Commercial ICP Licence ("Zheng") is required for any website that sells goods or services that directly generate revenue online.

Pepyaka making the web more friendly

The little-known Pepyaka web server has also been quietly growing. This server is used predominantly by Wix, an Israeli company that provides a friendly website building platform to millions of users. Wix provides free website hosting under the *.wix.com domain, or customers can get a free custom domain name when upgrading to one of Wix's yearly premium plans.

This business model has led to a high ratio of domains to hostnames amongst the Pepyaka install base. Wix-hosted websites alone account for 1.4% of all unique domain names in use on the web, which is no mean feat. All of these sites are hosted within the Amazon Web Services cloud, using Pepyaka version 1.11.3. This version number, coupled with Wix's previous uses of nginx, suggests that it could be based on last year's mainline version of nginx.