May 2017 Web Server Survey

In the May 2017 survey we received responses from 1,814,996,345 sites and 6,404,290 web-facing computers. Although the total number of sites has fallen slightly, by 1.4 million since April, the number of computers has grown by 83,000.

Large shifts in site counts

Although it fared well in other metrics (see below), nginx suffered a massive 30% loss of hostnames this month. With a net loss of more than 100 million sites, its market share has fallen by 5.71 percentage points to 13.5%. The majority of the sites that disappeared were Chinese-language spam sites hosted at Amazon Web Services in the United States and Japan.

Meanwhile, Microsoft gained 79 million sites, which has taken its market share up to 49.1%. This is Microsoft's highest market share in the 22-year history of the Web Server Survey, and also reflects a significant change in fortunes over the past year: Last June, Apache had the largest market share of websites, but now Microsoft's share is more than twice as large as Apache's, which fell by a further 1.73 percentage points this month to 20.95%.

nginx leads active site growth

Despite the large loss of hostnames, nginx gained nearly a million active sites, which has taken its active sites share back above 20%. This suggests most of the 104 million hostnames it lost did not have distinct content, and therefore were of little interest to ordinary web users, despite the apparent size of the change.

Apache continues to lead the active sites market quite comfortably with a 45.6% share, although a loss of 1.4 million active sites has brought this down by 0.67 percentage points. nginx stays firmly in second place, with more than twice as many active sites as Microsoft.

nginx's computer growth continues unabated

nginx's consistent computer growth has continued, making it look ever more likely to overtake Microsoft later in 2017. Its gain of 48,500 computers – combined with Microsoft's loss of 6,000 – has reduced the difference in their market shares by nearly a whole percentage point.

nginx is now within 3.46 percentage points of Microsoft's share of 23.8%; but Apache also maintains a comfortable lead in this market – it increased its web-facing computer count by 28,800 this month, keeping Apache's share above 43%.

nginx pushing others out of the top million

nginx is also continuing to make strong progress amongst the top million websites, where it has been ahead of Microsoft for the past few years. It was the only major vendor to increase its presence this month, resulting in thousands of competing vendors' websites being pushed out of the top million. Apache suffered most, with 3,200 Apache-powered sites departing the top million, but it still leads this market with a 40.5% presence.

Apache has exhibited a slow and steady decline over the past several years. Coupled with nginx's consistent growth within the top million sites, the gap between the two is ever decreasing; however, it looks like it will be a good year or two until nginx seriously starts to threaten Apache's lead.

New nginx 1.13 mainline branch

nginx 1.13.0 was released on 25th April. This is the first release on the new, actively-developed 1.13.x mainline branch, adding bugfixes and new features to what was essentially the most recent stable version of nginx (1.12.0).

The most notable new feature in nginx 1.13.0 is its support for TLS 1.3, which aims to be the latest and most secure version of the Transport Layer Security protocol – although not many underlying crypto libraries actually offer TLS 1.3 yet. The TLS 1.3 specification has not yet been finalised, although the working draft has been sufficient for some servers and clients to implement it. For example, Mozilla's NSS cryptographic library – which is used by Firefox – enables TLS 1.3 support by default.

Microsoft IIS Administration API enters general availability

The Microsoft IIS Administration API that we mentioned in February is now Generally Available. The newest 1.1.0 release also facilitates management of the IIS Central Certificate Store, and includes an improved certificate API. These features are intended to make it easier to manage certificates across entire farms of web servers.

LiteSpeed enters the Amazon cloud

LiteSpeed Technologies is now a Technology Partner in the Amazon Web Services Partner Network (APN), coinciding with the release of the LiteSpeed Web Server AMI. This Amazon Machine Image is based on CentOS 7 with the LiteSpeed Web Server pre-installed. AWS users can now use this AMI to quickly deploy ready-to-use virtual machines that run the LiteSpeed Web Server in the cloud.

This month's Web Server Survey found 6.4 million websites running LiteSpeed, which encompasses 2.3 million active sites, more than 2 million unique domain names, and nearly 11,000 web-facing computers. One of the busiest sites currently using LiteSpeed is FanFiction.Net, which is an automated fan fiction archive.

Enter the Beaver

China saw a large number of new sites being served by the relatively unknown "Beaver" web server. Just over a million sites now exhibit the Beaver Server header, and these make use of more than 110,000 unique domain names – mostly under the .cn top-level domain. Most of the sites are hosted by Aliyun, which is China's largest cloud hosting provider, while the majority of the rest are hosted by other Chinese companies. Only a single Beaver site is hosted outside of China – this solitary instance is hosted in Japan, at Amazon's Tokyo AWS region.

The behaviour of these sites suggests that Beaver might not be an entirely new web server, but possibly an application based on Microsoft's HTTP Server API (HTTPAPI 2.0). This API lets C/C++ programmers receive HTTP requests and send responses without using Microsoft IIS. At least 38 million other websites also use HTTPAPI 2.0.

But most of the Beaver sites are currently inaccessible and display the following message: "According to the filing requirements of China's Ministry of Industry and Information Technology (MIIT), the website is accessible only if the ICP information is accurate and the ICP license is filed". An ICP Filing ("Bei'an") is required by all content providers in China before they can use hosting and CDN products, but this only allows them to be used for informational purposes. A Commercial ICP Licence ("Zheng") is required for any website that sells goods or services that directly generate revenue online.

Pepyaka making the web more friendly

The little-known Pepyaka web server has also been quietly growing. This server is used predominantly by Wix, an Israeli company that provides a friendly website building platform to millions of users. Wix provides free website hosting under the *.wix.com domain, or customers can get a free custom domain name when upgrading to one of Wix's yearly premium plans.

This business model has led to a high ratio of domains to hostnames amongst the Pepyaka install base. Wix-hosted websites alone account for 1.4% of all unique domain names in use on the web, which is no mean feat. All of these sites are hosted within the Amazon Web Services cloud, using Pepyaka version 1.11.3. This version number, coupled with Wix's previous uses of nginx, suggests that it could be based on last year's mainline version of nginx.

Total number of websites

Web server market share

DeveloperApril 2017PercentMay 2017PercentChange
Microsoft812,157,80844.71%891,000,72149.09%4.38
Apache412,130,52622.69%380,321,10620.95%-1.73
nginx349,092,97519.22%245,114,31713.50%-5.71
Google19,121,6841.05%20,033,2291.10%0.05
Continue reading

April 2017 Web Server Survey

In the April 2017 survey we received responses from 1,816,416,499 sites and 6,320,910 web-facing computers. This reflects a gain of 56 million sites and 49,800 computers.

Microsoft had a noticeable gain of 108 million sites (+15.4%), recouping last month's loss and expanding its market share by nearly 5 percentage points to 44.7%. Microsoft is 10 million sites down from the start of the year, but its share is now nearly twice as large as Apache's. The only major vendor to suffer a loss of hostnames in April was nginx, which lost 1.4 million sites – this took its market share down by 0.41 points to 19.2%, leaving it slightly further behind Apache's share of 22.7%.

Despite nginx's loss of sites, it was once again the only major vendor to increase its presence within the top million sites, increasing its count by 2,164 sites, while Apache lost 1,962; but Apache stays in the lead with a 40.9% share, while nginx's increased to 28.7%.

nginx's growth is also reflected well in the web-facing computers market, where it had the largest increase of 29,900 computers (+2.44%). This took its share up by 0.32 points to 19.9%, while Apache's leading share fell by 0.27 points to 43.5%. Microsoft is still in second place with 24.2% of the web-facing computer market, but this position looks set to be taken by nginx within the next year if the trend of nginx's gains and Microsoft's losses continue.

Apache continues to reign supreme in terms of active sites, where it increased its market share slightly to 46.3%, putting it further ahead of nginx, which has a share of 19.6%; however, the long term trend over the past several years has seen the two vendors getting closer, with nginx slowly gaining market share while Apache has slowly declined. Microsoft's share of active sites is only 8.28%, but this is enough to keep it in third place, ahead of Google.

A new mainline version of nginx (1.11.13) was released on 4 April. This release included several bugfixes and formed the basis of the current stable version, nginx 1.12.0, which was later released on 12 April. As a consequence of including all bug fixes and new features from the entire 1.11.x branch, nginx 1.12.0 includes support for configuring multiple SSL certificates of different types, better support for dynamic modules, and several other new features.

Earlier this month, Netcraft examined the success of ICANN's New gTLD program, as well as the impact it has had on brand owners such as LEGO. The .loan gTLD saw the largest domain growth of any type of TLD this month, gaining 287,000 unique domains, yet losing more than 10 million websites. Only 3% of all .loan domains are considered active by Netcraft, indicating that large numbers share near-identical content, such as monetized domain holding pages.

From the TLD operator's perspective, the rise in .loan domains is much more significant than the large reduction in sites, as each unique domain will correspond to a domain registration, which invariably involves some type of transaction. The cost of registering a single .loan domain can vary between $20-$40 per year (including ICANN fees), depending which registrar is used, although some registrars – such as Namecheap – offer low introductory prices of $0.88 for the first year only. The prevalence of domain holding pages suggests that many of these domains may have been bought at introductory prices, so the estimated revenue from the 980,000 unique .loan domains currently in use on the web is likely to be much closer to $1m than $40m.

Total number of websites

Web server market share

DeveloperMarch 2017PercentApril 2017PercentChange
Microsoft704,000,53039.99%812,157,80844.71%4.73
Apache383,707,11221.79%412,130,52622.69%0.90
nginx350,540,37219.91%349,092,97519.22%-0.69
Google18,849,1711.07%19,121,6841.05%-0.02
Continue reading

March 2017 Web Server Survey

In the March 2017 survey we received responses from 1,760,630,795 sites and 6,271,146 computers, reflecting a loss of 31 million sites, but a gain of 34,000 computers.

nginx was the only major web server vendor to increase its market shares in all four metrics this month. Its share of websites grew by 0.49 percentage points, reaching nearly 20%, and its share of web-facing computers grew by 0.39 p.p. to 19.6%. The latter gain was driven by a 31,000 growth in the number of computers running nginx, which was by far the largest computer growth in the survey.

Microsoft suffered the largest loss in March, falling by 70 million sites and taking its share down by more than 3 percentage points. This drop paved the way for Apache to claw back 0.91 points with its gain of 9.4 million sites. Microsoft's market share of sites now stands just below 40%, though this remains nearly as much as Apache and nginx combined.

Microsoft also suffered the largest and only loss of active sites among the major vendors, reducing its total count of active sites by 421,000. Microsoft's active sites share is now less than 9%, far behind nginx's share of 19.7% and Apache's 45.8%.

nginx was the only major vendor to increase its presence within the top million sites, increasing its count by 1,592, while Apache lost 3,502 and Microsoft lost 746.

Microsoft web servers

Among the 704 million sites that are powered by Microsoft web server software, Windows Server 2008 is still the most commonly used platform. The original version of this operating system shipped with Microsoft IIS 7.0 as its web server, while the subsequent Windows Server 2008 R2 release included IIS 7.5. More than half a billion websites are hosted on Windows Server 2008 computers (including R2), which accounts for 72% of all Windows-hosted websites.

Although its R2 version was released more than 7 years ago, Windows Server 2008 is likely to remain prevalent for several more years. Last year's launch of the Windows Server Premium Assurance program allows customers to extend Windows Server 2008's support period from 10 to 16 years, giving access to security updates (as well as "critical" and "important" bulletins) until January 2026.

A further 185 million sites are still running on Windows Server 2003 computers, which are not covered by the Windows Server Premium Assurance program. The extended support period for Windows Server 2003 ended on July 14, 2015, so unless site operators have a special agreement in place, Microsoft will no longer be issuing security updates for any version of Windows Server 2003. US-CERT warned that these unsupported installations of Windows Server 2003 are exposed to an elevated risk of cybersecurity dangers, such as malicious attacks or electronic data loss.

Microsoft's newest operating system, Windows Server 2016, may still seem in its infancy, but it is now starting to show promising growth. 80,200 sites are now being served from Windows Server 2016 machines, which is nearly 20,000 more than last month. The number of web-facing computers running Windows Server 2016 also grew by 2,509, while Windows Server 2008 lost 2,316.

windows-computers

Windows Server 2016's computer growth was outpaced only by Window Server 2012, which gained 7,100 computers this month. Windows Server 2012 now accounts for 463,000 web-facing computers, which is nearly half as many as Windows Server 2008, but it is used to host far fewer websites – just 22 million compared with the 535 million sites hosted on Windows Server 2008 computers.

Total number of websites

Web server market share

DeveloperFebruary 2017PercentMarch 2017PercentChange
Microsoft773,552,45443.16%704,000,53039.99%-3.18
Apache374,297,08020.89%383,707,11221.79%0.91
nginx348,025,78819.42%350,540,37219.91%0.49
Google18,438,7021.03%18,849,1711.07%0.04
Continue reading

February 2017 Web Server Survey

In the February 2017 survey we received responses from 1,792,104,054 sites and 6,236,791 web-facing computers, reflecting a loss of 7.9 million sites and 91,200 computers.

nginx gains sites and computers

nginx had the largest growth of both sites and web-facing computers amongst the major vendors this month, enjoying a gain of 31 million sites and 13,400 computers, while hefty losses by Microsoft and Apache led to the overall losses seen in this month’s survey. Microsoft lost 48 million sites and 9,900 computers, while Apache lost 13 million sites and 85,700 computers.

Much of the loss of web-facing computers using Apache is the result of declining numbers of Western Digital My Cloud personal storage devices being found in Netcraft's survey. These devices allowed consumers to access their files remotely using public hostnames under the wd2go.com domain. This disappearing act might have been influenced by the three My Cloud firmware updates that were released in December – the first of these changed how files are accessed from the My Cloud web and mobile apps, and the other two resolved a security vulnerability related to remote access.

Despite suffering the largest loss, Microsoft web servers power 43.2% of all sites on the internet, more than twice Apache's share. Meanwhile, nginx's growth has increased its own count to 348 million, bringing it to within striking distance of Apache. This highlights a dramatic change in fortunes for Apache, which was comfortably in first place a year ago, but is now under threat of falling into third place.

In terms of web-facing computers, Apache continues to fare well. While its 3% decline is significant in the space of a month, Apache's 2.7 million computers still give it the lion's share of the market (44.1%). This is followed by Microsoft's 1.5 million computers (24.7%), and nginx's 1.2 million (19.2%).

nginx was also the only major vendor to make a gain within the top million busiest sites. Its share grew slightly to 28.34%, while Apache suffered the largest loss of 0.21 percentage points, taking its share down to 41.41%, though Apache maintained its first-place position with a lead of 13.1 percentage points over nginx.

Apache still strong in active sites

Despite its losses elsewhere, Apache gained 887,000 active sites this month. nginx made the second largest gain, with an increase of 757,000 active sites. The active sites metric is more appropriate for some applications, as it counts websites but excludes those that contain automatically generated content such as domain holding pages.

Apache also has the largest share of this market (45.8%), with its total number of active sites now reaching almost 80 million – comfortably ahead of nginx, which takes up second place with 34 million active sites.

LiteSpeed 5.1.13 addresses DDoS vulnerability

February saw some new releases of the LiteSpeed web server. Most notably, version 5.1.13 was released on 17 February, after some LiteSpeed Enterprise customers reported service disruptions. These were caused by a surge of distributed denial of service (DDoS) attacks that specifically targeted a bug in LiteSpeed servers earlier that day. Rather impressively, it took LiteSpeed less than two hours to identify the heap buffer overflow that was responsible for the problem, push a bug fix build of 5.1.12, and release 5.1.13.

Looking ahead, it is likely that the first release in the 5.2 branch of LiteSpeed will support HTTP/2 Server Push, which could speed up some websites by allowing the server to send resources to clients before the browser has requested them. This feature has already been implemented in the second release candidate (5.2RC2), which was made available on 13 February.

LiteSpeed gained 42 million sites this month as a large number of sites under the .science gTLD reappeared. This did not have a positive impact on its computer count, however, which fell by 666 to 23,240.

Other new releases from web server vendors

Apache 2.2.32 was released on 13 January. This is the latest version in the 2.2 legacy branch, which now enforces a stricter HTTP request grammar, corresponding to RFC 7230 for request lines and request headers. This addresses a security vulnerability (CVE-2016-8743) that might have allowed malicious clients or downstream proxies to carry out response splitting and cache pollution attacks. This release also mitigates the "httpoxy" (CVE-2016-5387) issues that were already addressed in the 2.4 stable branch.

New stable and mainline versions of nginx were also released in the past month. nginx 1.10.3 stable was released on 31 January, followed by nginx 1.11.10 mainline on Valentine's Day. Both versions include several bugfixes, while the mainline release also introduces a few new features.

Meanwhile, documentation for the Microsoft IIS Administration API is now available. This REST API allows IIS instances to be configured with any HTTP client, using tools such as the one available at manage.iis.net. The rationale for providing the API is to have an open and standard interface that can be used from any platform, unlike AppCmd.exe, which can only be run on Windows.

Total number of websites

Web server market share

DeveloperJanuary 2017PercentFebruary 2017PercentChange
Microsoft821,905,28345.66%773,552,45443.16%-2.50
Apache387,211,50321.51%374,297,08020.89%-0.63
nginx317,398,31717.63%348,025,78819.42%1.79
Google17,933,7621.00%18,438,7021.03%0.03
Continue reading

January 2017 Web Server Survey

In the January 2017 survey we received responses from 1,800,047,111 sites and 6,328,006 computers, reflecting a gain of 61 million sites and 159,000 computers.

Microsoft gained the largest number of sites this month – 38 million – although it was closely followed by Apache, which gained 32 million. Nearly 822 million sites (45.7%) are now powered by Microsoft webserver software.

Meanwhile, nginx gained 17 million sites, and has also continued to show strong and steady computer growth. This month's gain of 60,000 web-facing nginx computers was the largest seen by any vendor, outweighing Microsoft's and Apache's gains of 40,000 and 20,000. If last year's trends continue in 2017, it seems plausible to expect that nginx could overtake Microsoft to become the second largest vendor (by computers) in the second half of 2017.

Microsoft's latest version of Internet Information Services – IIS 10.0, which uses Windows Server 2016 as its primary platform – was found powering 45,000 websites this month. Future migration to IIS 10.0 may be slower than with previous IIS versions, however, as Microsoft announced Windows Server Premium Assurance in December 2016, which extends the support period from 10 to 16 years for existing Windows Server products. This means Premium Assurance customers will continue to receive security updates (as well as "critical" and "important" bulletins) for Windows Server 2008 until January 2026. In January 2017, more than 600 million sites are served from Windows Server 2008 machines.

Each of the other major server vendors released updates last month. nginx 1.11.7 mainline version was released on 13 December, followed by 1.11.8 on 27 December. Both releases included several bug fixes and a few new features.

The mainline 1.11.x branch of nginx is typically updated every 4-6 weeks and is aimed at users who require the latest features, whereas the 1.10.x stable branch is only updated when critical issues need to be fixed. Only two updates have been released on the stable branch since 1.10.0 was forked from mainline in April 2016. Stable is the most commonly used branch: nearly 24 million sites are using 1.10.x stable, compared with 2.2 million using 1.11.x mainline.

Apache 2.4.25 was released on 20 December 2016, incorporating security, feature and bug fixes (including many from the unreleased 2.4.24 version). The security fixes include a mitigation for issues caused by the httpoxy vulnerability, and better enforcement of the HTTP request grammar in RFC 7230 to reduce the likelihood of response splitting and cache pollution attacks.

While many sites still use older versions of Apache, such as the 2.2.x legacy versions, the Apache Project continues to point out that the latest release from the 2.4.x stable branch represents the best available version of Apache HTTP Server. Nonetheless, most sites—just over 100 million— report to be using 2.2.x legacy versions, compared with 69 million sites that use 2.4.x. The most commonly observed Apache Server banners are Apache/2.4.7 (Ubuntu) (36 million sites), followed by Apache/2.2.15 (CentOS) (25 million); however, these servers may not necessarily be as old and vulnerable as their version numbers imply. Netcraft previously discussed this "backporting" behaviour a few years ago.

LiteSpeed suffered the largest loss of sites this month, returning to October 2016 levels after plummeting by 42 million sites to leave a total of 5.5 million. Despite the large loss of sites, the number of web-facing computers using LiteSpeed increased modestly by 323 to 9,740. LiteSpeed 5.1.11 was released on 15 December, featuring improved caching and a few bug fixes.

December also saw the release of Tengine 2.2.0 development version, which came nearly two years after the previous development version, and a year after the most recent stable version. Not only does Tengine have a relatively sedate release cycle, but its latest version is based on nginx 1.8.1 (the final version of nginx's previous stable branch), which itself is already a year old.

Despite having relatively infrequent releases, 58 million sites are currently using Tengine. Most of these sites do not reveal which version has been installed, but among the 18 million that do, about two-thirds are using the relatively old 1.4.2 development version which was released in November 2012 and based on the nginx 1.2.x stable branch. Tengine was originally created by the Chinese marketplace Taobao, which modified the nginx core to better suit its requirements. It was released as an open source project in December 2011, and today sites under the taobao.com domain account for only 5% of its users.

Total number of websites

Web server market share

DeveloperDecember 2016PercentJanuary 2017PercentChange
Microsoft783,790,49245.07%821,905,28345.66%0.59
Apache354,949,19620.41%387,211,50321.51%1.10
nginx300,839,50717.30%317,398,31717.63%0.33
Google18,602,5441.07%17,933,7621.00%-0.07
Continue reading

December 2016 Web Server Survey

In the December 2016 survey we received responses from 1,739,031,487 sites and 6,169,471 web-facing computers; this reflects a large increase of 302 million sites, but a small loss of 55,900 computers.

All three of the largest server vendors gained sites this month, with growth concentrated at a handful of lesser-known web hosting providers. These changes have triggered large jumps in market share, with nginx (+3.2 percentage points) and Microsoft (+1.6 p.p.) gaining, and Apache losing (-2.2 p.p.).

The growth in number of sites was not reflected in other metrics this month, however, with small drops being seen in both the total number of web-facing computers (-0.9%) and the number of active sites (-0.8%).

Nginx was the only major web server vendor to experience gains across all metrics this month, including active sites, web-facing computers, and the million busiest sites. It now stands only 390,000 computers (6 percentage points) behind second place Microsoft, having surpassed Microsoft in terms of active sites in January 2012, and the top million busiest sites in May 2013.

Previously the 10th most commonly seen web server in terms of active sites, “Webs.com/1.0”, has vanished this month after the website builder began using the Cloudflare CDN service. Cloudflare uses a customised version of nginx (cloudflare-nginx), that powered over 3.6 million active sites in the December survey.

Looking back over 2016, the total number of sites seen in the survey has grown significantly, from 900 million in January to 1.7 billion in December. Much of this growth can be attributed to Microsoft, which has gained more than 520 million hostnames and risen from 28.95% market share to 45.07%. However, the more stable active sites metric shows nginx grew the most this year, having increased by 4.8 million active sites and 2.85 percentage points of market share over the course of 2016.

The year also saw the notable rise of OpenResty: in January, OpenResty was being used by 760,000 sites, 200,000 active sites, and just 3,100 web-facing computers; but in September, tumblr.com switched from using nginx to OpenResty for millions of Tumblr blogs. The number of sites using OpenResty in December is now nearly 15 million, the number of active sites is 5.8 million, and the number of web-facing computers seen running OpenResty has increased to nearly 8,000. The server is now the 9th most popular by number of web-facing computers.

Microsoft’s newest web server software, IIS 10.0, was released this year although it is yet to make a significant impact in the number of sites. Windows Server 2016, the primary operating system for running IIS 10.0 was only fully released in October after months of preview releases; the December 2016 survey finds the operating system being used on 5,800 web-facing computers to host 18,250 sites.

Another notable arrival during 2016 were the hundreds of thousands of Western Digital My Cloud personal storage devices. These Network Attached Storage (NAS) devices run an Apache web server and each has a hostname on the wd2go.com domain. The fact that these devices are placed in end-users' homes, using less-stable home Internet connections, and more commonly powered off than a typical web server has increased the volatility of the Apache web-facing computer data. The December 2016 survey found 680,000 sites for these devices.

Total number of websites

Web server market share

DeveloperNovember 2016PercentDecember 2016PercentChange
Microsoft625,173,66443.51%783,790,49245.07%1.56
Apache324,174,41722.56%354,949,19620.41%-2.15
nginx202,932,12214.12%300,839,50717.30%3.17
Google20,689,2731.44%18,602,5441.07%-0.37
Continue reading