September 2016 Web Server Survey

In the September 2016 survey we received responses from 1,285,759,146 sites and 6,118,785 web-facing computers, reflecting large gains in both metrics: 132 million additional sites, and 138,000 more computers.

Microsoft made up the majority of this month's website growth, with the largest gain of 97 million sites, although it showed only modest increases of 5,200 web-facing computers and 693,000 active sites.

Apache was responsible for most of this month's additional web-facing computers, increasing its count by 87,000 to 2.8 million (+3.2%). Similarly, nginx made a 3.0% gain of 30,000 computers. However, Microsoft's 0.3% gain was not enough to stop its share falling by half a percentage point to 25.3% as a result of the gains made by Apache and nginx.

Although nginx made a healthy gain in web-facing computers, it lost more than 5 million active sites and 5,600 sites within the top-million. 27.6% of the busiest million sites now use nginx (-0.56 pp from last month), while Apache retains its lead with a 42.5% share.

Along with nginx, all of the major web server vendors suffered losses within the top million sites, largely due to the growth of OpenResty this month. More than 10,000 of the top million sites are now using OpenResty, compared with fewer than 4,000 last month, after millions of Tumblr blogs switched from nginx. As well as tumblr.com, basecamp.com — the home of the Basecamp web-based project management tool — ranks amongst the most visited sites to use OpenResty.

Tumblr's adoption of OpenResty has caused the web server to leap up the rankings to become the seventh largest web server vendor by websites, and fifth by active sites. This month, 87% of all OpenResty sites appear under the tumblr.com domain.

Although most OpenResty sites reside under the tumblr.com domain, the number of unique domains using OpenResty also increased noticeably this month.

Although most OpenResty sites reside under the tumblr.com domain, the number of unique domains using OpenResty also increased noticeably this month.

Switching from nginx to OpenResty is not such a paradigm shift as moving to, say, Apache or Microsoft IIS. The OpenResty web application platform is built around the standard nginx core, which offers some familiarity, as well as allowing the use of third-party nginx modules. One of the key additional features provided by OpenResty is the integration of the LuaJIT compiler and many Lua libraries – this gives scope for high performance web applications to be run completely within the bundled nginx server, where developers can take advantage of non-blocking I/O.

Another web server that has gained prominence over the past year is Cowboy, a small and fast modular HTTP server written in Erlang. Optimised for low latency and low memory usage, it is currently the fifth most common web server software installed on web-facing computers that accept HTTP connections. Most of the computers used by Cowboy servers are powered by the Heroku Cloud Application Platform and hosted at Amazon Web Services.

Total number of websites

Web server market share

DeveloperAugust 2016PercentSeptember 2016PercentChange
Microsoft445,105,75538.58%542,498,79642.19%3.61
Apache300,028,83226.01%316,042,28924.58%-1.43
nginx181,606,29715.74%186,529,03814.51%-1.23
Google22,111,4311.92%21,467,7291.67%-0.25
Continue reading

August 2016 Web Server Survey

In the August 2016 survey we received responses from 1,153,659,413 sites and 5,980,524 web-facing computers. This reflects an increase of 80 million sites, but a loss of 78,000 computers.

While the overall number of sites increased this month, this growth was not felt evenly by each web server vendor: Microsoft gained the largest number of sites with an increase of 66 million, while second-placed Apache lost 41 million sites. Tengine, the nginx-based web server from Chinese online shopping giant Taobao, gained 28 million sites.

Whilst there were large changes in total number of sites, these were accompanied by much more modest changes in active sites – a more stable metric designed to ignore automatically generated bulk content. Apache and Microsoft both suffered small drops in the number of active sites, -0.5% and -0.8% respectively, whilst Tengine and nginx gained 120,000 (7.3%) and 81,000 (0.2%).

The majority of this month’s drop in web facing computers were running Apache, with a decrease of just over 107,000 (3.8%) using the open-source server. One of the primary contributors to this drop was the loss of a large number of consumer-NAS devices running Apache. While these devices have steadily increased in number since the start of 2016, this month has seen a marked decline. These devices are mostly connected via home internet lines and are therefore likely to come and go from month to month. As a result, the Apache losses this month are spread over a large number of consumer ISPs. On the other hand, Apache continued to see growth amongst web hosting providers.

A gain of 24,000 web-facing computers for nginx, the largest gain in web facing computers this month, once more boosts its market share, which now stands at 17.0%. Microsoft also experienced a small increase in market share, despite its loss of 4,000 web-facing computers, given Apache’s large loss this month.

Windows Server 2016 — which will be the main platform for Microsoft IIS 10.0 — is edging closer to its official launch at Microsoft's Ignite conference in September. In the meantime, developers can try out many of the new features in IIS 10 by either installing the latest Windows Server 2016 Technical Preview 5, or by installing the self-contained IIS 10.0 Express on Windows 7 SP1 or later.

More than 11,000 websites are already using Microsoft IIS 10.0, with almost all of these sites using a version of Windows Server 2016.

The previous month saw two new releases of the mainline version of nginx, mostly incorporating bug fixes and feature additions, while the release of Apache 2.4.23 addressed a security issue which could have allowed clients to gain unauthorised access to protected resources if a server was configured to use HTTP/2.

Several web servers were also updated following the disclosure of a set of vulnerabilities dubbed httpoxy. These vulnerabilities can affect web applications running in CGI or CGI-compatible environments.

The vulnerability stems from a simple namespace conflict where the client-provided HTTP Proxy header was placed into an HTTP_PROXY environment variable as is the custom for CGI applications; but where HTTP_PROXY was trusted by the application and used to configure an outgoing proxy.

This type of vulnerability was first discovered in libwww-perl more than 15 years ago, but in July it was found to be still exploitable in PHP and many other modern languages and libraries. Successful exploitation of these issues could allow a remote attacker to proxy outgoing HTTP requests made by a vulnerable web application, which may expose sensitive data.

To mitigate the httpoxy vulnerability, Apache 2.4.24-dev avoids populating the HTTP_PROXY variable from a Proxy header in httpd CGI environments. Similar mitigations have also been implemented in Lighttpd 1.4.41 and LiteSpeed, while nginx and Varnish have published mitigation advice.

Total number of websites

Web server market share

DeveloperJuly 2016PercentAugust 2016PercentChange
Microsoft378,655,75935.26%445,105,75538.58%3.32
Apache340,551,07431.72%300,028,83226.01%-5.71
nginx170,896,71615.92%181,606,29715.74%-0.17
Google22,552,9012.10%22,111,4311.92%-0.18
Continue reading

July 2016 Web Server Survey

In the July 2016 survey we received responses from 1,073,777,722 sites and 6,058,513 web-facing computers. This reflects an increase of 28 million sites and 107,000 computers.

Microsoft and Apache continue to fluctuate between 1st and 2nd places for total number of websites, with Microsoft retaking the lead again this month after an increase of 36 million sites, and the loss of 20 million Apache sites. However, it is only in the hostnames metric that Microsoft leads, with Apache coming out on top by a considerable margin in all other areas - active sites, domains, IP addresses and web-facing computers.

Looking at the number of active sites, a more stable metric created by Netcraft to ignore automatically generated bulk content, Apache leads with a 46.4% market share. nginx is the second most popular vendor, with a 21.8% market share, and is the only major vendor to consistently gain share in recent months. A similar percentage of both Apache’s and nginx’s sites are deemed to be active, 23.7% of Apache sites and 22.1% for nginx. In comparison, only 4.5% of sites using Microsoft server software are considered active, leaving Microsoft in 3rd place by this active sites metric with a 9.8% market share.

Apache, Microsoft and nginx all gained web-facing computers this month; however, nginx once again saw the only increase in market share. nginx gained 46,000 computers, a growth of 4.8% on last month, and now stands just shy of 1 million web-facing computers. Apache and Microsoft gained 31,000 and 11,000 computers.

nginx also continues to gain market share among the top million busiest websites, where it is now used by 27.9% of sites. Apache, while still holding a dominant position, continues to slowly lose market share, falling 0.55 percentage points to 43.2%.

Total number of websites

Web server market share

DeveloperJune 2016PercentJuly 2016PercentChange
Microsoft342,605,66632.77%378,655,75935.26%2.50
Apache360,458,01834.48%340,551,07431.72%-2.76
nginx169,316,54716.19%170,896,71615.92%-0.28
Google21,662,6732.07%22,552,9012.10%0.03
Continue reading

June 2016 Web Server Survey

In the June 2016 survey we received responses from 1,045,534,808 sites and 5,951,685 web-facing computers. This reflects an increase of 12 million sites, along with a modest gain of 4,700 computers.

Apache regained the lead from Microsoft this month, with a large increase of 60 million sites taking its total up to 360 million, while Microsoft lost 24 million. Microsoft enjoyed a brief foray at the top in April and May, thanks to a proliferation of link farming sites, but now stands 18 million sites behind Apache.

Apache's net growth included only 14 million new websites – the remainder consisted of existing websites that switched to Apache after previously using other web server software. Most notably, 52 million sites switched from Tengine to Apache, while 12 million switched from Microsoft. The number of websites using Tengine fell by more than 60%, largely as a result of the migration to Apache. Most of the sites involved in this switch were hosted by OVH in Canada, and not only changed server vendor, but also moved to a different hosting company—Data Foundry—in the United States.

Although the number of Tengine websites fell to 29 million, the number of active sites using Tengine actually increased slightly to 1.8 million. In a similar vein, the number of active sites running Apache fell by 630,000, even though the total number of sites grew by more than 60 million.

Out of the largest vendors, Microsoft has the lowest proportion of active sites, with only 4.8% of its 340 million sites being active, while Tengine's proportion has crept up to 6.3%. Both are significantly lower than Apache's proportion of 22.3% and nginx's 21.6%.

Newer versions of web server software generally attract a much higher proportion of active sites. For example, 56% of sites running on Microsoft IIS 10 (which will be included with Windows Server 2016, but is already available for Windows 10) are classified as active, while 23% of sites running IIS 8.5 are active, along with 14% of IIS 8.0 sites. This proportion dwindles to only 2.8% by the time we consider IIS 6.0, which remains a very popular choice of server in China despite no longer being supported by Microsoft.

nginx also demonstrates this trend, with the largest number of its active sites running on the latest 1.10.x stable branch. More than half of all sites using this version are active.

Across all versions, nginx continues to muscle its way into the market with confidence. This month it gained almost a million active sites, along with an additional 31,000 web-facing computers, giving it the largest growth in these important metrics. Conversely, Apache lost 26,000 computers, while Microsoft lost 4,500.

nginx has also continued to grow its presence amongst the top million websites, where it is now used by 27.6% of sites. Apache continues to lead with a 43.7% share of this market, although its share has generally been on the decline since 2011. If current trends continue, nginx could possibly take the lead from Apache within the next couple of years.

Total number of websites

Web server market share

DeveloperMay 2016PercentJune 2016PercentChange
Apache300,447,47029.06%360,458,01834.48%5.41
Microsoft366,964,00935.50%342,605,66632.77%-2.73
nginx163,902,97115.85%169,316,54716.19%0.34
Google21,567,2522.09%21,662,6732.07%-0.01
Continue reading

May 2016 Web Server Survey

In the May 2016 survey we received responses from 1,033,790,346 sites and 5,946,961 web-facing computers. This reflects a gain of 147,000 computers, coupled with a loss of 49 million sites.

While last month's survey recorded the largest number of sites ever, many of the Chinese sites running Microsoft IIS that appeared last month have since disappeared. Combined with other departures, Microsoft suffered a net loss of 75 million sites this month, which has played a major part in its market share falling by more than 5 percentage points to less than 36%. Nevertheless, it is still the most common server vendor by number of sites, with a total of nearly 370 million hosted on IIS servers.

Despite Microsoft's loss of 75 million sites, the number of active sites using IIS actually grew by 450,000, which is indicative of the low quality of the sites it lost. Most of the lost sites were engaged in link farming activity, with large numbers of these sites being served from relatively few computers. The loss of these sites therefore had little impact on the number of web-facing computers using Microsoft IIS, which grew by 14,000.

Microsoft's closest competitor, Apache, gained 8.4 million sites, with its increased market share of 29.1% putting it within 6.4 percentage points of Microsoft's leading share.

Although it has yet to reach the same level as Microsoft and Apache, nginx made the largest gains, growing by 21 million sites and increasing its market share by 2.6 points to 15.9%.

nginx also showed the strongest growth in the survey's other metrics: it gained nearly 7.5 million active sites, 74,100 web-facing computers, and increased its presence within the top million sites by 16,000. The most significant of these gains was nginx's active site count increasing by a whopping 27%, largely as a result of Tumblr sites now exhibiting the Server: nginx header (in previous months, most Tumblr sites did not reveal which server software they were using).

While Microsoft has shaken off many of its low-quality sites, Alibaba's nginx fork, Tengine, gained around 10 million. Most of the new sites served by Tengine this month make use of domains under the .science gTLD, which has proved popular with many Chinese link farms and webspam sites – most likely due to the sub-dollar registration costs. Tengine suffered a small net loss in active sites this month, which corroborates the low quality of the 10 million new sites.

Only 2.4% of the sites served by Tengine now qualify as active sites, which highlights just how many of them are used for displaying automatically generated content. Microsoft is still also fairly popular with link farm operators (particularly in China), with only 4.6% of its sites showing active content. In contrast, more than 26% of Apache sites, and nearly 22% of nginx sites feature active content.

Total number of websites

Web server market share

DeveloperApril 2016PercentMay 2016PercentChange
Microsoft441,470,89440.75%366,964,00935.50%-5.26
Apache292,043,54826.96%300,447,47029.06%2.10
nginx143,349,43913.23%163,902,97115.85%2.62
Google20,597,6051.90%21,567,2522.09%0.18
Continue reading

April 2016 Web Server Survey

In the April 2016 survey we received responses from 1,083,252,900 sites and 5,800,222 web-facing computers. This reflects a gain of nearly 80 million sites and 18,100 computers.

This is the largest number of sites the survey has ever seen, beating the previous maximum of 1,028,932,208 in October 2014. The number of web-facing computers is also at its largest, although this total has generally risen much more steadily than the number of sites.

Microsoft was the only major vendor to gain sites this month, and so it was solely responsible for this month's total reaching its highest value ever. Apache lost 33 million sites, while nginx and Google suffered much smaller losses. Many of the 124 million additional sites using Microsoft IIS are aimed at a Chinese audience. Several million are served from just a handful of IP addresses, using either IIS 6.0 or 7.5.

However, this proliferation of new Microsoft-powered websites is largely driven by automated processes. Many are "spam" sites that use link farming techniques to attract traffic. Although Microsoft's website count grew by a remarkable 38.9% in April, it lost 12,100 web-facing computers. High quality websites that attract genuine repeat traffic tend to have a very low number of sites per computer compared with the computers that are involved in link farming, which sometimes host millions of automatically-generated sites each. Corroborating this further, Microsoft suffered a loss of 341,000 active sites this month, taking its total down by 2.0%.

Meanwhile, nginx continued its relentless growth. It gained 19,500 web-facing computers this month (+2.4%), was the only major vendor to increase its active sites count, and increased its share within the top-million websites by 0.49 percentage points.

nginx is particularly prominent at Amazon and DigitalOcean, with the two hosting companies accounting for more than 25% of all nginx computers. In particular, nginx is the most commonly used server at DigitalOcean, being used by just under half of its web-facing droplets. At Amazon, despite its large share of all nginx computers, Apache is more than twice as common, with nginx only used on a quarter of EC2 instances.

Total number of websites

Web server market share

DeveloperMarch 2016PercentApril 2016PercentChange
Microsoft317,761,31831.65%441,470,89440.75%9.10
Apache325,285,18532.40%292,043,54826.96%-5.44
nginx143,464,29314.29%143,349,43913.23%-1.06
Google20,790,7672.07%20,597,6051.90%-0.17
Continue reading