June 2014 Web Server Survey

In the June 2014 survey we received responses from 968,882,453 sites, six million less than last month.

The battle between Microsoft and Apache heated up this month, with Apache losing 13 million sites and Microsoft gaining 26 million. The resultant changes in market share have left Apache barely clinging onto the lead — Microsoft is now only 0.15 percentage points behind. This is the closest Microsoft has ever been, giving it a good chance of taking the lead for the first time next month.

However, Apache continues to dominate in terms of active sites, i.e. sites which are actively managed by humans rather than being automatically generated for use in activities such as link farming and domain squatting. Under this metric, Apache's losses were less significant, still leaving it with more than half of the market share, and more than 36 percentage points ahead of its closest competitor, nginx.

In terms of all websites, nginx suffered the second largest loss of 8.6 million sites. nginx is very often used as a reverse proxy, although other web servers can also fulfill this role. Apache's mod_proxy module allows it to be configured as either a forward or reverse proxy, and Microsoft IIS can be configured to act as a reverse proxy with the URL Rewrite and Application Request Routing modules. Microsoft Azure Web Sites can also achieve the same functionality once the proxy feature has been enabled.

Tengine, which is based on nginx, also fell by three million sites this month. This web server software is used extensively by its originators, Taobao, and has been an open source project since 2011. Tengine supports all of the features of nginx 1.4.7, plus some additional features which are not present in the stable releases of nginx 1.4.x or 1.6.x, such as syslog and pipe support. However, the most recent mainline version (nginx 1.7.1), which was released on 27 May, does now allow the error_log and access_log directives to be logged to syslog.

IPv4 addresses nearing total exhaustion

On 20 May, ICANN announced that it had begun the process of allocating the remaining blocks of IPv4 addresses to the five Regional Internet Registries (RIRs). As the total number of available 32-bit IPv4 addresses dwindles, network operators are being encouraged to adopt the use of 128-bit IPv6 addresses, which will allow a significantly larger number of unique addresses: IPv4 can only provide 4.3 billion addresses, whereas IPv6 can provide nearly 8×1028 times as many.

Unfortunately, adoption of IPv6 is proving to be a slow process. Only 3% of the hostnames in this month's survey can be resolved to IPv6 addresses, and the total number of IPv6 addresses used by websites has increased by only 18% over the past 12 months.

ICANN's statement says the process of allocating the remaining blocks was triggered when Latin America and Caribbean Network Information Centre's (LACNIC) supply of IPv4 addresses dropped to below 8 million.

Topically, this month's survey saw ICANN's website at www.icann.org change its Server banner from Apache to BigIP. For the past few years, it had either been "Apache" or "Apache/2.2.3 CentOS", although the operating system has consistently been identified as F5 BIG-IP throughout. Adobe's community forums at forums.adobe.com also switched to BigIP this month, from Apache-Coyote/1.1.





DeveloperMay 2014PercentJune 2014PercentChange
Apache366,262,34637.56%353,672,43136.50%-1.05
Microsoft325,854,05433.41%352,208,48736.35%2.94
nginx142,426,53814.60%133,763,49413.81%-0.80
Google20,685,1652.12%20,192,5952.08%-0.04
Continue reading

May 2014 Web Server Survey

In the May 2014 survey we received responses from 975,262,468 sites — 16 million more than last month.

Microsoft threatening Apache's market lead

Microsoft gained nine million additional sites this month, increasing its market share by a further 0.37 percentage points. Meanwhile, despite gaining 4.3 million sites, Apache's market share fell by 0.18 points. Although Apache still leads with 37.6% of all sites, Microsoft is now just 4.1 percentage points behind. Apache has been the most commonly used web server for more than 18 years, but this is the closest Microsoft has ever been to threatening this position.

Apache's position is much stronger when considering only Active Sites — it retains an absolute majority of 52.3%, and second place is held by nginx (14.4%), rather than Microsoft (11.3%). By excluding much of the automatically-generated content present on the internet, the Active Sites metric better reflects web server market share amongst human-maintained web sites.

New releases of nginx

The total number of websites using nginx fell by 3.7 million this month; however, the losses were mostly isolated to just a few hosting companies. Nearly two million nginx sites disappeared at cloud hosting company Enzu, followed by 1.2 million at BurstNet. Within the top million sites, however, nginx was the only web server to grow — Microsoft, Apache, and Google all lost market share.

A new stable version of nginx was released on April 24. nginx 1.6.0 features SPDY 3.1 support and various SSL improvements that were originally implemented in the 1.5.x mainline branch of releases. The previous mainline branch has also been superseded by nginx 1.7.0, which added support for backend SSL certificate verification and support for SNI while working with SSL backends.

Seven million new sites using Microsoft IIS

Nearly seven million of this month's new websites are using Microsoft IIS. Around 11 thousand of these new sites are hosted on the Microsoft Azure platform (including a few phishing sites), helping to maintain Microsoft's position as the largest Windows hosting company in terms on web-facing computers. Most of the new IIS sites at Azure are hosted in the US, with more than a third of the total being hosted in the North Central US Azure region alone.

Notably, www.apachelounge.com is one of this month's websites which appears to have switched to Microsoft IIS. The Apache Lounge is a web forum primarily focused on running Apache in Windows environments, and has provided Windows Apache binaries for more than 10 years. The site understandably has a long history of being hosted from an Apache web server, most recently on a Windows Server 2008 platform, but is now using Microsoft IIS 8.5 on Windows Server 2012.

All may not be as it seems, however, as the web server is still sending an X-Powered-By: Apache/2.4.9 (Win64) header. The web server is also reporting X-Powered-By: ARR/2.5, indicating the use of IIS's load-balancing features. It is likely that Apache Lounge is powered by multiple Apache instances which are hidden behind a Microsoft IIS load balancer.

New gTLDs showing promising growth

Several more new generic top-level domains have shown rapid growth in this month's survey. The .berlin gTLD, which was used by only two websites in last month's survey, is now used by 40,000 websites. The domain started its sunrise registration period on February 14, when trademark owners were able to register .berlin domains ahead of the March 16 deadline.

German news magazine Der Spiegel – one of Europe's largest publications of its kind – criticised the availability of .berlin and other new top-level domains, likening them to a housing bubble and suggesting that such domain names are a waste of money. The company behind the .berlin domain, dotBERLIN GmbH & Co. KG, was quick to reply pointing out that not only had Der Spiegel applied for its own gTLD (.spiegel) two years ago, but it had also already registered the domain name spiegel.berlin before its own article was published.

The .email gTLD is another one that has been thriving since the closure of its sunrise period: More than 12 thousand websites are now using .email, compared with only two last month. This gTLD is one of many operated by Donuts Inc, and became generally available on March 19.  Donuts applied to ICANN for more than 300 TLDs in 2012, 65 of which are already generally available for registration, including .support, .domains, .photos, .estate, .guru and .plumbing. A further 66 TLDs are scheduled for general availability between now and September, including .expert, .parts, .media, .toys, .wtf and .fail.





DeveloperApril 2014PercentMay 2014PercentChange
Apache361,853,00337.74%366,262,34637.56%-0.18
Microsoft316,843,69533.04%325,854,05433.41%0.37
nginx146,204,06715.25%142,426,53814.60%-0.64
Google20,983,3102.19%20,685,1652.12%-0.07
Continue reading

April 2014 Web Server Survey

In the April 2014 survey we received responses from 958,919,789 sites — 39 million more than last month.

Microsoft made the largest gain this month, with nearly 31 million additional sites boosting its market share by 1.9 percentage points. IIS is now used by a third of the world's websites. Although this is not Microsoft's largest ever market share (it reached 37% in October 2007), this is the closest it has ever been to Apache's leading market share, leaving Apache only 4.7 points ahead. Although Apache gained 6.9 million sites, this was not enough to prevent its market share falling by 0.87 to 37.7%. nginx, which gained 3.1 million sites, also lost some of its market share.

More than 70% of this month's new IIS-powered websites are hosted in the US, followed by 22% in China. Nearly 20 million of the new IIS sites in the US are hosted by a single company, Nobis Technology Group, which was also responsible for much of Microsoft's growth in February. A smaller amount of Microsoft IIS growth was also seen on the Windows Azure platform (which will be renamed to Microsoft Azure on April 3), where the total number of active sites has grown by 25% since February, when we compared the platform against Amazon AWS. 84% of all active sites hosted on the Azure platform are running Microsoft web server software.

Many of the new IIS sites hosted by Nobis Technology Group feature similar content and form part of a Chinese link farm. Link farming is often an attempt to influence search engine results, and each individual site within a link farm is typically of little interest to a human. Netcraft's active sites metric therefore provides a better idea of how many websites are actively managed rather than being automatically generated en mass, such as link farm content and domain holding pages. Of the 114 million sites hosted by Nobis, only a fifth are counted as active sites.

In terms of active sites, Apache remains in a much stronger position with a 52% share of the market, compared with Microsoft's 11%. A significantly higher proportion of Apache sites are active: 26% of all Apache sites were deemed to be active, whereas only 6% of Microsoft's were. nginx takes a 14% share of the active sites market, putting it 3 points ahead of Microsoft.

Apache also fares well amongst the million busiest sites, where there is intrinsically very little interference from domain holding pages, link farms and other web spam. Here Apache takes a 53% share of the market, while nginx has 18% and Microsoft has 12%. Although only 3% of the top million sites use Google web server software, Google's dominance amongst the very busiest sites give it a presence on 8 of the top 10 sites.

Both Apache and nginx were affected by security vulnerabilities which were resolved during March, whereas Microsoft IIS has yet to be affected by publicly-known security issues this year.

The latest version of Apache (2.4.9) was released on March 17. The Apache Software Foundation describes this as representing fifteen years of innovation by the project, and this major release of the 2.4 stable branch is recommended over all previous releases. Nevertheless, it is still common for many websites to use the legacy 2.2 branch of releases, or even older versions. Apache 2.4.9 is primarily a security and bug fix release, although it also includes the changes introduced in 2.4.8, which was not actually released. A workaround for a bug in older versions of OpenSSL, which prevented the release of 2.4.8, has been included in 2.4.9.

Although Apache 2.4.8 was not released, the development version (Apache/2.4.8-dev) was found on 675 sites during this survey, which ran in March. Nearly all of these sites were running on FreeBSD servers which belonged to various Apache projects, mostly Apache HTTPD and Apache OpenOffice.

The stable branch of nginx was updated twice during March. Two bugs were resolved in nginx 1.4.6, which was released on March 4. nginx 1.4.7 was then released on March 18, addressing another bug and a heap buffer overflow vulnerability. This security vulnerability affected nginx's SPDY module, where a specially crafted request could allow a remote attacker to execute arbitrary code on a vulnerable web server. nginx is notable for its SPDY support, which is used extensively by CloudFlare and also by Automattic, which hosts millions of WordPress blogs and co-sponsored the development of the ngx_http_spdy_module. The same SPDY vulnerability also affected the mainline branch of nginx, which was resolved with the release of nginx 1.5.12.

Many of the new generic top level domains (gTLDs) are starting to appear in Netcraft's Web Server Survey in significant numbers. For example, the previous survey saw only one website using the .guru gTLD, whereas this month's survey (which ran during March) found 36 thousand. Other gTLDs which have shown significant growth since last month's survey include .photography, .today, .tips, .technology, .directory, .land, .gallery, .estate and .singles.

Amongst established TLDs, the number of sites using the .ga country code top level domain grew by 140% this month. The My GA website allows .ga domains to be registered for free from between 1 and 12 months, which has no doubt helped towards their goal of increasing the awareness of Gabon across the globe. The .ga ccTLD is administered by the Agence Nationale des Infrastructures Numériques et des Fréquences (ANINF) in Libreville, Gabon, while the registration process is provided by Freenom, who also provide free domain registrations for the more popular .tk ccTLD. Registered Freenom users are allowed an unlimited number of domain name renewals on both the .ga and .tk d domains, while paying customers can choose to register domains for as long as 10 years in one go and can automatically renew the registration.

Free and easily-registerable domain names are obviously attractive to fraudsters: During February, Netcraft blocked nearly 1,500 unique phishing sites hosted on .ga domains alone, and this figure jumped to more than 2,400 in March. The vast majority of these phishing attacks targeted Chinese companies, particularly the Taobao marketplace and the Alipay online payment escrow service.





DeveloperMarch 2014PercentApril 2014PercentChange
Apache354,956,66038.60%361,853,00337.74%-0.87
Microsoft286,014,56631.10%316,843,69533.04%1.94
nginx143,095,18115.56%146,204,06715.25%-0.31
Google20,960,4222.28%20,983,3102.19%-0.09
Continue reading

March 2014 Web Server Survey

In the March 2014 survey we received responses from 919,533,715 sites — around half a million fewer than last month.

Apache has gained some breathing space this month. Nearly a year of strong market share growth by Microsoft eventually culminated in the gap between Apache and Microsoft being reduced to only 5.4 percentage points. After the gap dropped to its lowest point last month, Microsoft had looked set to usurp Apache as the most common web server within a matter of months. This month's survey saw Microsoft lose 15.8 million sites and Apache gain 3.2 million, however. Apache's market share increased to 38.6%, 7.5 percentage points ahead of Microsoft and bucking the recent trend.

Most of Microsoft's losses this month were seen at Nobis Technology Group, where more than 30 million link-farming sites stopped operating. Nobis is a private holding company, which owns the DarkStar voice communications network and Ubiquity Hosting Solutions, which has seven data centres across the United States.

nginx gained 5 million sites this month, increasing its market share to 15.6%. The latest mainline version of nginx (1.5.10) now supports SPDY 3.1, which extends the flow control features of SPDY 3.0 by allowing different sessions within a single connection to send data at different rates. It is no surprise that SPDY 3.1 is already supported in the Google Chrome web browser; SPDY was primarily developed by Google, and is one of their trademarks. SPDY 3.1 has also been supported in Mozilla Firefox since 4th February 2014.

Content delivery network CloudFlare — which uses its own web server software based on nginx — rolled out SPDY 3.1 support for all of its customers in February. Since last month, Netcraft's SSL Survey has identified a four-fold increase in the number of HTTPS websites supporting SPDY 3.1, most of which are hosted by CloudFlare. A smaller number of these SPDY 3.1 sites are hosted by the owner of the WordPress.com blogging platform, Automattic, which was one of the sponsors of the ngx_http_spdy_module.

Mozilla has been planning to remove SPDY 2 support from Firefox since September 2013, and this looks set to happen with the release of Firefox 28. Some developers asked for SPDY 2 support to be retained, arguing that dropping support for SPDY 2 would effectively drop SPDY support in many SPDY-enabled websites. However, nginx and CloudFlare now supporting SPDY 3.1 allays some of that concern.

LibreOffice — the free open source office suite bundled with Ubuntu Linux — moved its website from an Apache web server to nginx at the end of January, apparently for performance reasons. Incidentally, this further distances LibreOffice from the Apache Software Foundation - LibreOffice was forked from OpenOffice.org in 2010, before the latter was given to the ASF where development continued under the name of Apache OpenOffice.

More than 30 new generic top-level domains (gTLDs) were delegated to the Root Zone during February, making them officially part of the internet. There are now 471 top level domains in total. The new ones added in February included .flights, .wiki, .xyz, .fish and .移动 (xn--6frz82g – Chinese for "mobile").

Many of these new gTLDs were applied for by Donuts Inc, a US domain registry which was founded in 2011. The company's CEO and co-founder, Paul Stahura, previously founded domain name registrar eNom in 1997. Donuts raised more than $100,000,000 in its Series A financing round and applied to ICANN for more than 300 TLDs in 2012. As a registry, Donuts does not sell domain names directly to the public; instead, customers must purchase them from one of its accredited registrars.





DeveloperFebruary 2014PercentMarch 2014PercentChange
Apache351,700,57238.22%354,956,66038.60%0.38
Microsoft301,781,99732.80%286,014,56631.10%-1.69
nginx138,056,44415.00%143,095,18115.56%0.56
Google21,129,5092.30%20,960,4222.28%-0.02
Continue reading

February 2014 Web Server Survey

In the February 2014 survey we received responses from 920,102,079 sites — over 58 million more than last month.

Microsoft gained a staggering 48 million sites this month, increasing its total by 19% — most of this growth is attributable to new sites hosted by Nobis Technology Group. Along with Microsoft, nginx also made a large gain of 14 million sites, whereas Apache fell by 7 million. Unsurprisingly, these changes have had a dramatic effect on the overall market share of each web server vendor, with Microsoft's share growing by 3.38 percentage points to 32.8% (302 million sites) while Apache's has fallen by 3.41 to 38.2% (352 million sites).

Microsoft's market share is now only 5.4 percentage points lower than Apache's, which is the closest it has ever been. If recent trends continue, Microsoft could overtake Apache within the next few months, ending Apache's 17+ year reign as the most common web server. Apache is faring much better in both the active sites and top million sites datasets, however, where it is still dominating with just over half of the market share in both metrics.

Nearly 2% of the top million websites are now being served by CloudFlare's customised version of nginx (cloudflare-nginx), which it uses to serve web content via its globally distributed CDN edge nodes. This month's survey saw more than a thousand of the top million sites migrate to cloudflare-nginx from other web server software, including pizzahut.co.uk, pet-supermarket.co.uk, the image server used by the popular Cheezburger network of blogs, and the official PRINCE2 website which switched from Microsoft IIS 6.0 running on Windows Server 2003.

Overall, nginx powers 17.5% of the top million sites, including popular overclocking forum www.overclock.net, despite its server headers declaring that it is now using Microsoft IIS 4.1. Responses from the server also include an X-Powered-By header which claims the application is running on Visual Basic 2.0 on Rails (Visual Basic 2.0 is a long-deprecated language which was released more than 20 years ago, while Microsoft IIS 4.1 never actually existed). The server claimed to be running nginx during Netcraft's previous survey, and indeed, it exhibits characteristics which suggest it is still using nginx.

The number of sites using the .pw country-code top-level domain (ccTLD) grew by more than half this month, reaching 10M sites in total. This ccTLD is assigned to Palau, but the .pw registry has branded the domain as the Professional Web and allows domains to be registered by the general public, regardless of which country they are in. 97% of this month's new .pw sites are hosted in the US (87% at Nobis Technology Group alone), and 2.7 million of them are running on Windows.

The busiest .pw domain is the single-letter u.pw, which is used by viral social media site Upworthy. Other than that, the ccTLD remains relatively obscure, with less than 0.02% of the top million sites using .pw domains, while other single-letter domains have been sold for the modest sums of $8,000 each. Only two single-letter .pw domains actually appear within the top million sites, and there are no two-letter domains at all. Last year, Symantec noted an increase in spam messages containing URLs with .pw TLDs, and .pw later became the first TLD to adopt the Uniform Rapid Suspension (URS) rights protection mechanism.

Version 1.0 of the URS Technical Requirements were published by ICANN in October 2013, and are intended to make it faster and cheaper for trademark holders to seek resolution when there are obvious cases of infringement. URS is intended to complement rather than replace the existing Uniform Domain-Name Dispute-Resolution Policy (UDRP).





DeveloperJanuary 2014PercentFebruary 2014PercentChange
Apache358,669,01241.64%351,700,57238.22%-3.41
Microsoft253,438,49329.42%301,781,99732.80%3.38
nginx124,052,99614.40%138,056,44415.00%0.60
Google21,280,6392.47%21,129,5092.30%-0.17
Continue reading

January 2014 Web Server Survey

In the January 2014 survey we received responses from 861,379,152 sites, an increase of 355,935 since last month.

2013 has been a year of significant change: the web has grown by more than one third, the importance of SSL has been highlighted by a series of spying revelations, Microsoft now power just below 30% of all web sites, and Apache has lost almost 14 percentage points of market share. Additionally, nginx, the relative newcomer, saw its market share peak at 16%, just shy of Microsoft’s position at the beginning of last year.

The total number of web sites discovered has increased dramatically this year — from 630 million web sites in January 2013 to 861 million in January 2014 (+37%) — though the growth does not compare to the doubling in size during 2011.

With the revelations from the NSA documents leaked by Edward Snowden providing months of mainstream publicity, 2013 has been a bumper year for the SSL industry. Websites are increasingly being served over HTTPS: 48% more sites within the million busiest are using SSL than in January 2013. In total, there are over half a million more SSL certificates (+22%) in use on the web since January 2013. The estimated total revenue of the industry has increased even more rapidly, by 28% from September 2012 to September 2013, reflecting the increased uptake of more expensive certificates including Extended Validation, multi-domain, and wildcard certificates.

Apache remains the most commonly used web server on the internet, 10 million more web sites are using it than this time last year; however, this growth has not been sufficient to maintain its share of a market which grew by more than 200 million web sites. As a result, Apache’s market share has fallen by 14 percentage points since January and now stands at 42%.

In stark contrast to Apache, Microsoft had a strong year — almost 150 million more web sites use a Microsoft web server than in January 2013. Microsoft’s share is close to 30% of the entire market and a combination of its strong growth and the corresponding lack of growth of sites using Apache has resulted in Apache’s lead shrinking by more than 26 percentage points to just 12. Microsoft's own cloud platform, Azure, has grown steadily throughout 2013 — there are 39% more web-facing computers hosted by Microsoft in January 2014 than the same time last year — and despite offering alternatives, Microsoft's IIS is by far the most common web server on Azure.

Open-source web server nginx has continued to gain acceptance, especially amongst the busiest web sites. Nginx is now used on 14% of all web sites found, up 2 percentage points since January 2013, but has fallen slightly from the peak of 16% it achieved in October. In May 2013, nginx overtook Microsoft to become the second most common web server within the top million busiest sites and now powers almost 16% of them.





DeveloperDecember 2013PercentJanuary 2014PercentChange
Apache355,244,90041.26%358,669,01241.64%0.38
Microsoft241,777,72328.08%253,438,49329.42%1.34
nginx126,485,20414.69%124,052,99614.40%-0.29
Google38,263,5254.44%21,280,6392.47%-1.97
Continue reading