July 2015 Web Server Survey

In the July 2015 survey we received responses from 849,602,745 sites and 5,350,323 web-facing computers. This represents a net loss of 13.5 million websites, but a gain of 3,700 additional computers.

One of the most significant changes in July was the net loss of nearly 13,000 web-facing computers powered by Microsoft web server software, accompanied by a decline of more than 29 million hostnames. The loss was predominantly seen for servers running Microsoft IIS 6.0, 7.0 and 7.5. These versions of IIS are used by Windows Server 2003, which is no longer supported, and Windows Server 2008 (including 2008 R2), whose mainstream support ended in January. The latest stable release of IIS (version 8.5) is however continuing to grow, this month increasing by over 9,000 web facing computers.

This month's decline has brought Microsoft's market share of hostnames down by nearly 3 percentage points, increasing Apache's lead. However, Apache's own market share also fell slightly, largely due to gains made by nginx and Tengine.

nginx gained 8.5 million sites this month, but more remarkably, it gained over 14,000 web-facing computers, with the largest gains in the US, China, Germany and the UK. Compounding Microsoft's losses, nearly 1.8 million existing websites switched from using Microsoft IIS to nginx in July.

nginx also fared well amongst the top million websites, where it gained a further 3,771 sites, causing losses for Apache, Microsoft and Google. Nonetheless, Apache is still used by nearly half of the top million sites, with its market share being almost 26 percentage points ahead of nginx.

Tengine now powers more websites than Google's web server software, after the number of sites using it grew by 7 million to a total of more than 25 million this month. The open source Tengine web server is based on nginx, and used extensively by the online marketplace Taobao. It currently supports all features found in nginx 1.6.2, plus several other features required by Taobao that were not able to be implemented as nginx modules. Neither nginx nor Tengine support HTTP/2 yet, but they were both early supporters of Google's SPDY protocol, on which HTTP/2 is based. nginx plans to provide support for HTTP/2 by the end of this year, and so it is likely that Tengine may also follow suit at a later date.

Tengine 2.1.0 is the latest development version of Taobao's nginx fork, but despite being released more than six months ago, only 25,000 websites currently claim to be using it. In contrast, Tengine 1.4.2 — which was released in 2012 and is also a development version — is used by nearly 10 million sites, making it by far the most commonly deployed version. The latest stable release, Tengine 1.5.2, is the second most commonly used version, but accounts for just under 200,000 sites.

But like Apache, more than half of the sites running Tengine do not reveal which version they are running, and so the true distribution of version numbers could vary greatly. For instance, 2.7 million of these version-less Tengine websites are used to host Taobao stores directly under the taobao.com domain (e.g. baobeiit.taobao.com). Given that Tengine was created by Taobao in order to provide the features they need, it is not unreasonable to assume that these sites might be using the latest release, or at least a relatively recent one.

Despite being used by a large number of sites, Tengine was found on only 4,240 web-facing computers in July 2015. Three-quarters of these computers are located in China, while nearly 10% are located in the US.

Total number of websites

Web server market share

DeveloperJune 2015PercentJuly 2015PercentChange
Apache334,731,03538.78%325,696,51438.34%-0.45
Microsoft254,408,17929.48%225,282,71326.52%-2.96
nginx122,965,52214.25%131,460,06315.47%1.23
Google20,130,7322.33%20,255,4242.38%0.05
Continue reading

June 2015 Web Server Survey

In the June 2015 survey we received responses from 863,105,652 sites and 5,346,650 web-facing computers, representing an increase of 5.2 million websites and 65,000 additional computers.

Microsoft was responsible for the majority of this month's hostname growth, with a gain of 6.6 million sites, but only contributed 11,700 additional web-facing computers. This has caused Microsoft's market share by hostnames to overtake its declining market share by computers, with both standing at just under 30%.

Apache led this month's web-facing computer growth with a net gain of 24,800, while nginx followed closely with 22,800. This has resulted in nginx's market share increasing by 0.28 to 12.4%, and despite showing the largest net growth, Apache's share fell slightly.

Apache, Microsoft and nginx together account for more than 88% of all web-facing computers in the world, making these vendors by far the most popular choices. However, nginx is the only vendor experiencing consistent increases in market share, up by 3 percentage points over the last year while both Apache and Microsoft have seen losses. The next most commonly used server is lighttpd (pronounced "lighty"), which is used by a mere 0.46% of web-facing computers.

nginx's market share has also been steadily increasing within the top million websites. Its share now stands at 21.9%, and although Apache's use within the million busiest sites has been steadily declining this decade, Apache looks likely to retain the lead for at least a few more years.

Three months after the death of Sir Terry Pratchett, approximately 84,000 websites are now serving the X-Clacks-Overhead: GNU Terry Pratchett header in tribute. Invisible to the majority of users, this HTTP header is a reference to the Discworld novel Going Postal, which features a series of communication towers called the clacks.

In the book, a similar header ("GNU John Dearheart"), is transmitted around the clacks after the inventor's son is killed in an accident while working on a clacks tower. The G means send the message on, N means do not log the message, and U means turn the message around at the end of the line and send it back again — this ensures that the message is transmitted indefinitely, allowing his son to be memorialised forever. Similarly, by transmitting Pratchett's name around the internet, the sites participating in this HTTP header tribute hope to keep his legacy alive. After all, as it says in the book, "A man is not dead while his name is still spoken."

One of the most popular sites to use the X-Clacks-Overhead: GNU Terry Pratchett header is www.theguardian.com, which alone reached more than 5 million unique browsers per day in 2014. With each header taking up 40 bytes of an uncompressed HTTP request, all of the sites involved in the tribute could be generating terrabytes of additional bandwidth usage every day.

Total number of websites

Web server market share

DeveloperMay 2015PercentJune 2015PercentChange
Apache336,813,95939.26%334,731,03538.78%-0.48
Microsoft247,784,66828.88%254,408,17929.48%0.59
nginx123,697,64514.42%122,965,52214.25%-0.17
Google20,103,0682.34%20,130,7322.33%-0.01
Continue reading

May 2015 Web Server Survey

In the May 2015 survey we received responses from 857,927,160 sites and 5,281,889 computers. The number of sites detected increased by nearly 9M this month after two consecutive months of losses. The number of web facing computers also increased by 54k.

Microsoft had the largest growth of web sites, gaining more than one percentage point of market share. Apache, the current leader with a market share of 39.26%, remained stable. Nginx, however, experienced the largest loss amongst major web server vendors and consequently saw a small loss in market share.

Nginx is performing well within the million busiest sites, and when counting the number of web facing computers using the web server — being responsible for the largest growth in each category. Nginx gained just over 2k of the million busiest sites, giving it a market share of 21.64%. Almost 16% of nginx's market share (or 3.4% of the top million sites) in the top million busiest sites is due to CloudFlare. While nginx is used to serve requests at CloudFlare, they may be proxied to backend servers running other web servers. Nginx contributed over half of the total net gain of web facing computers this month, with an increase of 27,500 computers.

DigitalOcean recently became the second largest hosting company by the number of web facing computers, overtaking OVH. Apache, with 48% of publicly visible DigitalOcean computers, is the leading web server, closely followed by nginx with 45.5%.

As the remaining IPv4 address space dwindles, companies are more often resorting to the emerging transfer market to acquire additional IPv4 ranges — for example, DigitalOcean recently appeared as one of the leading inbound receivers of IPv4 addresses from transfers within the RIPE NCC Service Region. With prices expected to increase as IPv4 becomes a more valuable asset, the adoption of IPv6 could accelerate. Netcraft currently finds 740k IPv6 addresses, an increase of 25% compared to the same time last year.

Total number of websites

Web server market share

DeveloperApril 2015PercentMay 2015PercentChange
Apache333,285,74139.25%336,813,95939.26%0.00
Microsoft236,288,84327.83%247,784,66828.88%1.05
nginx126,274,77814.87%123,697,64514.42%-0.45
Google20,051,4332.36%20,103,0682.34%-0.02
Continue reading

April 2015 Web Server Survey

In the April 2015 survey we received responses from 849,027,856 sites and 5,228,046 web-facing computers. Despite the sustained decline in the total number of sites, the number of web-facing computers has continued to grow, increasing by over 35,000 this month.

This month's loss of web sites was felt by each major web server vendor; however, some were hit harder than others. Microsoft once again suffered the largest loss (9.2 million web sites), followed by Apache (3.9 million) and nginx (917,000). Despite both Apache and Microsoft's shrinking numbers of sites, Apache's lead over Microsoft grew by one percentage point, to more than 11%.

Nginx continues to be the only major vendor consistently increasing in market share amongst the million busiest sites, gaining over 2,000 sites this month. It is now used by 21.43% of the top million sites, 9 percentage points more than Microsoft. Apache remains resilient, keeping its number-1 position with 49.19% of the top million sites, but continues its slow decline after losing nearly 1,600 sites.

The total number of web-facing computers found increased by more than 35,000 this month, with the growth split mostly between nginx and Apache, which gained 19,000 and 17,000 computers each. After a gain of 23,000 computers last month, Microsoft IIS gained only 804 computers, resuming its long-term decline in market share.

This month, there were a total of 433 new gTLDs with at least one website, 23 more than in March. The new gTLDs are now used by 1.5% of all sites seen in the survey. The largest increase was seen in .property which grew by more than 35,000 sites, overtaking the plural TLD .properties which has just under 10,000 sites. The new .property sites are mostly short premium domain names which have been registered for some time, and have now started advertising themselves for sale.

The decision by the ICANN New gTLD Program Committee to allow both plural and singular forms of the same word has been a controversial one, the Governmental Advisory Committee recommended reviewing it, but "after careful consideration" ICANN decided that no changes were needed in order to address the potential consumer confusion. The decision should, however, lead to increased competition, with .property being run by Uniregistry, Corp while .properties is run by Donuts Inc.

Total number of websites

Web server market share

DeveloperMarch 2015PercentApril 2015PercentChange
Apache337,175,53638.39%333,285,74139.25%0.87
Microsoft245,496,53327.95%236,288,84327.83%-0.12
nginx127,191,69614.48%126,274,77814.87%0.39
Google20,097,7022.29%20,051,4332.36%0.07
Continue reading

March 2015 Web Server Survey

In the March 2015 survey we received responses from 878,346,052 sites and 5,192,428 web-facing computers. Although the total number of websites fell by 5 million this month, the number of web-facing computers has grown by more than 57,000.

All major web server vendors suffered a loss of hostnames in March, with Microsoft losing the most (8.0 million), while Apache lost 5.3 million and nginx lost 2.9 million. Each also suffered a small loss in market share as a result of an increase in sites with missing server banners, and an unknown vendor, GSHD. However, many of these sites could still be using IIS, as they have previously been seen running IIS/6.0, and are still using Windows Server 2003 this month.

In terms of web-facing computers, all major server vendors showed absolute gains this month. Microsoft experienced the largest gain of 23,000 computers, breaking its recent declining trend with a small increase in market share to just over 30%. Apache and nginx experienced gains of 16,000 and 13,000 respectively. Apache's gain was not enough to increase its market share, however, which fell by 0.22 percentage points; nevertheless, it is still the most commonly installed web server, used on nearly 47% of all web-facing computers in the world.

More than 500 new generic top-level domains have been delegated since 2013, and many of these have shown promising growth. With so many new top-level domains to choose from, there are now more opportunities than ever for fraudsters to register deceptive domain names. Some phishing attacks have already made good use of the new gTLDs by hosting their fraudulent content on domains such as battlelogin.xyz and appleitunesprofile.club.

.xyz was the most commonly used new gTLD for phishing attacks during the previous month. In total, Netcraft blocked 239 phishing attacks across 39 distinct .xyz domains. Judging by their names, and the lack of legitimate content anywhere else on these sites, most of these domains appear to have been registered specifically for the purpose of fraud, rather than belonging to existing sites that had been compromised.

ICANN requires gTLD registries to agree to deal only with registrars that prohibit end-users from carrying out nefarious activities such as phishing, malware distribution and copyright infringement. However, each registry maintains its own safeguards, meaning that some are better than others at proactively defending against fraud.

Total number of websites

Web server market share

DeveloperFebruary 2015PercentMarch 2015PercentChange
Apache342,480,92038.77%337,175,53638.39%-0.38
Microsoft253,484,22128.69%245,496,53327.95%-0.74
nginx130,093,89914.73%127,191,69614.48%-0.25
Google20,238,0572.29%20,097,7022.29%-0.00
Continue reading

February 2015 Web Server Survey

In the February 2015 survey we received responses from 883,419,935 sites and 5,135,229 web-facing computers.

Microsoft showed the largest growth in terms of hostnames, with an additional 12 million sites taking its total up to 253 million. This has increased Microsoft's market share to 28.7%, but Apache continues to lead with a 38.8% share, despite a loss of 5.9 million sites.

Web-facing computer growth was fairly even across the board, with the top three server vendors all showing similar gains. nginx made the largest gain of just under 22,000 computers, while Microsoft and Apache each gained just over 20,000. This has resulted in nginx's market share growing slightly to 11.3%, but Apache maintains its comfortable lead with a 47.2% share, while Microsoft's stays at 29.9%.

Despite its impending lack of support, the number of hostnames using Microsoft IIS 6.0 grew by more than 5% this month; however, the number of web-facing computers using this platform fell by 2%. This version of IIS was released more than 10 years ago, alongside Windows Server 2003, both of which will reach the end of their Extended Support periods in July.

Several of the new generic top-level domains continue to show surprising growth. The number of sites using the .xyz TLD nearly doubled this month, and now totals more than 10 million. Strong growth was also seen by the .red TLD, which grew by nearly 3,000% to reach a total of 850,000. Other new colour-based gTLDs to have appeared in Netcraft's survey recently include .blue, .pink and .black; these are all run by Afilias, which also acts as the domain registry for other well-established TLDs such as .info, .mobi.

The .paris geographic TLD has shown a promising start by already reaching a total of 13,000 sites, outpacing growth seen by other new GeoTLDs which reached general availability around the same time. The .paris GeoTLD became available to all on 2 December 2014 and proclaims itself to be the most affordable address in Paris. The most visited .paris website is currently www.toureiffel.paris, which is where visitors will end up if they attempt to visit the Eiffel Tower's previous website at www.tour-eiffel.fr.

In January, Google added support for the Google Domains beta directly into Blogger, making it easier for users to purchase custom domain names for their blogs. Google has been an ICANN accredited domain registrar since 2005, allowing it to sell domain names under the most popular top-level domains such as .com, .net and .org, but it is also in the process of making a much larger range of new gTLDs available to the public under its role as a registry.

Google Registry is operated by Charleston Road Registry Inc, which is a wholly-owned subsidiary of Google. So far, it has launched three new TLDs: .みんな (which means "everyone" in Japanese), .soy (Spanish for "I am"), and most recently, .how. Google's other successful applications for gTLDs include .zip, .eat, .foo, .meme, and .new, but these are not yet available to register.

Google applied for more than 100 new gTLDs in total, costing it over $18M in ICANN application fees. Some of these applications were subsequently withdrawn, such as that for .and, which was not allowed as it corresponds to the ISO 3166-1 alpha-3 country code for Andorra. Many of the gTLDs that Google applied for also had other applicants competing for ownership, including Amazon in 21 cases.

Google and Amazon were the only applicants for the .dev gTLD, but Amazon withdrew its application after an assumed private deal or auction. Despite .dev being used by private domain names in some corporate development environments, the risk of name collisions was evidently deemed to be low enough to allow Google's application to succeed.

Total number of websites

Web server market share

DeveloperJanuary 2015PercentFebruary 2015PercentChange
Apache348,460,75339.74%342,480,92038.77%-0.97
Microsoft241,276,34727.52%253,484,22128.69%1.18
nginx128,083,92014.61%130,093,89914.73%0.12
Google20,209,6492.30%20,238,0572.29%-0.01
Continue reading