June 2015 Web Server Survey

In the June 2015 survey we received responses from 863,105,652 sites and 5,346,650 web-facing computers, representing an increase of 5.2 million websites and 65,000 additional computers.

Microsoft was responsible for the majority of this month's hostname growth, with a gain of 6.6 million sites, but only contributed 11,700 additional web-facing computers. This has caused Microsoft's market share by hostnames to overtake its declining market share by computers, with both standing at just under 30%.

Apache led this month's web-facing computer growth with a net gain of 24,800, while nginx followed closely with 22,800. This has resulted in nginx's market share increasing by 0.28 to 12.4%, and despite showing the largest net growth, Apache's share fell slightly.

Apache, Microsoft and nginx together account for more than 88% of all web-facing computers in the world, making these vendors by far the most popular choices. However, nginx is the only vendor experiencing consistent increases in market share, up by 3 percentage points over the last year while both Apache and Microsoft have seen losses. The next most commonly used server is lighttpd (pronounced "lighty"), which is used by a mere 0.46% of web-facing computers.

nginx's market share has also been steadily increasing within the top million websites. Its share now stands at 21.9%, and although Apache's use within the million busiest sites has been steadily declining this decade, Apache looks likely to retain the lead for at least a few more years.

Three months after the death of Sir Terry Pratchett, approximately 84,000 websites are now serving the X-Clacks-Overhead: GNU Terry Pratchett header in tribute. Invisible to the majority of users, this HTTP header is a reference to the Discworld novel Going Postal, which features a series of communication towers called the clacks.

In the book, a similar header ("GNU John Dearheart"), is transmitted around the clacks after the inventor's son is killed in an accident while working on a clacks tower. The G means send the message on, N means do not log the message, and U means turn the message around at the end of the line and send it back again — this ensures that the message is transmitted indefinitely, allowing his son to be memorialised forever. Similarly, by transmitting Pratchett's name around the internet, the sites participating in this HTTP header tribute hope to keep his legacy alive. After all, as it says in the book, "A man is not dead while his name is still spoken."

One of the most popular sites to use the X-Clacks-Overhead: GNU Terry Pratchett header is www.theguardian.com, which alone reached more than 5 million unique browsers per day in 2014. With each header taking up 40 bytes of an uncompressed HTTP request, all of the sites involved in the tribute could be generating terrabytes of additional bandwidth usage every day.

Total number of websites

Web server market share

DeveloperMay 2015PercentJune 2015PercentChange
Apache336,813,95939.26%334,731,03538.78%-0.48
Microsoft247,784,66828.88%254,408,17929.48%0.59
nginx123,697,64514.42%122,965,52214.25%-0.17
Google20,103,0682.34%20,130,7322.33%-0.01
Continue reading

May 2015 Web Server Survey

In the May 2015 survey we received responses from 857,927,160 sites and 5,281,889 computers. The number of sites detected increased by nearly 9M this month after two consecutive months of losses. The number of web facing computers also increased by 54k.

Microsoft had the largest growth of web sites, gaining more than one percentage point of market share. Apache, the current leader with a market share of 39.26%, remained stable. Nginx, however, experienced the largest loss amongst major web server vendors and consequently saw a small loss in market share.

Nginx is performing well within the million busiest sites, and when counting the number of web facing computers using the web server — being responsible for the largest growth in each category. Nginx gained just over 2k of the million busiest sites, giving it a market share of 21.64%. Almost 16% of nginx's market share (or 3.4% of the top million sites) in the top million busiest sites is due to CloudFlare. While nginx is used to serve requests at CloudFlare, they may be proxied to backend servers running other web servers. Nginx contributed over half of the total net gain of web facing computers this month, with an increase of 27,500 computers.

DigitalOcean recently became the second largest hosting company by the number of web facing computers, overtaking OVH. Apache, with 48% of publicly visible DigitalOcean computers, is the leading web server, closely followed by nginx with 45.5%.

As the remaining IPv4 address space dwindles, companies are more often resorting to the emerging transfer market to acquire additional IPv4 ranges — for example, DigitalOcean recently appeared as one of the leading inbound receivers of IPv4 addresses from transfers within the RIPE NCC Service Region. With prices expected to increase as IPv4 becomes a more valuable asset, the adoption of IPv6 could accelerate. Netcraft currently finds 740k IPv6 addresses, an increase of 25% compared to the same time last year.

Total number of websites

Web server market share

DeveloperApril 2015PercentMay 2015PercentChange
Apache333,285,74139.25%336,813,95939.26%0.00
Microsoft236,288,84327.83%247,784,66828.88%1.05
nginx126,274,77814.87%123,697,64514.42%-0.45
Google20,051,4332.36%20,103,0682.34%-0.02
Continue reading

April 2015 Web Server Survey

In the April 2015 survey we received responses from 849,027,856 sites and 5,228,046 web-facing computers. Despite the sustained decline in the total number of sites, the number of web-facing computers has continued to grow, increasing by over 35,000 this month.

This month's loss of web sites was felt by each major web server vendor; however, some were hit harder than others. Microsoft once again suffered the largest loss (9.2 million web sites), followed by Apache (3.9 million) and nginx (917,000). Despite both Apache and Microsoft's shrinking numbers of sites, Apache's lead over Microsoft grew by one percentage point, to more than 11%.

Nginx continues to be the only major vendor consistently increasing in market share amongst the million busiest sites, gaining over 2,000 sites this month. It is now used by 21.43% of the top million sites, 9 percentage points more than Microsoft. Apache remains resilient, keeping its number-1 position with 49.19% of the top million sites, but continues its slow decline after losing nearly 1,600 sites.

The total number of web-facing computers found increased by more than 35,000 this month, with the growth split mostly between nginx and Apache, which gained 19,000 and 17,000 computers each. After a gain of 23,000 computers last month, Microsoft IIS gained only 804 computers, resuming its long-term decline in market share.

This month, there were a total of 433 new gTLDs with at least one website, 23 more than in March. The new gTLDs are now used by 1.5% of all sites seen in the survey. The largest increase was seen in .property which grew by more than 35,000 sites, overtaking the plural TLD .properties which has just under 10,000 sites. The new .property sites are mostly short premium domain names which have been registered for some time, and have now started advertising themselves for sale.

The decision by the ICANN New gTLD Program Committee to allow both plural and singular forms of the same word has been a controversial one, the Governmental Advisory Committee recommended reviewing it, but "after careful consideration" ICANN decided that no changes were needed in order to address the potential consumer confusion. The decision should, however, lead to increased competition, with .property being run by Uniregistry, Corp while .properties is run by Donuts Inc.

Total number of websites

Web server market share

DeveloperMarch 2015PercentApril 2015PercentChange
Apache337,175,53638.39%333,285,74139.25%0.87
Microsoft245,496,53327.95%236,288,84327.83%-0.12
nginx127,191,69614.48%126,274,77814.87%0.39
Google20,097,7022.29%20,051,4332.36%0.07
Continue reading

March 2015 Web Server Survey

In the March 2015 survey we received responses from 878,346,052 sites and 5,192,428 web-facing computers. Although the total number of websites fell by 5 million this month, the number of web-facing computers has grown by more than 57,000.

All major web server vendors suffered a loss of hostnames in March, with Microsoft losing the most (8.0 million), while Apache lost 5.3 million and nginx lost 2.9 million. Each also suffered a small loss in market share as a result of an increase in sites with missing server banners, and an unknown vendor, GSHD. However, many of these sites could still be using IIS, as they have previously been seen running IIS/6.0, and are still using Windows Server 2003 this month.

In terms of web-facing computers, all major server vendors showed absolute gains this month. Microsoft experienced the largest gain of 23,000 computers, breaking its recent declining trend with a small increase in market share to just over 30%. Apache and nginx experienced gains of 16,000 and 13,000 respectively. Apache's gain was not enough to increase its market share, however, which fell by 0.22 percentage points; nevertheless, it is still the most commonly installed web server, used on nearly 47% of all web-facing computers in the world.

More than 500 new generic top-level domains have been delegated since 2013, and many of these have shown promising growth. With so many new top-level domains to choose from, there are now more opportunities than ever for fraudsters to register deceptive domain names. Some phishing attacks have already made good use of the new gTLDs by hosting their fraudulent content on domains such as battlelogin.xyz and appleitunesprofile.club.

.xyz was the most commonly used new gTLD for phishing attacks during the previous month. In total, Netcraft blocked 239 phishing attacks across 39 distinct .xyz domains. Judging by their names, and the lack of legitimate content anywhere else on these sites, most of these domains appear to have been registered specifically for the purpose of fraud, rather than belonging to existing sites that had been compromised.

ICANN requires gTLD registries to agree to deal only with registrars that prohibit end-users from carrying out nefarious activities such as phishing, malware distribution and copyright infringement. However, each registry maintains its own safeguards, meaning that some are better than others at proactively defending against fraud.

Total number of websites

Web server market share

DeveloperFebruary 2015PercentMarch 2015PercentChange
Apache342,480,92038.77%337,175,53638.39%-0.38
Microsoft253,484,22128.69%245,496,53327.95%-0.74
nginx130,093,89914.73%127,191,69614.48%-0.25
Google20,238,0572.29%20,097,7022.29%-0.00
Continue reading

February 2015 Web Server Survey

In the February 2015 survey we received responses from 883,419,935 sites and 5,135,229 web-facing computers.

Microsoft showed the largest growth in terms of hostnames, with an additional 12 million sites taking its total up to 253 million. This has increased Microsoft's market share to 28.7%, but Apache continues to lead with a 38.8% share, despite a loss of 5.9 million sites.

Web-facing computer growth was fairly even across the board, with the top three server vendors all showing similar gains. nginx made the largest gain of just under 22,000 computers, while Microsoft and Apache each gained just over 20,000. This has resulted in nginx's market share growing slightly to 11.3%, but Apache maintains its comfortable lead with a 47.2% share, while Microsoft's stays at 29.9%.

Despite its impending lack of support, the number of hostnames using Microsoft IIS 6.0 grew by more than 5% this month; however, the number of web-facing computers using this platform fell by 2%. This version of IIS was released more than 10 years ago, alongside Windows Server 2003, both of which will reach the end of their Extended Support periods in July.

Several of the new generic top-level domains continue to show surprising growth. The number of sites using the .xyz TLD nearly doubled this month, and now totals more than 10 million. Strong growth was also seen by the .red TLD, which grew by nearly 3,000% to reach a total of 850,000. Other new colour-based gTLDs to have appeared in Netcraft's survey recently include .blue, .pink and .black; these are all run by Afilias, which also acts as the domain registry for other well-established TLDs such as .info, .mobi.

The .paris geographic TLD has shown a promising start by already reaching a total of 13,000 sites, outpacing growth seen by other new GeoTLDs which reached general availability around the same time. The .paris GeoTLD became available to all on 2 December 2014 and proclaims itself to be the most affordable address in Paris. The most visited .paris website is currently www.toureiffel.paris, which is where visitors will end up if they attempt to visit the Eiffel Tower's previous website at www.tour-eiffel.fr.

In January, Google added support for the Google Domains beta directly into Blogger, making it easier for users to purchase custom domain names for their blogs. Google has been an ICANN accredited domain registrar since 2005, allowing it to sell domain names under the most popular top-level domains such as .com, .net and .org, but it is also in the process of making a much larger range of new gTLDs available to the public under its role as a registry.

Google Registry is operated by Charleston Road Registry Inc, which is a wholly-owned subsidiary of Google. So far, it has launched three new TLDs: .みんな (which means "everyone" in Japanese), .soy (Spanish for "I am"), and most recently, .how. Google's other successful applications for gTLDs include .zip, .eat, .foo, .meme, and .new, but these are not yet available to register.

Google applied for more than 100 new gTLDs in total, costing it over $18M in ICANN application fees. Some of these applications were subsequently withdrawn, such as that for .and, which was not allowed as it corresponds to the ISO 3166-1 alpha-3 country code for Andorra. Many of the gTLDs that Google applied for also had other applicants competing for ownership, including Amazon in 21 cases.

Google and Amazon were the only applicants for the .dev gTLD, but Amazon withdrew its application after an assumed private deal or auction. Despite .dev being used by private domain names in some corporate development environments, the risk of name collisions was evidently deemed to be low enough to allow Google's application to succeed.

Total number of websites

Web server market share

DeveloperJanuary 2015PercentFebruary 2015PercentChange
Apache348,460,75339.74%342,480,92038.77%-0.97
Microsoft241,276,34727.52%253,484,22128.69%1.18
nginx128,083,92014.61%130,093,89914.73%0.12
Google20,209,6492.30%20,238,0572.29%-0.01
Continue reading

January 2015 Web Server Survey

In the January 2015 survey we received responses from 876,812,666 sites and 5,061,365 web-facing computers.

This is the lowest website count since last January, and the third month in a row which has seen a significant drop in the total number of websites. As was the case in the last two months, the loss was heavily concentrated at just a few hosting companies, and a single IP address that was previously hosting parked websites was responsible for over 50% of the drop.

Microsoft continues to be impacted most by the decline. Having overtaken Apache in the July 2014 survey their market share now stands at just 27.5%, giving Apache a lead of more than 12 percentage points.

Microsoft's decline seems far less dramatic when looking at the number of web-facing computers that use its server software. A net loss of 6,200 computers this month resulted in its computer share falling by only 0.28 percentage points, while Apache's went up by 0.18 to 47.5%.

These losses included many sites running on Microsoft IIS 6.0, which along with Windows Server 2003, will reach the end of its Extended Support period in July. Further abandonment of these platforms is therefore expected in the first half of this year, although Microsoft does offer custom support relationships which go beyond the Extended Support period.

Apache made an impressive gain of 22,000 web-facing computers this month. Half of this net growth can be attributed to the Russian social networking company V Kontakte, which hosts nearly 13,000 computers. Almost all of these were running nginx last month, but 11,000 have since defected to Apache, leaving less than 2,000 of V Kontakte's computers still using nginx.

OVH is still the second largest hosting company in terms of web-facing computers (although DigitalOcean is hot on its heels), but demand for its own relatively new .ovh top-level domain appears to be waning. Last month, we reported that the number of sites using the new .ovh TLD had shot up from 6,000 to 63,000. These sites were spread across just under 50,000 unique .ovh domains, and the number of domains grew by only 2,000 this month.

Only the first 50,000 .ovh domains were given away for free, while subsequent ones were charged at EUR 0.99. Despite being less than a third of the planned usual price of EUR 2.99, this shows how even a tiny cost can have a dramatic impact on slowing down the uptake in domain registrations.

Other new top-level domains which have shown early signs of strong hostname growth include .click, .restaurant, .help, .property, .top, .gifts, .quebec, .market and .ooo, each of which were almost non-existent last month but now number in their thousands.

The proliferation of new top level domains is evidently generating a lot of money for registrars and ICANN, but for some parties it has caused expenditure that was previously unnecessary. Take the new .hosting TLD for example: you would expect this domain to only be of interest to hosting companies, but US bank Wells Fargo has also registered some .hosting domains, including wellsfargo.hosting, wellsfargoadvisors.hosting and wellsfargohomemortgage.hosting. These domains are not used to serve any content, and instead redirect customers to Wells Fargo's main site at wellsfargo.com. The sole purpose of registering these domains appears to be to stop any other party from doing so, which protects the bank's brand and prevents the domains being used to host phishing sites.

In a similar move, Microsoft has also registered several .hosting domains including xbox.hosting, bing.hosting, windows.hosting, skype.hosting, kinect.hosting and dynamics.hosting. Browsing to any of these domains causes the user to be redirected to bing.com, which displays search results for the second-level string (i.e. "xbox", "windows", etc.).

Of course, with many other new TLDs continually popping up, brand protection becomes an increasingly costly exercise. Microsoft has also recently registered hundreds of other nonsensical domains which are used to redirect browsers to bing.com, such as lumia.ninja, lync.lawyer, xboxone.guitars, windowsphone.futbol, microsoft.airforce, azure.luxury, yammer.singles, xboxlive.codes, halo.tattoo, internetexplorer.fishing, and so on.

However, the race to register domain names is not always won by Microsoft — bing.click is a prime example of a domain that someone else got to first. This domain is currently offered for sale, highlighting the fact that it's not just ICANN and the registrars that stand to gain money from the influx of new TLDs.

Total number of websites

Web server market share

DeveloperDecember 2014PercentJanuary 2015PercentChange
Apache358,159,40539.11%348,460,75339.74%0.63
Microsoft272,967,29429.81%241,276,34727.52%-2.29
nginx132,467,76314.47%128,083,92014.61%0.14
Google20,011,2602.19%20,209,6492.30%0.12
Continue reading