April 2015 Web Server Survey

In the April 2015 survey we received responses from 849,027,856 sites and 5,228,046 web-facing computers. Despite the sustained decline in the total number of sites, the number of web-facing computers has continued to grow, increasing by over 35,000 this month.

This month's loss of web sites was felt by each major web server vendor; however, some were hit harder than others. Microsoft once again suffered the largest loss (9.2 million web sites), followed by Apache (3.9 million) and nginx (917,000). Despite both Apache and Microsoft's shrinking numbers of sites, Apache's lead over Microsoft grew by one percentage point, to more than 11%.

Nginx continues to be the only major vendor consistently increasing in market share amongst the million busiest sites, gaining over 2,000 sites this month. It is now used by 21.43% of the top million sites, 9 percentage points more than Microsoft. Apache remains resilient, keeping its number-1 position with 49.19% of the top million sites, but continues its slow decline after losing nearly 1,600 sites.

The total number of web-facing computers found increased by more than 35,000 this month, with the growth split mostly between nginx and Apache, which gained 19,000 and 17,000 computers each. After a gain of 23,000 computers last month, Microsoft IIS gained only 804 computers, resuming its long-term decline in market share.

This month, there were a total of 433 new gTLDs with at least one website, 23 more than in March. The new gTLDs are now used by 1.5% of all sites seen in the survey. The largest increase was seen in .property which grew by more than 35,000 sites, overtaking the plural TLD .properties which has just under 10,000 sites. The new .property sites are mostly short premium domain names which have been registered for some time, and have now started advertising themselves for sale.

The decision by the ICANN New gTLD Program Committee to allow both plural and singular forms of the same word has been a controversial one, the Governmental Advisory Committee recommended reviewing it, but "after careful consideration" ICANN decided that no changes were needed in order to address the potential consumer confusion. The decision should, however, lead to increased competition, with .property being run by Uniregistry, Corp while .properties is run by Donuts Inc.

Total number of websites

Web server market share

DeveloperMarch 2015PercentApril 2015PercentChange
Apache337,175,53638.39%333,285,74139.25%0.87
Microsoft245,496,53327.95%236,288,84327.83%-0.12
nginx127,191,69614.48%126,274,77814.87%0.39
Google20,097,7022.29%20,051,4332.36%0.07
Continue reading

March 2015 Web Server Survey

In the March 2015 survey we received responses from 878,346,052 sites and 5,192,428 web-facing computers. Although the total number of websites fell by 5 million this month, the number of web-facing computers has grown by more than 57,000.

All major web server vendors suffered a loss of hostnames in March, with Microsoft losing the most (8.0 million), while Apache lost 5.3 million and nginx lost 2.9 million. Each also suffered a small loss in market share as a result of an increase in sites with missing server banners, and an unknown vendor, GSHD. However, many of these sites could still be using IIS, as they have previously been seen running IIS/6.0, and are still using Windows Server 2003 this month.

In terms of web-facing computers, all major server vendors showed absolute gains this month. Microsoft experienced the largest gain of 23,000 computers, breaking its recent declining trend with a small increase in market share to just over 30%. Apache and nginx experienced gains of 16,000 and 13,000 respectively. Apache's gain was not enough to increase its market share, however, which fell by 0.22 percentage points; nevertheless, it is still the most commonly installed web server, used on nearly 47% of all web-facing computers in the world.

More than 500 new generic top-level domains have been delegated since 2013, and many of these have shown promising growth. With so many new top-level domains to choose from, there are now more opportunities than ever for fraudsters to register deceptive domain names. Some phishing attacks have already made good use of the new gTLDs by hosting their fraudulent content on domains such as battlelogin.xyz and appleitunesprofile.club.

.xyz was the most commonly used new gTLD for phishing attacks during the previous month. In total, Netcraft blocked 239 phishing attacks across 39 distinct .xyz domains. Judging by their names, and the lack of legitimate content anywhere else on these sites, most of these domains appear to have been registered specifically for the purpose of fraud, rather than belonging to existing sites that had been compromised.

ICANN requires gTLD registries to agree to deal only with registrars that prohibit end-users from carrying out nefarious activities such as phishing, malware distribution and copyright infringement. However, each registry maintains its own safeguards, meaning that some are better than others at proactively defending against fraud.

Total number of websites

Web server market share

DeveloperFebruary 2015PercentMarch 2015PercentChange
Apache342,480,92038.77%337,175,53638.39%-0.38
Microsoft253,484,22128.69%245,496,53327.95%-0.74
nginx130,093,89914.73%127,191,69614.48%-0.25
Google20,238,0572.29%20,097,7022.29%-0.00
Continue reading

February 2015 Web Server Survey

In the February 2015 survey we received responses from 883,419,935 sites and 5,135,229 web-facing computers.

Microsoft showed the largest growth in terms of hostnames, with an additional 12 million sites taking its total up to 253 million. This has increased Microsoft's market share to 28.7%, but Apache continues to lead with a 38.8% share, despite a loss of 5.9 million sites.

Web-facing computer growth was fairly even across the board, with the top three server vendors all showing similar gains. nginx made the largest gain of just under 22,000 computers, while Microsoft and Apache each gained just over 20,000. This has resulted in nginx's market share growing slightly to 11.3%, but Apache maintains its comfortable lead with a 47.2% share, while Microsoft's stays at 29.9%.

Despite its impending lack of support, the number of hostnames using Microsoft IIS 6.0 grew by more than 5% this month; however, the number of web-facing computers using this platform fell by 2%. This version of IIS was released more than 10 years ago, alongside Windows Server 2003, both of which will reach the end of their Extended Support periods in July.

Several of the new generic top-level domains continue to show surprising growth. The number of sites using the .xyz TLD nearly doubled this month, and now totals more than 10 million. Strong growth was also seen by the .red TLD, which grew by nearly 3,000% to reach a total of 850,000. Other new colour-based gTLDs to have appeared in Netcraft's survey recently include .blue, .pink and .black; these are all run by Afilias, which also acts as the domain registry for other well-established TLDs such as .info, .mobi.

The .paris geographic TLD has shown a promising start by already reaching a total of 13,000 sites, outpacing growth seen by other new GeoTLDs which reached general availability around the same time. The .paris GeoTLD became available to all on 2 December 2014 and proclaims itself to be the most affordable address in Paris. The most visited .paris website is currently www.toureiffel.paris, which is where visitors will end up if they attempt to visit the Eiffel Tower's previous website at www.tour-eiffel.fr.

In January, Google added support for the Google Domains beta directly into Blogger, making it easier for users to purchase custom domain names for their blogs. Google has been an ICANN accredited domain registrar since 2005, allowing it to sell domain names under the most popular top-level domains such as .com, .net and .org, but it is also in the process of making a much larger range of new gTLDs available to the public under its role as a registry.

Google Registry is operated by Charleston Road Registry Inc, which is a wholly-owned subsidiary of Google. So far, it has launched three new TLDs: .みんな (which means "everyone" in Japanese), .soy (Spanish for "I am"), and most recently, .how. Google's other successful applications for gTLDs include .zip, .eat, .foo, .meme, and .new, but these are not yet available to register.

Google applied for more than 100 new gTLDs in total, costing it over $18M in ICANN application fees. Some of these applications were subsequently withdrawn, such as that for .and, which was not allowed as it corresponds to the ISO 3166-1 alpha-3 country code for Andorra. Many of the gTLDs that Google applied for also had other applicants competing for ownership, including Amazon in 21 cases.

Google and Amazon were the only applicants for the .dev gTLD, but Amazon withdrew its application after an assumed private deal or auction. Despite .dev being used by private domain names in some corporate development environments, the risk of name collisions was evidently deemed to be low enough to allow Google's application to succeed.

Total number of websites

Web server market share

DeveloperJanuary 2015PercentFebruary 2015PercentChange
Apache348,460,75339.74%342,480,92038.77%-0.97
Microsoft241,276,34727.52%253,484,22128.69%1.18
nginx128,083,92014.61%130,093,89914.73%0.12
Google20,209,6492.30%20,238,0572.29%-0.01
Continue reading

January 2015 Web Server Survey

In the January 2015 survey we received responses from 876,812,666 sites and 5,061,365 web-facing computers.

This is the lowest website count since last January, and the third month in a row which has seen a significant drop in the total number of websites. As was the case in the last two months, the loss was heavily concentrated at just a few hosting companies, and a single IP address that was previously hosting parked websites was responsible for over 50% of the drop.

Microsoft continues to be impacted most by the decline. Having overtaken Apache in the July 2014 survey their market share now stands at just 27.5%, giving Apache a lead of more than 12 percentage points.

Microsoft's decline seems far less dramatic when looking at the number of web-facing computers that use its server software. A net loss of 6,200 computers this month resulted in its computer share falling by only 0.28 percentage points, while Apache's went up by 0.18 to 47.5%.

These losses included many sites running on Microsoft IIS 6.0, which along with Windows Server 2003, will reach the end of its Extended Support period in July. Further abandonment of these platforms is therefore expected in the first half of this year, although Microsoft does offer custom support relationships which go beyond the Extended Support period.

Apache made an impressive gain of 22,000 web-facing computers this month. Half of this net growth can be attributed to the Russian social networking company V Kontakte, which hosts nearly 13,000 computers. Almost all of these were running nginx last month, but 11,000 have since defected to Apache, leaving less than 2,000 of V Kontakte's computers still using nginx.

OVH is still the second largest hosting company in terms of web-facing computers (although DigitalOcean is hot on its heels), but demand for its own relatively new .ovh top-level domain appears to be waning. Last month, we reported that the number of sites using the new .ovh TLD had shot up from 6,000 to 63,000. These sites were spread across just under 50,000 unique .ovh domains, and the number of domains grew by only 2,000 this month.

Only the first 50,000 .ovh domains were given away for free, while subsequent ones were charged at EUR 0.99. Despite being less than a third of the planned usual price of EUR 2.99, this shows how even a tiny cost can have a dramatic impact on slowing down the uptake in domain registrations.

Other new top-level domains which have shown early signs of strong hostname growth include .click, .restaurant, .help, .property, .top, .gifts, .quebec, .market and .ooo, each of which were almost non-existent last month but now number in their thousands.

The proliferation of new top level domains is evidently generating a lot of money for registrars and ICANN, but for some parties it has caused expenditure that was previously unnecessary. Take the new .hosting TLD for example: you would expect this domain to only be of interest to hosting companies, but US bank Wells Fargo has also registered some .hosting domains, including wellsfargo.hosting, wellsfargoadvisors.hosting and wellsfargohomemortgage.hosting. These domains are not used to serve any content, and instead redirect customers to Wells Fargo's main site at wellsfargo.com. The sole purpose of registering these domains appears to be to stop any other party from doing so, which protects the bank's brand and prevents the domains being used to host phishing sites.

In a similar move, Microsoft has also registered several .hosting domains including xbox.hosting, bing.hosting, windows.hosting, skype.hosting, kinect.hosting and dynamics.hosting. Browsing to any of these domains causes the user to be redirected to bing.com, which displays search results for the second-level string (i.e. "xbox", "windows", etc.).

Of course, with many other new TLDs continually popping up, brand protection becomes an increasingly costly exercise. Microsoft has also recently registered hundreds of other nonsensical domains which are used to redirect browsers to bing.com, such as lumia.ninja, lync.lawyer, xboxone.guitars, windowsphone.futbol, microsoft.airforce, azure.luxury, yammer.singles, xboxlive.codes, halo.tattoo, internetexplorer.fishing, and so on.

However, the race to register domain names is not always won by Microsoft — bing.click is a prime example of a domain that someone else got to first. This domain is currently offered for sale, highlighting the fact that it's not just ICANN and the registrars that stand to gain money from the influx of new TLDs.

Total number of websites

Web server market share

DeveloperDecember 2014PercentJanuary 2015PercentChange
Apache358,159,40539.11%348,460,75339.74%0.63
Microsoft272,967,29429.81%241,276,34727.52%-2.29
nginx132,467,76314.47%128,083,92014.61%0.14
Google20,011,2602.19%20,209,6492.30%0.12
Continue reading

December 2014 Web Server Survey

In the December 2014 survey we received responses from 915,780,262 sites and 5,034,578 web-facing computers.

This is the second month in a row where there has been a large drop in the total number of websites, giving this month the lowest count since January. As was the case in November, the loss has been concentrated at just a small number of hosting companies, with the ten largest drops accounting for over 52 million hostnames. The active sites and web facing computers metrics were not affected by the loss, with the sites involved being mostly advertising linkfarms, having very little unique content. The majority of these sites were running on Microsoft IIS, causing it to overtake Apache in the July 2014 survey. However the recent losses have resulted in its market share dropping to 29.8%, leaving it now over 10 percentage points behind Apache.

Despite losing more than six million hostnames this month, nginx outpaced all other major server vendors by gaining 22,300 web-facing computers. nginx is now used by nearly 11% of all web-facing computers – twice the share that it had two years ago.

Overall, the total number of web-facing computers in our survey increased by just over 40,000 this month, making nginx responsible for more than half the increase. Despite an increase of over 11,000 computers for Apache, and 1,700 for IIS, both continue to lose market share.

Thanks to continued strong growth at Amazon Web Services, Amazon is the largest hosting company by a considerable margin in terms of our web-facing computers metric (which includes web-facing virtual machines, providing that each has its own kernel and IP address). With nearly 300,000 web-facing computers in total, Amazon has just over twice as many as second-place OVH. In October, we reported that DigitalOcean had become the 4th largest hosting company in under 2 years, but it quickly reached third place in November and is continuing to close the gap on OVH.

Cloud growth

Both Amazon Web Services and Microsoft Azure expanded their cloud hosting footprints recently. Amazon opened a new European AWS region in Frankfurt, which augments its existing EU region in Ireland. Besides being able to host services closer to the center of Europe, the new region means that customers can now build multi-region applications with the assurance that their data will stay within the EU. The new Frankfurt region houses two EC2 availability zones and three AWS edge locations.

Microsoft's new Azure "geo" is in Australia, and consists of two geographically redundant regions in New South Wales and Victoria. This will help Microsoft to compete with Amazon's two EC2 availability zones in Sydney.

New TLDs

More new top-level domains showing strong growth in this month's survey include .nyc, which is targeted for use by New Yorkers, and .realtor, which is only allowed to be used by members of the National Association of Realtors or the Canadian Real Estate Association. These have grown from virtually nothing to a total of 40,000 and 80,000 sites respectively.

Ironically, one of this month's fastest growing new top-level domains started off as an April Fool's Day joke in 2009, when the founder of OVH announced the creation of the .ovh TLD – years before such things were actually possible. This joke resulted in over 22,000 requests to register .ovh domains within a few hours, demonstrating the potential demand for such domains. OVH eventually entered into a Registry Agreement with ICANN in January 2014, and the sunrise period for the .ovh TLD began in September. This month's survey saw the number of sites using .ovh domains grow from 6,000 to 63,000, likely due to the first 50,000 .ovh domains being given away for free, and with subsequent growth being fuelled by attractive pricing: new .ovh domains can currently be registered for only EUR 0.99 per year and renewed for EUR 1.99.

Total number of websites

Web server market share

DeveloperNovember 2014PercentDecember 2014PercentChange
Apache350,853,79837.05%358,159,40539.11%2.06
Microsoft306,029,30732.31%272,967,29429.81%-2.51
nginx139,130,99214.69%132,467,76314.47%-0.23
Google19,560,2062.07%20,011,2602.19%0.12
Continue reading

November 2014 Web Server Survey

In the November 2014 survey we received responses from 947,029,805 sites and 4,994,577 web-facing computers.

Despite the dramatic loss of almost 82 million sites since October, the number of web-facing computers increased by over 23 thousand (+0.47%) this month. Automatically generated content and wildcard sites are often used for activities such as domain holding and search-engine results manipulation, where the majority of the sites will not receive human visitors and therefore the resources required to run the sites are minimal. This is clearly demonstrated this month, with over half of the net loss of sites being attributed to a single IP address that was previously hosting parked websites.

The most significant contributor to the increase in web-facing computers was nginx, with almost 16k more computers using the web server this month. Apache and Microsoft both experienced small losses in market share as a result, continuing the trend seen over the past few years.

nginx gained a number of notable high-traffic websites in this month's survey. After several years of using lighttpd, the SourceForge download site (downloads.sourceforge.net, traffic rank #411) has switched to nginx, and gaming news site pcgamer.com (#1339) has also started using nginx with Rackspace's Cloud Load Balancing as a Service product.

Amongst the top million websites, Apache's market share has continued its slow decline. Ever since Netcraft started publishing these figures, Apache has commanded more than half of this market sector. This month, however, Apache has reached its lowest market share ever: 50.01%. Although it still clings on to more than half of the market this month, it looks unlikely to retain it for long. Microsoft and Google have also been losing share amongst the top million sites, all making room for nginx, which has muscled its way up to 20.4%.

New top level domains are continuing to add a modest number of sites to the survey each month. One of this month's fastest growing new TLDs is .audio, which is run by Uniregistry, where it is pitched as a dedicated online space for sound, musicians, engineers and producers. More than 17,000 sites are already using this TLD.

German states and cities are also contributing to the growth in new TLDs. For example, .koeln is now used by 14,000 sites, and .bayern is already used by more than 10,000 websites since it entered general availability on 30 September 2014.

Just in time for this year's Black Friday events on November 28th, the .blackfriday TLD was also launched recently by Uniregistry. More than 10,000 sites have adopted this domain in the space of a month, including amazon.blackfriday and target.blackfriday, which redirect to each company's main US website. Uniregistry also provides another seasonal TLD, .christmas, which was found to be used by 12,000 sites in this month's survey.

NCC Group released its 124-page .trust Technical Policy [pdf] last month, which outlines a stringent set of criteria which must be met by registrants of .trust domains. Many of these policies relate to security, dictating both what must and must not be done – the table of contents alone contains 71 instances of "Do not". The policies go far beyond the typical fraud prevention requirements set by most other registries, with a heavy emphasis on web application security. For example, websites which use .trust domains must serve all content over an encrypted HTTPS connection, must not use inline JavaScript or the eval() function, must provide an appropriate Content Security Policy header, and must be free from open redirects.

NCC Group bought the .trust TLD earlier this year from Deutsche Post, which had originally obtained it from ICANN in 2013. With NCC Group claiming that security compliance is externally verifiable, it anticipates that all sites using the .trust TLD will be recognised as trustworthy.

Total number of websites

Web server market share

DeveloperOctober 2014PercentNovember 2014PercentChange
Apache385,354,99437.45%350,853,79837.05%-0.40
Microsoft345,485,41933.58%306,029,30732.31%-1.26
nginx148,330,19014.42%139,130,99214.69%0.28
Google19,431,0261.89%19,560,2062.07%0.18
Continue reading