February 2014 Web Server Survey

In the February 2014 survey we received responses from 920,102,079 sites — over 58 million more than last month.

Microsoft gained a staggering 48 million sites this month, increasing its total by 19% — most of this growth is attributable to new sites hosted by Nobis Technology Group. Along with Microsoft, nginx also made a large gain of 14 million sites, whereas Apache fell by 7 million. Unsurprisingly, these changes have had a dramatic effect on the overall market share of each web server vendor, with Microsoft's share growing by 3.38 percentage points to 32.8% (302 million sites) while Apache's has fallen by 3.41 to 38.2% (352 million sites).

Microsoft's market share is now only 5.4 percentage points lower than Apache's, which is the closest it has ever been. If recent trends continue, Microsoft could overtake Apache within the next few months, ending Apache's 17+ year reign as the most common web server. Apache is faring much better in both the active sites and top million sites datasets, however, where it is still dominating with just over half of the market share in both metrics.

Nearly 2% of the top million websites are now being served by CloudFlare's customised version of nginx (cloudflare-nginx), which it uses to serve web content via its globally distributed CDN edge nodes. This month's survey saw more than a thousand of the top million sites migrate to cloudflare-nginx from other web server software, including pizzahut.co.uk, pet-supermarket.co.uk, the image server used by the popular Cheezburger network of blogs, and the official PRINCE2 website which switched from Microsoft IIS 6.0 running on Windows Server 2003.

Overall, nginx powers 17.5% of the top million sites, including popular overclocking forum www.overclock.net, despite its server headers declaring that it is now using Microsoft IIS 4.1. Responses from the server also include an X-Powered-By header which claims the application is running on Visual Basic 2.0 on Rails (Visual Basic 2.0 is a long-deprecated language which was released more than 20 years ago, while Microsoft IIS 4.1 never actually existed). The server claimed to be running nginx during Netcraft's previous survey, and indeed, it exhibits characteristics which suggest it is still using nginx.

The number of sites using the .pw country-code top-level domain (ccTLD) grew by more than half this month, reaching 10M sites in total. This ccTLD is assigned to Palau, but the .pw registry has branded the domain as the Professional Web and allows domains to be registered by the general public, regardless of which country they are in. 97% of this month's new .pw sites are hosted in the US (87% at Nobis Technology Group alone), and 2.7 million of them are running on Windows.

The busiest .pw domain is the single-letter u.pw, which is used by viral social media site Upworthy. Other than that, the ccTLD remains relatively obscure, with less than 0.02% of the top million sites using .pw domains, while other single-letter domains have been sold for the modest sums of $8,000 each. Only two single-letter .pw domains actually appear within the top million sites, and there are no two-letter domains at all. Last year, Symantec noted an increase in spam messages containing URLs with .pw TLDs, and .pw later became the first TLD to adopt the Uniform Rapid Suspension (URS) rights protection mechanism.

Version 1.0 of the URS Technical Requirements were published by ICANN in October 2013, and are intended to make it faster and cheaper for trademark holders to seek resolution when there are obvious cases of infringement. URS is intended to complement rather than replace the existing Uniform Domain-Name Dispute-Resolution Policy (UDRP).

DeveloperJanuary 2014PercentFebruary 2014PercentChange
Continue reading

January 2014 Web Server Survey

In the January 2014 survey we received responses from 861,379,152 sites, an increase of 355,935 since last month.

2013 has been a year of significant change: the web has grown by more than one third, the importance of SSL has been highlighted by a series of spying revelations, Microsoft now power just below 30% of all web sites, and Apache has lost almost 14 percentage points of market share. Additionally, nginx, the relative newcomer, saw its market share peak at 16%, just shy of Microsoft’s position at the beginning of last year.

The total number of web sites discovered has increased dramatically this year — from 630 million web sites in January 2013 to 861 million in January 2014 (+37%) — though the growth does not compare to the doubling in size during 2011.

With the revelations from the NSA documents leaked by Edward Snowden providing months of mainstream publicity, 2013 has been a bumper year for the SSL industry. Websites are increasingly being served over HTTPS: 48% more sites within the million busiest are using SSL than in January 2013. In total, there are over half a million more SSL certificates (+22%) in use on the web since January 2013. The estimated total revenue of the industry has increased even more rapidly, by 28% from September 2012 to September 2013, reflecting the increased uptake of more expensive certificates including Extended Validation, multi-domain, and wildcard certificates.

Apache remains the most commonly used web server on the internet, 10 million more web sites are using it than this time last year; however, this growth has not been sufficient to maintain its share of a market which grew by more than 200 million web sites. As a result, Apache’s market share has fallen by 14 percentage points since January and now stands at 42%.

In stark contrast to Apache, Microsoft had a strong year — almost 150 million more web sites use a Microsoft web server than in January 2013. Microsoft’s share is close to 30% of the entire market and a combination of its strong growth and the corresponding lack of growth of sites using Apache has resulted in Apache’s lead shrinking by more than 26 percentage points to just 12. Microsoft's own cloud platform, Azure, has grown steadily throughout 2013 — there are 39% more web-facing computers hosted by Microsoft in January 2014 than the same time last year — and despite offering alternatives, Microsoft's IIS is by far the most common web server on Azure.

Open-source web server nginx has continued to gain acceptance, especially amongst the busiest web sites. Nginx is now used on 14% of all web sites found, up 2 percentage points since January 2013, but has fallen slightly from the peak of 16% it achieved in October. In May 2013, nginx overtook Microsoft to become the second most common web server within the top million busiest sites and now powers almost 16% of them.

DeveloperDecember 2013PercentJanuary 2014PercentChange
Continue reading

Most Reliable Hosting Company Sites in December 2013

Rank Performance Graph OS Outage
DNS Connect First
1 Qube Managed Services Linux 0:00:00 0.004 0.100 0.043 0.087 0.088
2 Hosting 4 Less Linux 0:00:00 0.004 0.171 0.124 0.245 0.627
3 New York Internet FreeBSD 0:00:00 0.007 0.140 0.074 0.148 0.577
4 Pair Networks FreeBSD 0:00:00 0.007 0.234 0.083 0.169 0.572
5 www.dinahosting.com Linux 0:00:00 0.007 0.245 0.087 0.177 0.177
6 Webair Internet Development FreeBSD 0:00:00 0.011 0.167 0.073 0.154 0.353
7 Server Intellect Windows Server 2012 0:00:00 0.011 0.073 0.101 0.330 0.701
8 XILO Communications Ltd. Linux 0:00:00 0.019 0.202 0.068 0.136 0.234
9 Swishmail FreeBSD 0:00:00 0.019 0.126 0.074 0.146 0.192
10 ServerStack Linux 0:00:00 0.019 0.088 0.076 0.151 0.152

See full table

Qube Managed Services had the most reliable hosting company site in December 2013, making December Qube's fourth consecutive month in the top ten after attaining first place in September. Qube provides managed hosting services out of data centers in London, New York and Zurich. Qube and Hosting 4 Less, which placed second, both saw only a single failed request over the month with Hosting 4 Less losing out on the top spot due to a slightly longer average connection time (0.04s vs. 0.12s).

Swishmail made an appearance at ninth in the table, taking its total number of appearances in the top ten in 2013 to nine. This means that both iWeb and Swishmail jointly hold the record for the most frequent appearances in the top ten in 2013. Swishmail provides corporate email services on a FreeBSD platform, while iWeb provides dedicated servers, managed hosting and colocation services on a Linux platform.

To sign off the year, WebAir made its first appearance in the table for 2013, placing sixth. WebAir offers hosting services out of data centers in Los Angeles, Amsterdam, Montreal and a 'flagship' facility in New York (NY1), which WebAir claims offers 'the lowest network latency to Europe via direct feeds to Transatlantic fiber'.

Five of the top ten most reliable hosting company sites were running Linux, while four ran FreeBSD. Server Intellect ran the single Windows Server-based site to complete the list: Windows Server makes a reappearance after not placing in the top ten in November.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

DigitalOcean now growing faster than Amazon

Cloud computing provider DigitalOcean is now growing faster than Amazon Web Services. Our December 2013 Web Server Survey showed a month-on-month gain of 6,514 web-facing computers at DigitalOcean; Amazon, meanwhile, grew by an almost as huge 6,269 web-facing computers. Together, the two companies accounted for more than a third of the internet-wide growth in web-facing computers in December.

DigitalOcean is now the 15th largest hosting company in terms of web-facing computers — a remarkable feat considering DigitalOcean had only 280 web-facing computers at the start of this year. Although Amazon is still the largest hosting company (by web-facing computers) and has nearly six times as many web-facing computers in total, the rapid growth at DigitalOcean may have startled those at Amazon who thought their major competitors were Microsoft Azure and Rackspace.

Whilst DigitalOcean competes directly with Amazon EC2, there are a number of Amazon Web Services which do not have a DigitalOcean equivalent. For example, Amazon offer file storage (S3), load balancing (Elastic Load Balancing), and a Content Delivery Network (CloudFront). However, by simplifying their offering — lack of support for Microsoft Windows is notable — and not offering enterprise features, DigitalOcean appeals to users with straightforward requirements such as small businesses and developers.

The cheapest virtual computer ("droplet") at DigitalOcean uses solid state storage and costs less than one cent per hour, about a third of the price of Amazon's cheapest on-demand instance. Unsurprisingly, such competitive pricing is attracting a large number of completely new customers as well as enticing other hosting companies' customers to switch to DigitalOcean.

Sites migrating from Amazon to DigitalOcean

818 existing websites migrated from Amazon to DigitalOcean this month, whereas only 88 sites moved in the opposite direction. Although the difference seems significant, the largest gains at DigitalOcean actually consist of new sites: DigitalOcean is currently hosting 490,000 websites, 120,000 of which were not present in last month's survey as per the Netcraft December 2013 Hosting Provider Switching Analysis.

Sites which migrated from Amazon to DigitalOcean include text messaging service Phonify (plus its API at api.phonify.io), several Windshield Guru sites, and real-time crowd photo sharing site zingly.

DigitalOcean growth trends and a timeline of events can be viewed at http://trends.netcraft.com/www.digitalocean.com

December 2013 Web Server Survey

In the December 2013 survey we received responses from 861,023,217 sites, an increase of 75.7M since last month.

For the third consecutive month Microsoft experienced the largest growth in web server market share; an additional 51M sites boosted its market share by almost 4 percentage points. Apache had the biggest loss in market share, despite seeing an increase of 7M sites. Nginx’s growth this month has led to them regaining some of the market share lost last month, and results in a net gain of 3M sites over the last two months (a loss of 13M in November, followed by a gain this month of 16M).

Considering only active sites, Microsoft’s growth was significantly lower (compared to its gain of 51M sites) with an increase of 478k active sites. Apache saw the largest increase of 622k active sites, and increased its market share by 2 percentage points to 54%. Google meanwhile saw a large loss of 8.6M active sites within its App Engine service.

Microsoft's web server market share could see further gains from its own Windows Azure cloud service — Microsoft has made Azure more cost-effective with price reductions and a new enterprise plan. Microsoft also has plans to expand Azure into Brazil in 2014. Unsurprisingly Microsoft IIS is the most common Web Server seen on Azure, used by 87% of sites. Amazon followed Microsoft's price cuts by introducing price reductions on EC2 M3 Instances. At Amazon, 36% of sites are running on nginx, followed by Apache with 27% and Microsoft IIS with 14%.

ICANN has begun delegating new generic Top-Level Domains (gTLDs) from its new gTLD expansion program, adding them into the Internet's Root Zone. The first four gTLDs (شبكة, онлайн, сайт, 游戏) were delegated on October 23rd and a further 30 new gTLDs have been delegated throughout November. ICANN has a mandatory 30-day sunrise period following delegation where registries "complete a final process built into the new gTLD Program to protect trademark rights holders"; after this has passed the new gTLD can be made available to the general public at the registry's discretion. The شبكة dotShabaka registry was first to start its sunrise period on 31 October, this is to be followed by an (optional) Landrush Period before general availability commences on February 4th. So far the only sites found by Netcraft have been for the NICs (Network Information Center) themselves, such as: nic.游戏, nic.сайт, nic.онлайн, nic.شبكة.

The first gTLDs signed agreements back in July and more are being agreed each month. Recent notable gTLDs that have been agreed include the cities .london, .berlin, .budapest, .wein (Vienna) and .tokyo.

DeveloperNovember 2013PercentDecember 2013PercentChange
Continue reading

Most Reliable Hosting Company Sites in November 2013

Rank Performance Graph OS Outage
DNS Connect First
1 Krystal Hosting Linux 0:00:00 0.007 0.137 0.091 0.189 0.189
2 Swishmail FreeBSD 0:00:00 0.010 0.128 0.077 0.151 0.201
3 iWeb Linux 0:00:00 0.014 0.144 0.089 0.172 0.172
4 dinahosting Linux 0:00:00 0.014 0.218 0.097 0.194 0.194
5 Kattare Internet Services Linux 0:00:00 0.014 0.182 0.124 0.246 0.506
6 Datapipe FreeBSD 0:00:00 0.017 0.080 0.019 0.039 0.059
7 ServerStack Linux 0:00:00 0.017 0.088 0.076 0.151 0.151
8 Qube Managed Services Linux 0:00:00 0.024 0.117 0.056 0.112 0.112
9 New York Internet FreeBSD 0:00:00 0.024 0.140 0.074 0.149 0.570
10 Bigstep Linux 0:00:00 0.028 0.284 0.073 0.155 0.230

See full table

With only two failed requests, Krystal Hosting was the most reliable hosting company in November 2013. Krystal, which was founded in 2002 by Simon Blackler when he was just 17 years old, recently launched a VPS hosting service, Kloud. The UK-based hosting company has placed great emphasis on security — it is currently based in an underground ex-military bunker outside London which is protected by former members of the military and police services.

Swishmail was second most reliable hosting company in November, with only three failed requests. Swishmail primarily provide a business email service on a FreeBSD-based platform. Earlier this year it became a bronze sponsor of the FreeBSD project.

Close behind Swishmail came iWeb (third most reliable), dinahosting (fourth), and Kattare Internet Services (fifth). As all three hosting companies had the same number of failed requests in November the tie was broken by examining the average connection time.

Datapipe ranked 6th with five failed requests. It is regularly one of the fastest and most reliable hosting companies we monitor. Datapipe has more than seven years of continuous uptime on www.datapipe.net.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.