July 2015 Web Server Survey

In the July 2015 survey we received responses from 849,602,745 sites and 5,350,323 web-facing computers. This represents a net loss of 13.5 million websites, but a gain of 3,700 additional computers.

One of the most significant changes in July was the net loss of nearly 13,000 web-facing computers powered by Microsoft web server software, accompanied by a decline of more than 29 million hostnames. The loss was predominantly seen for servers running Microsoft IIS 6.0, 7.0 and 7.5. These versions of IIS are used by Windows Server 2003, which is no longer supported, and Windows Server 2008 (including 2008 R2), whose mainstream support ended in January. The latest stable release of IIS (version 8.5) is however continuing to grow, this month increasing by over 9,000 web facing computers.

This month's decline has brought Microsoft's market share of hostnames down by nearly 3 percentage points, increasing Apache's lead. However, Apache's own market share also fell slightly, largely due to gains made by nginx and Tengine.

nginx gained 8.5 million sites this month, but more remarkably, it gained over 14,000 web-facing computers, with the largest gains in the US, China, Germany and the UK. Compounding Microsoft's losses, nearly 1.8 million existing websites switched from using Microsoft IIS to nginx in July.

nginx also fared well amongst the top million websites, where it gained a further 3,771 sites, causing losses for Apache, Microsoft and Google. Nonetheless, Apache is still used by nearly half of the top million sites, with its market share being almost 26 percentage points ahead of nginx.

Tengine now powers more websites than Google's web server software, after the number of sites using it grew by 7 million to a total of more than 25 million this month. The open source Tengine web server is based on nginx, and used extensively by the online marketplace Taobao. It currently supports all features found in nginx 1.6.2, plus several other features required by Taobao that were not able to be implemented as nginx modules. Neither nginx nor Tengine support HTTP/2 yet, but they were both early supporters of Google's SPDY protocol, on which HTTP/2 is based. nginx plans to provide support for HTTP/2 by the end of this year, and so it is likely that Tengine may also follow suit at a later date.

Tengine 2.1.0 is the latest development version of Taobao's nginx fork, but despite being released more than six months ago, only 25,000 websites currently claim to be using it. In contrast, Tengine 1.4.2 — which was released in 2012 and is also a development version — is used by nearly 10 million sites, making it by far the most commonly deployed version. The latest stable release, Tengine 1.5.2, is the second most commonly used version, but accounts for just under 200,000 sites.

But like Apache, more than half of the sites running Tengine do not reveal which version they are running, and so the true distribution of version numbers could vary greatly. For instance, 2.7 million of these version-less Tengine websites are used to host Taobao stores directly under the taobao.com domain (e.g. baobeiit.taobao.com). Given that Tengine was created by Taobao in order to provide the features they need, it is not unreasonable to assume that these sites might be using the latest release, or at least a relatively recent one.

Despite being used by a large number of sites, Tengine was found on only 4,240 web-facing computers in July 2015. Three-quarters of these computers are located in China, while nearly 10% are located in the US.

Total number of websites

Web server market share

DeveloperJune 2015PercentJuly 2015PercentChange
Apache334,731,03538.78%325,696,51438.34%-0.45
Microsoft254,408,17929.48%225,282,71326.52%-2.96
nginx122,965,52214.25%131,460,06315.47%1.23
Google20,130,7322.33%20,255,4242.38%0.05
Continue reading

Most Reliable Hosting Company Sites in June 2015

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 Qube Managed Services Linux 0:00:00 0.000 0.102 0.052 0.104 0.104
2 ServerStack Linux 0:00:00 0.000 0.074 0.076 0.152 0.152
3 Bigstep Linux 0:00:00 0.005 0.117 0.060 0.123 0.123
4 iWeb Linux 0:00:00 0.005 0.135 0.078 0.156 0.156
5 Anexia Linux 0:00:00 0.005 0.541 0.085 0.172 0.172
6 GoDaddy.com Inc Linux 0:00:00 0.010 0.121 0.009 0.022 0.023
7 Datapipe Linux 0:00:00 0.015 0.095 0.013 0.027 0.037
8 Netcetera Windows Server 2012 0:00:00 0.015 0.055 0.083 0.165 0.165
9 LeaseWeb Linux 0:00:00 0.025 0.224 0.028 0.061 0.061
10 One.com Linux 0:00:00 0.025 0.166 0.058 0.217 0.218

See full table

Qube Managed Services had the most reliable website during June, responding successfully to all of Netcraft's requests. This is Qube's fourth appearance in the top ten in 2015, continuing its strong showing from 2014 when it placed in the top ten in eleven months, and came first on four occasions. Qube is based in London and offers managed private cloud hosting services from datacentres in London, New York and Zurich.

In second place, ServerStack also successfully responded to all requests in June, placing second only as a result of a slightly slower average connection time. ServerStack provides managed hosting services to enterprises from three datacentres in Amsterdam, New Jersey and San Jose. It has appeared in the top 10 list frequently in the past few years.

Bigstep, iWeb and Anexia also did well this month, each responding to all but one request.

Linux remains the most popular choice of operating system, with 9 of the top 10 companies using the OS to power their website, while the remaining one uses Windows Server 2012.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

June 2015 Web Server Survey

In the June 2015 survey we received responses from 863,105,652 sites and 5,346,650 web-facing computers, representing an increase of 5.2 million websites and 65,000 additional computers.

Microsoft was responsible for the majority of this month's hostname growth, with a gain of 6.6 million sites, but only contributed 11,700 additional web-facing computers. This has caused Microsoft's market share by hostnames to overtake its declining market share by computers, with both standing at just under 30%.

Apache led this month's web-facing computer growth with a net gain of 24,800, while nginx followed closely with 22,800. This has resulted in nginx's market share increasing by 0.28 to 12.4%, and despite showing the largest net growth, Apache's share fell slightly.

Apache, Microsoft and nginx together account for more than 88% of all web-facing computers in the world, making these vendors by far the most popular choices. However, nginx is the only vendor experiencing consistent increases in market share, up by 3 percentage points over the last year while both Apache and Microsoft have seen losses. The next most commonly used server is lighttpd (pronounced "lighty"), which is used by a mere 0.46% of web-facing computers.

nginx's market share has also been steadily increasing within the top million websites. Its share now stands at 21.9%, and although Apache's use within the million busiest sites has been steadily declining this decade, Apache looks likely to retain the lead for at least a few more years.

Three months after the death of Sir Terry Pratchett, approximately 84,000 websites are now serving the X-Clacks-Overhead: GNU Terry Pratchett header in tribute. Invisible to the majority of users, this HTTP header is a reference to the Discworld novel Going Postal, which features a series of communication towers called the clacks.

In the book, a similar header ("GNU John Dearheart"), is transmitted around the clacks after the inventor's son is killed in an accident while working on a clacks tower. The G means send the message on, N means do not log the message, and U means turn the message around at the end of the line and send it back again — this ensures that the message is transmitted indefinitely, allowing his son to be memorialised forever. Similarly, by transmitting Pratchett's name around the internet, the sites participating in this HTTP header tribute hope to keep his legacy alive. After all, as it says in the book, "A man is not dead while his name is still spoken."

One of the most popular sites to use the X-Clacks-Overhead: GNU Terry Pratchett header is www.theguardian.com, which alone reached more than 5 million unique browsers per day in 2014. With each header taking up 40 bytes of an uncompressed HTTP request, all of the sites involved in the tribute could be generating terrabytes of additional bandwidth usage every day.

Total number of websites

Web server market share

DeveloperMay 2015PercentJune 2015PercentChange
Apache336,813,95939.26%334,731,03538.78%-0.48
Microsoft247,784,66828.88%254,408,17929.48%0.59
nginx123,697,64514.42%122,965,52214.25%-0.17
Google20,103,0682.34%20,130,7322.33%-0.01
Continue reading

PayPal.com unexpectedly offline

PayPal.com experienced some unexpected outages today, leaving many of its 165 million active customers unable to access the site via its paypal.com domain.

p-3894.652

All of Netcraft's geographically distributed performance monitors have showed outages of at least an hour so far today.

But requests to www.paypal.com continued to work throughout the outage, and customers could successfully log in by browsing directly to that hostname; however, it is likely that many users visit the site by typing only "paypal.com" into their browser's address bar.

The shorter paypal.com address does not serve a homepage; its sole purpose is to redirect visitors directly to the main www.paypal.com website. This shortcut saves the user having to type "www." into his browser, but during the outage, anyone attempting to reach the site via this commonly used method would have faced error messages similar to this (unless the redirect has been cached from a previous visit):

timeout

Even a short outage can have a huge impact on a financial giant like PayPal. In 2014, PayPal handled 4 billion payments with a total payment volume of $235 billion.

The cause of the outage is as yet unknown: The PayPal Notifications website does not show any scheduled maintenance for the live site until tomorrow, and does not currently list any unscheduled events corresponding to today's outage.

status

阿里云的发展壮大已让阿里集团成为中国最大的服务器主机提供商

[Read this article in English]

中国云主机服务提供商阿里云正在以前所未有的速度发展壮大。从今年4月到5月仅一个月的时间,从阿里云直接连接到国际互联网的Web服务器数量就增长了8千多台。这样的迅猛增长已让其母公司阿里巴巴集团一跃成为世界第四大同时也是中国第一大服务器主机提供商。

Continue reading

Most Reliable Hosting Company Sites in May 2015

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 EveryCity SmartOS 0:00:00 0.008 0.079 0.066 0.133 0.133
2 Datapipe Linux 0:00:00 0.017 0.085 0.012 0.024 0.032
3 Anexia Linux 0:00:00 0.017 0.633 0.088 0.176 0.176
4 CWCS Linux 0:00:00 0.017 0.188 0.104 0.186 0.187
5 XILO Communications Ltd. Linux 0:00:00 0.030 0.179 0.064 0.129 0.129
6 INetU Windows Server 2008 0:00:00 0.038 0.098 0.068 0.199 0.404
7 Netcetera Windows Server 2012 0:00:00 0.038 0.056 0.081 0.161 0.161
8 New York Internet FreeBSD 0:00:00 0.047 0.186 0.030 0.062 0.180
9 Hyve Managed Hosting Linux 0:17:02 0.051 0.156 0.064 0.127 0.128
10 Umbee Hosting Linux 0:00:00 0.076 0.072 0.064 0.481 1.231

See full table

EveryCity had the most reliable website during May 2015, responding to all but two of Netcraft's requests. Since Netcraft began monitoring its site in April 2014, EveryCity has maintained a 100% uptime record. Besides SmartOS-based hosting, EveryCity also specialises in Solaris hosting on both SPARC and Intel platforms.

In second place is Datapipe, with just four failed requests to its website during May 2015. Two other companies had the same number of failures: Anexia and CWCS. Datapipe's impressively quick average connection time (0.012 seconds) gives it the edge in the tie-breaker, fending off both Anexia (0.088 seconds) and CWCS (0.104 seconds). Datapipe has data centres in key technology and financial hubs including New York, Silicon Valley, London, and Hong Kong.

Anexia (3rd place) and CWCS (4th place), both with four failed requests and both of which are based in Europe, differ in their focus: Anexia operates 58 data centres across the world, whereas CWCS concentrates on the UK market, owning two "state of the art" British data centres. Anexia announced in early May that it had become an official Debian mirror, providing support to the open-source project.

Making it into the top 10 for the first time since its monitoring began in February 2015, Umbee Hosting's website responded to all but a handful of Netcraft's requests. Umbee Hosting has five data centres, spread across three continents: two in the London region, two in metro New York, and one in Sydney.

Linux remains a popular choice amongst the top 10 hosting company websites, powering six. The use of SmartOS and FreeBSD brings up the total of UNIX-based operating systems to eight. The remaining two are powered by Windows Server 2008 and 2012.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.