September 2017 Web Server Survey

In the September 2017 survey we received responses from 1,805,060,730 sites and 6,798,101 web-facing computers, reflecting a gain of 4.49 million sites and 84,800 computers. This is the largest number of web-facing computers in history, and cloud computing has helped fuel this growth considerably. The top three cloud hosting providers—Amazon, Alibaba and DigitalOcean—now account for nearly 22% of all web-facing computers, as well as more than a third of this month's net computer growth.

nginx has dominated this month's computer growth once again, with an additional 49,900 computers taking its market share up to 22.2%. This puts it within 20,000 computers of Microsoft's 22.5% share, giving nginx a realistic chance of taking second place by next month. Meanwhile, Apache remains far ahead in the lead with 2.88 million web-facing computers giving it a 42.3% share of the market.

While more than half of the websites in the survey are using Microsoft web server software, relatively few of these are active sites. Discounting link farms, domain holding pages and other automatically generated content, Microsoft accounts for only 7.3% of all active sites, while Apache leads with 44.9%, and nginx follows with 20.7%. Microsoft's active sites share has never exceeded Apache's, and ever since it peaked at 38% in early 2009, it has experienced a general decline.

Microsoft's presence within the top million websites has also been steadily declining – in 2008, it had a 20% share, but this has since dwindled to less than 10%. Today, 9.73% of the top million sites use Microsoft web server software (mostly IIS – Internet Information Services), while Apache leads with 38.9%. However, Apache has also been experiencing a general decline since 2011, while nginx has continued to muscle its way in. A further 1,299 nginx-powered sites made their way into the top million this month, lifting nginx's share to 29.4%.

Web-facing operating systems

Apache has been the most commonly installed web server for many years, but which underlying operating systems are used by the world's web facing computers?

By far the most commonly used operating system is Linux, which runs on more than two-thirds of all web-facing computers. This month alone, the number of Linux computers increased by more than 91,000; and again, this strong growth can largely be attributed to cloud hosting providers, where Linux-based instances are typically the cheapest and most commonly available.

It is difficult to distinguish different Linux distributions by examining their TCP/IP characteristics, but inferences can be made from other clues, such as Server headers. The Linux distribution can be positively identified in around 30% of cases, and of these 1.39 million Linux computers, just over half are running Ubuntu Linux, nearly a quarter are running CentOS, and around fifth are running Debian Linux.

Microsoft is next up, with 26.8% of all web-facing computers running various versions of Windows Server. Interestingly, this is higher than Microsoft's web server share (22.5%), and can be explained by around 20% of Windows computers being used to run other web server software – such as Apache and nginx – rather than its native Microsoft IIS web server. Conversely, around 51,000 Linux computers expose Microsoft IIS web servers to the internet – in these cases, the Linux frontends are typically acting as load balancers or reverse proxies to backend Windows computers running IIS.

Windows Server 2008 is the most commonly used version of the Windows Server family of operating systems – it currently runs on 971,000 web-facing computers, and this is followed by Windows Server 2012, which is installed on 505,000.

Windows and Linux are essentially the "big two" when it comes to web-facing operating systems. FreeBSD was once notorious for its reliability and impressively large uptimes when used as a server platform; indeed, Netcraft's infrastructure made extensive use of it in the past, but it is now a relatively niche operating system compared with its heyday. Today it is used by only 1.3% of web-facing computers, more than half of which are being used to run the Apache web server.

Total number of websites

Web server market share

DeveloperAugust 2017PercentSeptember 2017PercentChange
Microsoft912,072,08550.65%927,540,45451.39%0.73
Apache356,768,92819.81%329,105,83218.23%-1.58
nginx280,659,49815.59%315,530,74617.48%1.89
Google20,680,1071.15%20,906,8491.16%0.01
Continue reading

Most Reliable Hosting Company Sites in August 2017

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 Swishmail FreeBSD 0:00:00 0.000 0.139 0.058 0.116 0.156
2 Qube Managed Services Linux 0:00:00 0.000 0.138 0.058 0.116 0.116
3 One.com Linux 0:00:00 0.004 0.196 0.037 0.104 0.104
4 EveryCity SmartOS 0:00:00 0.004 0.107 0.064 0.290 0.290
5 Pair Networks FreeBSD 0:00:00 0.004 0.243 0.071 0.141 0.141
6 ReliableServers.com Linux 0:00:00 0.009 0.171 0.005 0.157 0.157
7 SimpleServers Linux 0:00:00 0.009 0.105 0.005 0.142 0.142
8 Lightcrest unknown 0:00:00 0.009 0.172 0.020 0.048 0.146
9 Hyve Managed Hosting Linux 0:00:00 0.009 0.091 0.059 0.122 0.122
10 Rackspace F5 0:00:00 0.009 0.161 0.074 0.150 0.150

See full table

Swishmail had the most reliable hosting company website in August 2017. It responded to all of Netcraft's requests from several points around the internet, with not a single failure, and an average connect time of 57.6 milliseconds. The US-based company provides email and web hosting services based on FreeBSD, and this is its fourth appearance in the top ten so far this year.

There was an incredibly close battle for first place this month, as Qube Managed Services also managed to respond to all of Netcraft's requests, but its average connect time was fractionally longer at 57.7 milliseconds – less than 0.2 milliseconds slower than Swishmail's average. This puts Qube in second place, but only just. Impressively, this is Qube's seventh time in the top ten for 2017. It operates data centres in London, Zurich and New York, where it offers managed hosting, colocation, and cloud-based solutions.

In third place, with a single failed request, was One.com. Its average connect time was noticeably faster than those of Swishmail and Qube; but reliability trumps speed in this contest. This is One.com's fifth top-ten appearance this year, including another first place ranking in April. Founded in 2002, the company was established in Denmark, and offers a variety of internet services including 1-click WordPress installations, domain registrations, email hosting, and Office 365.

Five of this month's top-ten hosting companies used the Linux operating system for their main sites during August, while two used FreeBSD, one used SmartOS, and another used an F5 device. Conspicuous by its absence is Windows, which has not appeared in the top ten since January, despite being found on more than half of the 1.8 billion websites in our Web Server Survey.

Netcraft measures and makes available the response times of around thirty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.