July 2016 Web Server Survey

In the July 2016 survey we received responses from 1,073,777,722 sites and 6,058,513 web-facing computers. This reflects an increase of 28 million sites and 107,000 computers.

Microsoft and Apache continue to fluctuate between 1st and 2nd places for total number of websites, with Microsoft retaking the lead again this month after an increase of 36 million sites, and the loss of 20 million Apache sites. However, it is only in the hostnames metric that Microsoft leads, with Apache coming out on top by a considerable margin in all other areas - active sites, domains, IP addresses and web-facing computers.

Looking at the number of active sites, a more stable metric created by Netcraft to ignore automatically generated bulk content, Apache leads with a 46.4% market share. nginx is the second most popular vendor, with a 21.8% market share, and is the only major vendor to consistently gain share in recent months. A similar percentage of both Apache’s and nginx’s sites are deemed to be active, 23.7% of Apache sites and 22.1% for nginx. In comparison, only 4.5% of sites using Microsoft server software are considered active, leaving Microsoft in 3rd place by this active sites metric with a 9.8% market share.

Apache, Microsoft and nginx all gained web-facing computers this month; however, nginx once again saw the only increase in market share. nginx gained 46,000 computers, a growth of 4.8% on last month, and now stands just shy of 1 million web-facing computers. Apache and Microsoft gained 31,000 and 11,000 computers.

nginx also continues to gain market share among the top million busiest websites, where it is now used by 27.9% of sites. Apache, while still holding a dominant position, continues to slowly lose market share, falling 0.55 percentage points to 43.2%.

Total number of websites

Web server market share

DeveloperJune 2016PercentJuly 2016PercentChange
Microsoft342,605,66632.77%378,655,75935.26%2.50
Apache360,458,01834.48%340,551,07431.72%-2.76
nginx169,316,54716.19%170,896,71615.92%-0.28
Google21,662,6732.07%22,552,9012.10%0.03
Continue reading

Most Reliable Hosting Company Sites in June 2016

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 Pair Networks FreeBSD 0:00:00 0.004 0.231 0.067 0.137 0.137
2 Netcetera Linux 0:00:00 0.004 0.087 0.080 0.163 0.163
3 Lightcrest unknown 0:00:00 0.009 0.252 0.005 0.017 0.034
4 GoDaddy.com Inc Linux 0:00:00 0.009 0.202 0.012 0.031 0.031
5 Qube Managed Services Linux 0:00:00 0.009 0.131 0.057 0.116 0.116
6 XILO Communications Ltd. Linux 0:00:00 0.009 0.205 0.063 0.126 0.126
7 Kattare Internet Services Citrix Netscaler 0:00:00 0.009 0.498 0.117 0.230 0.231
8 New York Internet FreeBSD 0:00:00 0.013 0.552 0.037 0.077 0.297
9 Swishmail FreeBSD 0:00:00 0.013 0.137 0.059 0.118 0.159
10 EveryCity SmartOS 0:00:00 0.013 0.106 0.060 0.121 0.121

See full table

Pair Networks had the most reliable hosting company site in June, with only one failed request. Pair Networks was founded in 1995, putting it among the oldest web hosting companies. Its custom-built data centre is based in Pittsburgh, where it provides a range of premium hosting services.

In second place this month, Netcetera also had just one failed request but with a slightly longer average connection time. Netcetera provides carbon-neutral shared, dedicated, and cloud hosting from its Dataport data centre on the Isle of Man, and recently partnered with vCita LiveSite to offer document sharing, appointment scheduling and online payments.

With two failed requests, Lightcrest took third place. Lightcrest operates its cloud services using its own Kahu Compute Fabric infrastructure, without outsourcing any components to third-party cloud providers. Since Netcraft began monitoring its site in September 2014, Lightcrest has maintained a 99.97% uptime record.

Four of June's top ten hosting company sites ran on Linux operating systems, with a further three using FreeBSD. Also appearing were Citrix Netscaler and SmartOS, powering one site each. Windows operating systems have now been absent from the top ten for a full year, last appearing in June 2015.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

June 2016 Web Server Survey

In the June 2016 survey we received responses from 1,045,534,808 sites and 5,951,685 web-facing computers. This reflects an increase of 12 million sites, along with a modest gain of 4,700 computers.

Apache regained the lead from Microsoft this month, with a large increase of 60 million sites taking its total up to 360 million, while Microsoft lost 24 million. Microsoft enjoyed a brief foray at the top in April and May, thanks to a proliferation of link farming sites, but now stands 18 million sites behind Apache.

Apache's net growth included only 14 million new websites – the remainder consisted of existing websites that switched to Apache after previously using other web server software. Most notably, 52 million sites switched from Tengine to Apache, while 12 million switched from Microsoft. The number of websites using Tengine fell by more than 60%, largely as a result of the migration to Apache. Most of the sites involved in this switch were hosted by OVH in Canada, and not only changed server vendor, but also moved to a different hosting company—Data Foundry—in the United States.

Although the number of Tengine websites fell to 29 million, the number of active sites using Tengine actually increased slightly to 1.8 million. In a similar vein, the number of active sites running Apache fell by 630,000, even though the total number of sites grew by more than 60 million.

Out of the largest vendors, Microsoft has the lowest proportion of active sites, with only 4.8% of its 340 million sites being active, while Tengine's proportion has crept up to 6.3%. Both are significantly lower than Apache's proportion of 22.3% and nginx's 21.6%.

Newer versions of web server software generally attract a much higher proportion of active sites. For example, 56% of sites running on Microsoft IIS 10 (which will be included with Windows Server 2016, but is already available for Windows 10) are classified as active, while 23% of sites running IIS 8.5 are active, along with 14% of IIS 8.0 sites. This proportion dwindles to only 2.8% by the time we consider IIS 6.0, which remains a very popular choice of server in China despite no longer being supported by Microsoft.

nginx also demonstrates this trend, with the largest number of its active sites running on the latest 1.10.x stable branch. More than half of all sites using this version are active.

Across all versions, nginx continues to muscle its way into the market with confidence. This month it gained almost a million active sites, along with an additional 31,000 web-facing computers, giving it the largest growth in these important metrics. Conversely, Apache lost 26,000 computers, while Microsoft lost 4,500.

nginx has also continued to grow its presence amongst the top million websites, where it is now used by 27.6% of sites. Apache continues to lead with a 43.7% share of this market, although its share has generally been on the decline since 2011. If current trends continue, nginx could possibly take the lead from Apache within the next couple of years.

Total number of websites

Web server market share

DeveloperMay 2016PercentJune 2016PercentChange
Apache300,447,47029.06%360,458,01834.48%5.41
Microsoft366,964,00935.50%342,605,66632.77%-2.73
nginx163,902,97115.85%169,316,54716.19%0.34
Google21,567,2522.09%21,662,6732.07%-0.01
Continue reading

Most Reliable Hosting Company Sites in May 2016

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 Datapipe Linux 0:00:00 0.004 0.142 0.012 0.024 0.030
2 XILO Communications Ltd. Linux 0:00:00 0.008 0.206 0.064 0.127 0.127
3 GoDaddy.com Inc Linux 0:00:00 0.013 0.226 0.016 0.037 0.037
4 Anexia Linux 0:00:00 0.013 0.180 0.088 0.186 0.186
5 One.com Linux 0:00:00 0.017 0.201 0.037 0.108 0.108
6 dinahosting Linux 0:00:00 0.017 0.230 0.083 0.165 0.165
7 ServerStack Linux 0:00:00 0.025 0.112 0.064 0.129 0.129
8 Aspserveur Linux 0:00:00 0.025 0.301 0.077 0.327 0.484
9 LeaseWeb Linux 0:00:00 0.029 0.303 0.028 0.055 0.055
10 Webair Internet Development Linux 0:00:00 0.029 0.156 0.054 0.108 0.109

See full table

May is the second month in a row that Datapipe has had the most reliable hosting company site. Its own site once again has lived up to its 100% Network Uptime Guarantee, as has been the case during over 10 years of monitoring by Netcraft. Datapipe has 25 data centres globally including locations in key technology and financial hubs: New York, Silicon Valley, London, and Hong Kong.

In second place in May was XILO Communications Ltd with just two failed requests. XILO’s website has maintained 100% uptime over the past two years, and 99.990% since October 2011. XILO offers services from shared hosting to dedicated servers and operate from the UK.

GoDaddy reached third place in May, marking its twelfth consecutive appearance in the top 10. Since May 2015, GoDaddy has had excellent connection times ranging from 5-16ms. GoDaddy recently enhanced its offering to small businesses by acquiring FreedomVoice, a cloud VOIP provider, with the aim of accelerating the delivery of communications services.

During May, Anexia ranked fourth in our top 10 and had just three failed requests. It was narrowly beaten by GoDaddy who had the same number of failed requests but a faster connection time. Anexia also placed impressively in the Inc. 5000 Europe, ranking as the 517th fastest growing company in Europe.

For the second time in 2016, all of the top 10 hosting company websites were powered by Linux. The last time a site using Windows appeared in the top 10 was June 2015

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

May 2016 Web Server Survey

In the May 2016 survey we received responses from 1,033,790,346 sites and 5,946,961 web-facing computers. This reflects a gain of 147,000 computers, coupled with a loss of 49 million sites.

While last month's survey recorded the largest number of sites ever, many of the Chinese sites running Microsoft IIS that appeared last month have since disappeared. Combined with other departures, Microsoft suffered a net loss of 75 million sites this month, which has played a major part in its market share falling by more than 5 percentage points to less than 36%. Nevertheless, it is still the most common server vendor by number of sites, with a total of nearly 370 million hosted on IIS servers.

Despite Microsoft's loss of 75 million sites, the number of active sites using IIS actually grew by 450,000, which is indicative of the low quality of the sites it lost. Most of the lost sites were engaged in link farming activity, with large numbers of these sites being served from relatively few computers. The loss of these sites therefore had little impact on the number of web-facing computers using Microsoft IIS, which grew by 14,000.

Microsoft's closest competitor, Apache, gained 8.4 million sites, with its increased market share of 29.1% putting it within 6.4 percentage points of Microsoft's leading share.

Although it has yet to reach the same level as Microsoft and Apache, nginx made the largest gains, growing by 21 million sites and increasing its market share by 2.6 points to 15.9%.

nginx also showed the strongest growth in the survey's other metrics: it gained nearly 7.5 million active sites, 74,100 web-facing computers, and increased its presence within the top million sites by 16,000. The most significant of these gains was nginx's active site count increasing by a whopping 27%, largely as a result of Tumblr sites now exhibiting the Server: nginx header (in previous months, most Tumblr sites did not reveal which server software they were using).

While Microsoft has shaken off many of its low-quality sites, Alibaba's nginx fork, Tengine, gained around 10 million. Most of the new sites served by Tengine this month make use of domains under the .science gTLD, which has proved popular with many Chinese link farms and webspam sites – most likely due to the sub-dollar registration costs. Tengine suffered a small net loss in active sites this month, which corroborates the low quality of the 10 million new sites.

Only 2.4% of the sites served by Tengine now qualify as active sites, which highlights just how many of them are used for displaying automatically generated content. Microsoft is still also fairly popular with link farm operators (particularly in China), with only 4.6% of its sites showing active content. In contrast, more than 26% of Apache sites, and nearly 22% of nginx sites feature active content.

Total number of websites

Web server market share

DeveloperApril 2016PercentMay 2016PercentChange
Microsoft441,470,89440.75%366,964,00935.50%-5.26
Apache292,043,54826.96%300,447,47029.06%2.10
nginx143,349,43913.23%163,902,97115.85%2.62
Google20,597,6051.90%21,567,2522.09%0.18
Continue reading

Bangladesh government exporting live phish

Bangladesh is one of the world's largest producers of fish; but lately, its government has also become an inadvertent exporter of phish.

Over the past week, several phishing sites have popped up on Bangladeshi government websites, under the .gov.bd second-level domain. These fraudulent sites have been used in phishing attacks against customers of Wells Fargo bank, Google, AOL, and other email providers.

One of the phishing sites currently using a .gov.bd domain is hosted on a website belonging to the Bandarban Technical Training Center in Bangladesh. This site imitates Google Docs in an attempt to steal victims' email credentials, whichever mail providers they use.

One of the phishing sites currently using a .gov.bd domain is hosted on a website belonging to the Bandarban Technical Training Center in Bangladesh. The fraudulent content imitates Google Docs in an attempt to steal victims' email credentials, whichever mail providers they use.

Domain name registrations under .gov.bd are restricted to government-related entities in Bangladesh, although it is unlikely that the government is directly responsible for these attacks. As with most phishing sites, the fraudulent content has probably been placed on these government sites by remote hackers; nonetheless, this would make the Bangladesh government at least responsible for poor security.

The vast majority of websites under .gov.bd are hosted within Bangladesh, but the apparently-compromised server involved in these attacks is one of a few that are hosted in the United Kingdom, on a static IP address used by the hosting company Nibs Solutions. No Bangladeshi servers are currently serving phishing sites from .gov.bd domains.

After more than a week since this spate of phishing attacks started appearing on UK-hosted .gov.bd sites, none of the fraudulent content has been removed. The presence of multiple live phishing sites on the affected server, and the fact that the previous compromises have not yet been cleaned up, suggests that whatever security vulnerabilities might have affected the server are yet to be resolved.

Detected just over a week ago, the oldest phishing site in this spate of attacks targets Wells Fargo customers and remains accessible today on the Jessore Technical Training Center website at jessorettc.gov.bd. This training center was established by the Government of the People's Republic of Bangladesh in 2004, hence its eligibility to use the .gov.bd domain.

Detected just over a week ago, the oldest phishing site in this spate of attacks targets Wells Fargo customers and remains accessible today on the Jessore Technical Training Center website at jessorettc.gov.bd. This training center was established by the Government of the People's Republic of Bangladesh in 2004, hence its eligibility to use the .gov.bd domain.

Bangladesh has a relatively small presence on the web, with just over 30,000 websites making use of the entire .bd country code top-level domain. However, the ratio of phishing incidents to sites is quite high at roughly 1 in 100.

Users of the Netcraft anti-phishing extension are already protected from these attacks, including the examples shown above, even though the fraudulent content has not yet been removed by the sites' administrators.