September 2011 Web Server Survey

In the September 2011 survey we received responses from 485,173,671 sites, a growth of nearly 4.8% (just over 22M hostnames) compared to last month.

Apache servers saw the largest growth this month (roughly 14M new hostnames), taking its total to over 315M, yet it continued to lose market share. Apache now serves twice as many hostnames as this time last year.

Nearly a quarter of this growth comes from new sites seen at Softlayer. By the end of the year, Softlayer’s global network will include points of presence (PoPs) and data centers throughout Europe and Asia, with the goal of bringing Softlayer’s network within 40ms of everyone on the planet. Both BurstNet and AmeriNOC also saw growths of over a million hostnames.

Nginx had the biggest increase in market share, with a growth of 0.36 percentage points. This is the largest increase that we have seen for nginx since January. Nginx also continues to increase its share of active sites, gaining ground on Microsoft and remaining as the 2nd most popular open source webserver. Google on the other hand saw a moderate loss in active sites this month, down by nearly 3.8M.

Total Sites Across All Domains
August 1995 - September 2011

Total Sites Across All Domains, August 1995 - September 2011


Market Share for Top Servers Across All Domains
August 1995 - September 2011

Graph of market share for top servers across all domains, August 1995 - September 2011


DeveloperAugust 2011PercentSeptember 2011PercentChange
Apache301,771,51865.18%315,605,33565.05%-0.13
Microsoft73,415,91615.86%76,323,01815.73%-0.13
nginx35,533,4397.67%38,970,6838.03%0.36
Google17,061,0033.68%17,265,3083.56%-0.13
Continue reading

Most Reliable Hosting Company Sites in August 2011

Rank Company site OS Outage hh:mm:ss Failed Req% DNS Connect First byte Total
1 Datapipe FreeBSD 0:00:00 0.007 0.083 0.020 0.042 0.056
2 iWeb Technologies Linux 0:00:00 0.007 0.105 0.072 0.144 0.144
3 Swishmail FreeBSD 0:00:00 0.011 0.083 0.044 0.089 0.231
4 INetU FreeBSD 0:00:00 0.015 0.112 0.042 0.118 0.261
5 New York Internet FreeBSD 0:00:00 0.015 0.108 0.046 0.094 0.274
6 Multacom FreeBSD 0:00:00 0.015 0.114 0.075 0.152 0.428
7 www.cwcs.co.uk Linux 0:00:00 0.019 0.228 0.114 0.225 0.608
8 ServInt Linux 0:00:00 0.022 0.149 0.041 0.089 0.227
9 www.netcetera.co.uk Windows Server 2008 0:00:00 0.022 0.059 0.093 0.189 0.386
10 Iomart plc. Linux 0:00:00 0.022 0.167 0.097 0.206 0.360
See full table

Both of the top two hosting companies had the same number of failed requests this month and they are therefore ranked by average connection time.

After slipping to third last month, the top spot was regained by Datapipe in August. They offer a range of services including managed hosting, compliance, security and cloud computing. Datapipe have recently expanded their UK datacenters.

In second place this month is iWeb Technologies, a Canadian hosting company based in Montreal. iWeb Technologies provide web hosting, dedicated servers, managed hosting and colocation to customers from around the world. They have recently launched a new product range, Smart Servers, which aim to give the benefits of both virtualization in the cloud and dedicated hardware.

Swishmail come in at third place this month, who offer a variety of managed web hosting plans in addition to their core service of enterprise-grade email hosting.

For the first time this year, FreeBSD has the largest share of hosting providers in the top 10 with half of them running FreeBSD servers. Of the other hosting providers in the top 10, 4 run Linux and 1 uses Windows Server 2008.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

Governments hosted 146 new phishing sites in July

Disclosures by groups including LulzSec, Anonymous and Wikileaks have highlighted that poor security is endemic throughout government websites. During July, Netcraft's anti-phishing toolbar blocked 146 new phishing attacks hosted on government websites. Nearly half of these used the gob.pe secondary top-level domain, making the government of Peru the phishiest in the world.

Unusually, more than a third of these government-hosted phishing attacks targeted a single British financial institution, Lloyds Banking Group. Halifax and Lloyds TSB, which are subsidiaries of the group, were each targeted more times than any other organisation, including PayPal.

Phishing sites hosted on government domains (top 10)
(source: Netcraft Phish Feed)

Government
TLD
Country New phishing sites
in July 2011
gob.pe Peru 69
gov.br Brazil 12
go.th Thailand 11
gob.mx Mexico 9
gov.cn China 9
gov.ar Argentina 6
gov.za South Africa 6
gov.pk Pakistan 3
gov.ec Ecuador 3
gov.tr Turkey 3

Some of these government-hosted phishing sites were even used to attack other governments. The website of the UK government, Directgov, was targeted in 14 of these phishing attacks. All of the phishing content used in these attacks was hosted on Peruvian government domains, but has since been removed.

Not all of the phishing sites blocked last month have been taken down. Near the beginning of July, Netcraft blocked an Amazon phishing site on a Polish government TLD, but the fraudulent content is still being served today, more than a month later. This was the only new phishing site to appear on a Polish government website in July, and was unusually served from a compromised FTP server. By supplying valid FTP credentials as part of the URL, a fraudster can make the site viewable in any modern web browser as if it were an ordinary website served over HTTP or HTTPS:

Phishing sites are regularly hosted on compromised servers. By using someone else's web server, an attacker can create a phishing site without having to register any domain names or pay for hosting services. Certain TLDs may also be perceived as being more trustworthy than others, but this is not always a safe assumption.

For example, the .gov TLD is restricted for use by government entities in the United States, so content on these sites is often assumed to be legitimate without much thought. However, www.caldwelltx.gov was presumably compromised at some point, as it was seen serving a Halifax phishing site in July.

Netcraft also found several government websites that had been defaced by "Al3x 0wn5". One Indonesian government site was still defaced at the time of publication, and displayed a mock root prompt on its homepage:

The demonstrably poor security of some government websites may have much wider implications for a country's population. For example, if a remote attacker is able to deface a site or upload arbitrary phishing content, it may also be possible to gain unauthorised access to backend databases or divert any payment details transmitted through the site.

August 2011 Web Server Survey

In the August 2011 survey we received responses from 463,000,317 sites.

This month there has been a large increase in the number of sites found by the survey, which is now approaching half a billion hostnames. There were particularly large increases in the numbers of sites hosted by Softlayer, AmeriNOC, China Telecom, and the social networking communities Tumblr, Skyrock and Hyves.

Amazon also showed substantial growth and now hosts several million sites at its EC2 facility. EC2 (Elastic Compute Cloud) is Amazon's cloud computing platform, allowing website owners to quickly rescale resources to meet demand. It has expanded significantly with a new data center recently opened in Tokyo, and is expected to have a revenue of $750M (Source: UBS via Business Week) this year.

However, even with the large increase in sites, the market share of the different server software has changed only a little. Of the leading web servers, just nginx gained share this month, whilst Apache, Microsoft and Google each showed small losses.


Total Sites Across All Domains
August 1995 - August 2011

Total Sites Across All Domains, August 1995 - August 2011


Market Share for Top Servers Across All Domains
August 1995 - August 2011

Graph of market share for top servers across all domains, August 1995 - August 2011


Developer July 2011 Percent August 2011 Percent Change
Apache 235,326,985 65.86% 301,771,518 65.18% -0.69
Microsoft 60,086,346 16.82% 73,415,916 15.86% -0.96
nginx 23,357,497 6.54% 35,533,439 7.67% 1.14
Google 15,641,574 4.38% 17,061,003 3.68% -0.69
Continue reading

Most Reliable Hosting Company Sites in July 2011

Rank Company site OS Outage hh:mm:ss Failed Req% DNS Connect First byte Total
1 Hosting 4 Less Linux 0:00:00 0.015 0.128 0.064 0.130 0.185
2 www.poundhost.com Linux 0:00:00 0.015 0.286 0.096 0.203 0.440
3 Datapipe FreeBSD 0:00:00 0.019 0.121 0.017 0.037 0.048
4 INetU FreeBSD 0:00:00 0.019 0.064 0.035 0.097 0.223
5 www.logicworks.net Linux 0:00:00 0.022 0.056 0.044 0.106 0.238
6 iWeb Technologies Linux 0:00:00 0.022 0.085 0.059 0.119 0.119
7 www.qubenet.net Linux 0:00:00 0.026 0.128 0.053 0.105 0.105
8 www.micfo.com Linux 0:00:00 0.026 0.135 0.055 0.304 0.462
9 www.netcetera.co.uk Windows Server 2008 0:00:00 0.030 0.051 0.102 0.207 0.416
10 www.westhost.com Linux 0:00:00 0.034 0.051 0.102 0.207 0.416
See full table

Both of the top two hosting companies had the same number of failed requests this month and they are therefore ranked by average connection time.

Top of the table this month is Hosting 4 Less. The company guarantees 99.9% uptime, with its own OC48 Sonet Ring connecting their secure data center to the internet. Hosting 4 Less has been running since 1998. They offer both Linux and Windows hosting as well as e-commerce hosting.

The second most reliable hosting company for July was PoundHost who are based in the UK and offer a range of hosting solutions, including dedicated servers, co-location and virtual hosting.

Third this month was Datapipe, which offers a range of services including managed hosting, compliance, security and cloud computing. Datapipe offers a 100% network uptime guarantee.

Seven sites in the top 10 this month were running Linux, two were running FreeBSD and one was running Windows Server 2008.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

Most Reliable Hosting Company Sites in June 2011

Rank Company site OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 Datapipe FreeBSD 0:00:00 0.004 0.107 0.007 0.015 0.021
2 INetU FreeBSD 0:00:00 0.004 0.169 0.039 0.103 0.248
3 New York Internet FreeBSD 0:00:00 0.004 0.198 0.062 0.124 0.374
4 Hosting 4 Less Linux 0:00:00 0.008 0.136 0.075 0.153 0.337
5 www.logicworks.net Linux 0:00:00 0.012 0.128 0.068 0.152 0.366
6 www.micfo.com Linux 0:00:00 0.019 0.130 0.052 0.291 0.410
7 ReliableServers.com FreeBSD 0:00:00 0.023 0.232 0.069 0.144 0.352
8 aruba.it Windows Server 2003 0:00:00 0.023 0.119 0.095 0.191 0.191
9 Kattare Internet Services Linux 0:00:00 0.023 0.132 0.097 0.194 0.348
10 www.qubenet.net Linux 0:00:00 0.031 0.102 0.046 0.094 0.094

See full table

Each of the top three hosting companies had only one failed request this month and they are therefore ranked by average connection time.

Top of the table this month is Datapipe, which offers a range of services including managed hosting, compliance, security and cloud computing. Datapipe is committed to customer service and offers a 100% network uptime guarantee.

The second most reliable hosting company in June was INetU, which offers dedicated managed hosting services from data centers with resilient networks, backup power and strong physical and network security. InetU also offers compliance and consultancy services to customers around the world.

Third this month was New York Internet. NYI offers colocation, dedicated servers and virtual hosting from data centers in New York. The company focuses on reliability and technical support that is responsive round the clock.

Half of the sites in the top ten in June were running Linux, four were running FreeBSD, and one was running Windows Server 2003.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.