December 2008 Web Server Survey

In the December 2008 survey, we received responses from 186,727,854 sites. This total has grown by 1.56 million sites since last month.

Apache shows the largest growth this month, gaining a further 2.47 million sites. Just over half of this growth is due to the net hostname growth at ThePlanet.com, which once again includes a large number of .pl domains. Many of these new sites redirect to another site hosted by ThePlanet.com, which appears to offer PornTube videos, but in fact directs visitors towards a site which Google believes to be malware .

Yahoo! Traffic Server shows another large gain since it was uncloaked at Yahoo! last month. This month's survey now finds 1.68 million sites running on YTS, which is used exclusively by Yahoo! as a reverse proxy and connection management server for a number of its services.

nginx shows the 3rd largest growth this month, climbing by more than 10% to reach 3.35 million sites. This server now has nearly 1.8% of the worldwide market share — an impressive feat, given that it is the work of just one man, Igor Sysoev.

Total Sites Across All Domains August 1995 - December 2008

Total Sites Across All Domains, August 1995 - December 2008

Graph of market share for top servers across all domains, August 1995 - December 2008

Top Developers
Developer Nov-08 Share Dec-08 Share Change
Apache 93,207,591 50.34% 95,678,052 51.24% 0.90
Microsoft 63,871,279 34.49% 63,126,940 33.81% -0.69
Google 10,996,941 5.94% 10,455,103 5.60% -0.34
nginx 3,023,369 1.63% 3,354,329 1.80% 0.16
lighttpd 3,030,958 1.64% 3,046,333 1.63% -0.01

Continue reading

Most Reliable Hosting Companies in November 2008

Rank Company site OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 DataPipe FreeBSD  0:00:00  0.005  0.038 0.019 0.039 0.039
2 www.aplus.net FreeBSD  0:00:00  0.005  0.154 0.068 0.227 9.232
3 iWeb.com Linux  0:00:00  0.005  0.005 0.071 0.142 0.142
4 www.westhost.com Linux  0:00:00  0.005  0.005 0.073 0.149 0.312
5 www.he.net Linux  0:00:00  0.005  0.005 0.073 0.153 0.227
6 Server Intellect Windows Server 2003  0:00:00  0.005  0.088 0.103 0.210 0.573
7 www.canadawebhosting.com Windows Server 2003  0:00:00  0.005  0.102 0.111 0.224 0.573
8 webhosting.tiscali.it Linux  0:00:00  0.005  0.024 0.135 0.271 0.554
9 Swishmail FreeBSD  0:00:00  0.010  0.007 0.059 0.120 0.308
10 New York Internet FreeBSD  0:00:00  0.010  0.013 0.062 0.127 0.297

See full table

DataPipe, Aplus.net, iWeb, WestHost, Hurricane Electric, Server Intellect, Canada Web Hosting and Tiscali are the most reliable hosting company sites for November 2008. Unusually, there are eight companies sharing the top spot this month, each showing only 1 failed request throughout November.

Of these eight companies, the top two by average connection time (DataPipe and Aplus.net) both use FreeBSD to run their main websites. In November, DataPipe was named among New Jersey's fastest growing companies.

Linux is used by four of November's most reliable hosting companies. This includes Montreal-based iWeb, which has been providing internet hosting infrastructure for 12 years, and WestHost, which has been providing web hosting for 10 years. Linux is also used by Hurricane Electric and Tiscali, both of which have already featured as the most reliable hosting companies earlier this year.

Two of this month's most reliable hosting companies use Windows Server 2003 to power their sites: Server Intellect is a privately owned company located in Florida and offers dedicated servers, shared hosting and virtual servers. Canada Web Hosting also uses Windows Server 2003 for its main site, but offers managed hosting on both Windows and Linux.

Continue reading

November 2008 Web Server Survey

The November 2008 survey shows worldwide monthly growth of nearly three million websites, with responses now being received from a total of 185,167,897 sites.

Apache once again tops this month's growth, gaining 1.3 million sites to 93 million, but Microsoft-IIS follows closely gaining 1.1 million extra sites to reach 64 million. Google has grown by 509 thousand this month to approach the 11 million mark.

One interesting change this month is the appearance of 221,000 sites hosted by Yahoo! that now identify themselves as running on the Yahoo! Traffic Server proxy. Last month's survey found only 521 sites that claimed to be running on YTS.

Yahoo! is thought to use YTS to provide reverse proxy and connection management in a number of its services, although many of the company's sites were previously configured to omit the Server header in their HTTP responses. Yahoo! sites thought to use YTS include Bix, delicious, Flickr and Yahoo Groups.

Yahoo! Traffic Server is used to serve 12 billion requests per day. It was originally developed by Inktomi Corporation as a proxy cache for web traffic and streaming media. Websense acquired the technology behind Inktomi's proxy server, modifying it for use in their WebBlazer Web Threat Management System. Inktomi was then acquired by Yahoo! in 2002.

Total Sites Across All Domains August 1995 - November 2008

Total Sites Across All Domains, August 1995 - November 2008

Graph of market share for top servers across all domains, August 1995 - November 2008

Top Developers
DeveloperOctober 2008PercentNovember 2008PercentChange
Apache91,888,50850.43%93,207,59150.34%-0.09
Microsoft62,766,92834.44%63,871,27934.49%0.05
Google10,487,6075.76%10,996,9415.94%0.18
lighttpd3,072,4571.69%3,030,9581.64%-0.05

Continue reading

green.ch is the Most Reliable Hosting Company in October 2008

Ranking by Failed Requests and Connection time,
October 1st – 31th 2008

Rank Company site OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 www.green.ch F5 Big-IP    0.000  0.197 0.107 0.270 0.581
2 New York Internet FreeBSD  0:00:00  0.004  0.008 0.055 0.110 0.254
3 Rackspace Linux  0:00:00  0.004  0.018 0.061 0.122 0.122
4 www.reliableservers.com unknown  0:00:00  0.004  0.006 0.065 0.138 0.206
5 www.he.net Linux  0:00:00  0.004  0.005 0.069 0.143 0.213
6 Hosting 4 Less Linux  0:00:00  0.008  0.114 0.115 0.233 0.461
7 www.swishmail.com FreeBSD  0:00:00  0.013  0.005 0.059 0.118 0.304
8 Verio Linux  0:00:00  0.013  0.074 0.073 0.147 0.147
9 www.godaddy.com Windows Server 2003  0:00:00  0.017  0.028 0.054 0.131 0.575
10 www.navisite.com Linux  0:00:00  0.017  0.065 0.059 0.143 0.615

See full table

green.ch is the most reliable hosting company site for October 2008. This is the only site that responded to every request made by Netcraft's performance collectors throughout the month.

With more than 70,000 customers, green.ch is one of the leading Swiss internet service providers for small and medium sized businesses. Originally known as agri.ch, the company was formed from a management buyout of the SME and Private Customer division of Cable & Wireless.

green.ch provides broadband internet access and uses its own data centre in Switzerland to focus on providing website hosting, email, SharePoint and VoIP telephony solutions.

green.ch uses Microsoft IIS 6.0 to serve its main site via an F5 BIG-IP device. The company also uses F5 BIG-IP for over 90% of its customers' websites, making green.ch the largest hoster of sites on F5 BIG-IP in Switzerland.

Linux is used by five of October's top ten hosting companies, while two use FreeBSD and Go Daddy uses Windows Server 2003.

Continue reading

October 2008 Web Server Survey

In the October 2008 survey we received responses from 182,226,259 sites, which reflects growth of 948 thousand since last month.

Apache once again shows the largest growth, gaining 463 thousand sites this month. ThePlanet.com gains 1.3 million sites this month — nearly all of which are running on Apache — but this includes a large number of 'link farm' sites that use .pl domains to propagate search terms using pornographic phrases.

Google shows the next largest growth and boosts its total by 411 thousand sites. Google now runs 10.5 million sites on its own webserver software, which is used to host its own services in addition to user-generated applications and blogs. Some server names include:

  • GFE/1.3, which is used by Google's Blogger service to publish third party blogs under the blogspot.com domain, and spreadsheets and other documents under docs.google.com.
  • GWS-GRFE/0.50, which runs Google Groups.
  • gws. This simple, lowercase name is used by Google's main search site at google.com and Google Image Search.
  • Google Frontend, which is used to run third party applications on Google App Engine (often using the appspot.com domain) and Google Mashups.

Total Sites Across All Domains August 1995 - October 2008

Total Sites Across All Domains, August 1995 - October 2008

Graph of market share for top servers across all domains, August 1995 - October 2008

Top Developers
DeveloperSeptember 2008PercentOctober 2008PercentChange
Apache91,425,29550.43%91,888,50850.43%-0.01
Microsoft62,374,82334.41%62,766,92834.44%0.04
Google10,076,4055.56%10,487,6075.76%0.20
lighttpd3,095,9281.71%3,072,4571.69%-0.02

Continue reading

Ongoing Phishing Attack Exposes Yahoo Accounts

Update 2008-10-28: The attack is no longer ongoing. Yahoo has provided us with the following in a statement:

The team was made aware of this particular Cross-Site Scripting issue yesterday morning (Sunday, Oct. 26) and a fix was deployed within a matter of hours. Yahoo! appreciates Netcraft's assistance in identifying this issue.

As a safety precaution, we recommend users change their passwords, should they still be concerned. Users should always verify via their Sign-in Seal that they are giving their passwords to Yahoo.com.

Our original article follows:

The Netcraft toolbar community has detected a vulnerability on a Yahoo website, which (at the time of writing) is currently being used to steal authentication cookies from Yahoo users — transmitting them to a website under the control of a remote attacker. With these stolen details, the attacker can gain access to his victims' Yahoo accounts, such as Yahoo Mail.

The attack exploits a cross-site scripting vulnerability on Yahoo's HotJobs site at hotjobs.yahoo.com, which currently allows the attacker to inject obfuscated JavaScript into the affected page. The script steals the authentication cookies that are sent for the yahoo.com domain and passes them to a different website in the United States, where the attacker is harvesting stolen authentication details.

When websites use cookies to handle authenticated sessions, it is extremely important to protect the cookie values and ensure they are not seen by other parties. Cross-site scripting vulnerabilities often allow these values to be accessed by an attacker and transmitted to a website under their control, which then allows the attacker to use the same cookie values to hijack their victim's session without needing to log in. This type of attack can be mitigated to some extent by using HttpOnly cookies to prevent scripts gaining access to the cookies — a feature that is now supported by most modern browsers.

Earlier this year, Netcraft blocked a similar flaw on another Yahoo website. The previous attack targeted a cross-site scripting vulnerability on Yahoo's ychat.help.yahoo.com site, which was served securely using a valid SSL certificate, adding further credibility to the attack. The attacker used the vulnerability to inject malign JavaScript into one of the site's webpages. Unlike the current attack, the injected code was sourced from a server in Spain, but also resulted in the victim's cookies being stolen and transmitted to a PHP script on the same server.

pula.js-resized.png
The small cookie-stealing script injected by the attacker.

hotjobs-yahoo-xss.png
A similar technique employed by the current attack.

In both cases, Netcraft found that the Yahoo cookies stolen by the attacker would have allowed him to hijack his victims' browser sessions, letting him gain access to all of their Yahoo Mail emails and any other account which uses cookies for the yahoo.com domain.

Simply visiting the malign URLs on yahoo.com can be enough for a victim to fall prey to the attacker, letting him steal the necessary session cookies to gain access to the victim's email — the victim does not even have to type in their username and password for the attacker to do this. Both attacks send the victim to a blank webpage, leaving them unlikely to realise that their own account has just been compromised.

ychat-resized.png
Both attacks send victims to a innocuous-looking, blank webpage.

The Netcraft Toolbar protects users against both of these attacks, warning that the malformed Yahoo URLs contain cross-site scripting elements, and that the URLs have been classified as known phishing sites.

Netcraft has informed Yahoo of the latest attack, although at the time of writing, the HotJobs vulnerability and the attacker's cookie harvesting script are both still present.