December 2013 Web Server Survey

In the December 2013 survey we received responses from 861,023,217 sites, an increase of 75.7M since last month.

For the third consecutive month Microsoft experienced the largest growth in web server market share; an additional 51M sites boosted its market share by almost 4 percentage points. Apache had the biggest loss in market share, despite seeing an increase of 7M sites. Nginx’s growth this month has led to them regaining some of the market share lost last month, and results in a net gain of 3M sites over the last two months (a loss of 13M in November, followed by a gain this month of 16M).

Considering only active sites, Microsoft’s growth was significantly lower (compared to its gain of 51M sites) with an increase of 478k active sites. Apache saw the largest increase of 622k active sites, and increased its market share by 2 percentage points to 54%. Google meanwhile saw a large loss of 8.6M active sites within its App Engine service.

Microsoft's web server market share could see further gains from its own Windows Azure cloud service — Microsoft has made Azure more cost-effective with price reductions and a new enterprise plan. Microsoft also has plans to expand Azure into Brazil in 2014. Unsurprisingly Microsoft IIS is the most common Web Server seen on Azure, used by 87% of sites. Amazon followed Microsoft's price cuts by introducing price reductions on EC2 M3 Instances. At Amazon, 36% of sites are running on nginx, followed by Apache with 27% and Microsoft IIS with 14%.

ICANN has begun delegating new generic Top-Level Domains (gTLDs) from its new gTLD expansion program, adding them into the Internet's Root Zone. The first four gTLDs (شبكة, онлайн, сайт, 游戏) were delegated on October 23rd and a further 30 new gTLDs have been delegated throughout November. ICANN has a mandatory 30-day sunrise period following delegation where registries "complete a final process built into the new gTLD Program to protect trademark rights holders"; after this has passed the new gTLD can be made available to the general public at the registry's discretion. The شبكة dotShabaka registry was first to start its sunrise period on 31 October, this is to be followed by an (optional) Landrush Period before general availability commences on February 4th. So far the only sites found by Netcraft have been for the NICs (Network Information Center) themselves, such as: nic.游戏, nic.сайт, nic.онлайн, nic.شبكة.

The first gTLDs signed agreements back in July and more are being agreed each month. Recent notable gTLDs that have been agreed include the cities .london, .berlin, .budapest, .wein (Vienna) and .tokyo.





DeveloperNovember 2013PercentDecember 2013PercentChange
Apache348,159,70244.33%355,244,90041.26%-3.08
Microsoft190,451,70224.25%241,777,72328.08%3.83
nginx109,968,56414.00%126,485,20414.69%0.69
Google37,748,7434.81%38,263,5254.44%-0.36
Continue reading

Most Reliable Hosting Company Sites in November 2013

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 Krystal Hosting Linux 0:00:00 0.007 0.137 0.091 0.189 0.189
2 Swishmail FreeBSD 0:00:00 0.010 0.128 0.077 0.151 0.201
3 iWeb Linux 0:00:00 0.014 0.144 0.089 0.172 0.172
4 dinahosting Linux 0:00:00 0.014 0.218 0.097 0.194 0.194
5 Kattare Internet Services Linux 0:00:00 0.014 0.182 0.124 0.246 0.506
6 Datapipe FreeBSD 0:00:00 0.017 0.080 0.019 0.039 0.059
7 ServerStack Linux 0:00:00 0.017 0.088 0.076 0.151 0.151
8 Qube Managed Services Linux 0:00:00 0.024 0.117 0.056 0.112 0.112
9 New York Internet FreeBSD 0:00:00 0.024 0.140 0.074 0.149 0.570
10 Bigstep Linux 0:00:00 0.028 0.284 0.073 0.155 0.230

See full table

With only two failed requests, Krystal Hosting was the most reliable hosting company in November 2013. Krystal, which was founded in 2002 by Simon Blackler when he was just 17 years old, recently launched a VPS hosting service, Kloud. The UK-based hosting company has placed great emphasis on security — it is currently based in an underground ex-military bunker outside London which is protected by former members of the military and police services.

Swishmail was second most reliable hosting company in November, with only three failed requests. Swishmail primarily provide a business email service on a FreeBSD-based platform. Earlier this year it became a bronze sponsor of the FreeBSD project.

Close behind Swishmail came iWeb (third most reliable), dinahosting (fourth), and Kattare Internet Services (fifth). As all three hosting companies had the same number of failed requests in November the tie was broken by examining the average connection time.

Datapipe ranked 6th with five failed requests. It is regularly one of the fastest and most reliable hosting companies we monitor. Datapipe has more than seven years of continuous uptime on www.datapipe.net.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

Incentives for Phishing Site Reporters

As of the 1st November 2013, the Netcraft Anti-Phishing community has helped to block over 6.9 million phishing attacks worldwide. We incentivise phishing reports from the community, and have now added a Netcraft USB Flash Drive to our list of incentives:

Prize When
Netcraft USB Flash Drive after 100 validated phishing reports
Netcraft Mug after 250
Netcraft Polo Shirt after 500
Targus Laptop Backpack after 1,000
iPad after 5,000

On reaching 5,000 validated reports you become eligible for a monthly competition to incentivise large reporters.

To report phishing sites to us, please use the form at http://toolbar.netcraft.com/report_url, or forward any phishing URLs or emails you receive to scam@netcraft.com.

The Netcraft Extension, which is available for Firefox, Google Chrome™ and Opera, serves as a giant neighbourhood watch scheme for the Internet. Members who encounter a phishing site can act to defend the larger community of users against the attack. Once the first recipients of a phishing mail have reported the attack URL, it is blocked for community members as they subsequently access the URL. Widely disseminated attacks simply mean that the phishing attack will be reported and blocked sooner.

Anti-Phishing Chrome Extension Netcraft Toolbar for Firefox Netcraft Toolbar for Opera

Most Reliable Hosting Company Sites in October 2013

Rank Performance Graph OS Outage hh:mm:ss Failed Req% DNS Connect First byte Total
1 iWeb Linux 0:00:00 0.006 0.153 0.090 0.178 0.178
2 Hosting 4 Less Linux 0:00:00 0.006 0.181 0.120 0.239 0.610
3 Qube Managed Services Linux 0:00:00 0.009 0.128 0.064 0.129 0.129
4 www.uk2.net Linux 0:00:00 0.009 0.172 0.090 0.184 0.320
5 Swishmail FreeBSD 0:00:00 0.012 0.145 0.090 0.177 0.249
6 Virtual Internet Linux 0:00:00 0.012 0.154 0.093 0.269 0.476
7 XILO Communications Ltd. Linux 0:00:00 0.015 0.238 0.093 0.193 0.348
8 Pair Networks FreeBSD 0:00:00 0.018 0.240 0.090 0.183 0.579
9 Data Centers Canada Linux 0:00:00 0.021 0.077 0.092 0.191 0.409
10 Anexia Linux 0:00:00 0.021 0.250 0.122 0.483 0.844

See full table

Montreal based iWeb was the most reliable hosting company in October 2013 with only two failed requests. Two days ago the US IT-infrastructure company Internap announced plans to buy iWeb in a deal worth $145m. Both companies provide IaaS and are corporate sponsors of the OpenStack foundation. Internap has data centres around the world which will allow iWeb to expand on its four data centres, all based in Canada.

Hosting 4 Less was the second most reliable hosting company; it also only had two failed requests but its slower average connection time was used as the decider. 2nd place is its highest ranking this year, having also been ranked 2nd in September 2012. Qube Managed Services, who regularly feature in the top 10, ranked third this month having been the most reliable hosting company in September.

UK2 celebrated their 15th year in business this month and are again the fourth most reliable hosting company. UK2 cater to a wide range of customers, from individuals and small businesses wanting to create an online presence with their Website Builder tool to larger websites using its Xen based Virtual Servers and dedicated hosting.

Linux is used by eight of the top ten with the remaining two, Swishmail and Pair Networks using FreeBSD. Netcetera was the most reliable Windows hosting company again this month - although just outside the top 10 in 12th place. 8 of the top 10 are using the Apache web server including the top four companies.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

November 2013 Web Server Survey

In the November 2013 survey we received responses from 785,293,473 sites, reflecting net growth of more than 18 million sites since last month.

Microsoft experienced the largest gains this month, with an additional 13.2 million sites taking its market share up by 1.15 percentage points. In contrast to recent trends, nginx's market share fell by more than 2 percentage points to 14.0% after it lost 13.1 million sites. Despite the absolute gain at Microsoft being almost the same as the number of lost nginx sites, this is merely a coincidence — only 1.2 million nginx sites actually switched to using IIS this month (0.8 million of which opted for IIS 6.0), whereas 1.4 million switched to Apache. 23 million nginx sites that were present in last month's survey have since expired, including a large number of .ru domains previously hosted by Hetzner Online.

nginx enjoyed better fortunes amongst the million busiest sites, where it extended its market share by 0.22 percentage points to 15.31%, placing it 2.46 points ahead of Microsoft. File sharing site rapidshare.com has recently started displaying an nginx Server header; previously the site did not reveal which server software it was running, and New Zealand Post has switched from Apache to nginx.

Google's market share went up to 4.81% this month (+0.36) after gaining 3.6 million sites, and could be set to grow even further now that the Google App Engine PHP runtime is widely available. In January, 244 million sites were using PHP (mostly on Apache), highlighting the strong demand. Once a PHP application has been deployed on App Engine, it can make direct use of Google Cloud Storage through existing PHP filesystem functions such as fopen and file_put_contents.

Google is specifically targeting WordPress users to migrate to App Engine — Google have produced an App Engine plugin for WordPress to allow it to interact with App Engine-specific services such as mail and storage. Google cites motherboard.vice.com as one of the early adopters of the App Engine PHP runtime, having moved from nginx running on Amazon EC2. Vice's in-house content management system is powered by the Yii PHP framework, and was moved fully over to App Engine during a limited preview period.





DeveloperOctober 2013PercentNovember 2013PercentChange
Apache344,408,38744.89%348,159,70244.33%-0.55
Microsoft177,216,29623.10%190,451,70224.25%1.15
nginx123,114,80016.05%109,968,56414.00%-2.04
Google34,127,4824.45%37,748,7434.81%0.36
Continue reading

PHP.net blocked by Google: False positive or not?

Rasmus Lerdorf – the creator of PHP – is currently trying to get Google to stop blocking the whole php.net website after it was suspected of containing malware. In a tweet earlier this morning, Rasmus posted a screenshot and suggested that the block was caused by a false positive:

Google's Webmaster Tools flag the inclusion of the script at http://static.php.net/www.php.net/userprefs.js as suspicious, although this file currently appears benign. However, Google's Safe Browsing diagnostics for php.net do suggest that malware has been present on the site in the last 90 days:

"Of the 1513 pages we tested on the site over the past 90 days, 4 page(s) resulted in malicious software being downloaded and installed without user consent."

The block was added to add chunk 127721 in the Google Safe Browsing goog-malware-shavar list. At the time of writing, php.net is still blocked in Firefox and Chrome, both of which make use of Google's blocklist. Visiting php.net from a Google search results page or the bitly URL shortener causes an interstitial warning page to be displayed.

A seemingly benign, yet obfuscated, JavaScript file called functions.js was removed from the PHP website repository this morning. The developer behind this change speculated that the file "Could be the reason why Google is blocking us today.."

However, a short moment ago, a Hacker News user posted some obfuscated JavaScript that was found appended to a possibly cached version of the userprefs.js script, suggesting that the PHP.net website may have been compromised recently. The obfuscated JavaScript inserts an iframe into the webpage, which loads content from an external site known for distributing malware. Google Chrome blocks the inclusion of any content from known malware domains, although the injected content in this case no longer appears to be accessible.


Using Firebug to display the injection point of the iframe (iframe has been moved to a visible location)

Update [Monday 28th October]: The administrators of PHP.net have since confirmed that two web servers were compromised and at least one was serving malware. The affected servers have been taken offline and the SSL certificate in use has been revoked by Comodo. The PHP source packages and code repository were reportedly not compromised.