May 2013 Web Server Survey

In the May 2013 survey we received responses from 672,837,096 sites, which is 23.8M more than last month.

Apache had the largest growth this month, gaining 28.3M websites and increasing its market share by 2.41 percentage points to 53.4%. The majority of this growth was attributable to Apache Traffic Server (ATS), which gained 28M websites and increased its market share from 0.03% to 4.2%. Nearly all of the Apache Traffic Server growth occurred at Go Daddy — 75% of websites hosted by Go Daddy now use ATS and Go Daddy now hosts 99% of all sites using this server software.

Originally created as a commercial product by Inktomi in 1997, Apache Traffic Server is an extensible multi-threaded event-driven caching proxy server which is claimed to scale well on modern multi-core systems. Yahoo! acquired Inktomi in 2005, and in November 2009, the project was donated to the Apache Software Foundation.

The vast majority of the ATS served websites at Go Daddy were previously served by Microsoft IIS, resulting in the rather noticeable loss of 3.26 percentage points of market share. Microsoft IIS's market share is now 16.7%. Despite the loss at Go Daddy it gained more new sites than any competitor this month, with 43% of all new websites being served on Microsoft IIS, while accounting for only 30% of expired websites (this includes inactive blogs, as well as sites which no longer exist).

nginx reached a new milestone this month: it is now used by more than 100M websites, and within the Million Busiest Websites has overtaken Microsoft IIS to take second place with a market share of 13.5%. Overall, nginx's market share now stands at 15.5%, just 1.2 percentage points behind Microsoft, helped by a growth of 8.3M sites this month.

The latest stable version, nginx 1.4.0, was released last week, integrating OCSP stapling and experimental SPDY draft 2 support. nginx is used extensively by the blog hosting service, whose owners – Automattic – sponsored development of the ngx_http_spdy_module. Development of OCSP stapling support was sponsored by Comodo, DigiCert, and GlobalSign.

Developer April 2013 Percent May 2013 Percent Change
Continue reading

Most Reliable Hosting Company Sites in April 2013

Rank Performance Graph OS Outage hh:mm:ss Failed Req% DNS Connect First byte Total
1 Swishmail FreeBSD 0:00:00 0.000 0.106 0.062 0.124 0.267
2 INetU Windows Server 2008 0:00:00 0.000 0.125 0.073 0.236 0.454
3 iWeb Linux 0:00:00 0.003 0.127 0.071 0.142 0.142
4 Server Intellect Windows Server 2008 0:00:00 0.003 0.074 0.092 0.185 0.464
5 Midphase Linux 0:00:00 0.003 0.215 0.109 0.222 0.338
6 Qube Managed Services Linux 0:00:00 0.006 0.100 0.046 0.093 0.093
7 Bigstep Linux 0:00:00 0.006 0.266 0.071 0.143 0.143
8 Hyve Managed Hosting Linux 0:00:00 0.006 0.252 0.074 0.145 0.151
9 Datapipe FreeBSD 0:00:00 0.009 0.068 0.016 0.032 0.049
10 Pair Networks FreeBSD 0:00:00 0.016 0.231 0.077 0.157 0.486
See full table

Swishmail had the most reliable hosting company site in April 2013, with no failed requests. Swishmail has a presence in three New York data centres which proved to be resilient when Swishmail stayed online in October whilst being hit by Hurricane Sandy, despite New York being in the centre of much of the damage. Swishmail offers a variety of managed web hosting plans in addition to its core service of enterprise-grade email hosting. Swishmail has been monitored by Netcraft since April 2007.

In second place is INetU which also had no failed requests, but it missed the top spot by just 11ms due to using the average connect time as the tie-breaker. INetU offers dedicated managed hosting services and cloud hosting services from ten data centres in the US and Europe including a new data centre in Seattle. Netcraft has been monitoring INetU since June 2003.

iWeb is in third place again following last month's success, it narrowly missed second place by having a single failed request. iWeb is based in Montréal where it has four data centres.

Newcomers Bigstep and Midphase have made their debut top 10 entries, after being monitored for one month and six months respectively. Hyve placed 8th this month, its third appearance since Netcraft began monitoring it in November having maintained 100% uptime over 5 months.

Swishmail, April's most reliable hosting company, runs its site on FreeBSD. Two other sites in this month's top ten are running FreeBSD – Datapipe, which was top last month and has an impressive 100% uptime over 7 years, and Pair Networks. Both INetU in second place, and Server Intellect in fourth place, are running Windows Server 2008. The remaining five – including iWeb in third place – use Linux.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

OCSP Server Performance in March 2013

Rank Company site OS Outage
DNS Connect First
1 Linux 0:00:00 0.003 0.076 0.024 0.043 0.043
2 Citrix Netscaler 0:00:00 0.006 0.051 0.081 0.162 0.162
3 Citrix Netscaler 0:00:00 0.006 0.041 0.083 0.164 0.164
4 Linux 0:00:00 0.015 0.161 0.025 0.044 0.044
5 Linux 0:00:00 0.018 0.068 0.011 0.056 0.056
6 Citrix Netscaler 0:00:00 0.018 0.228 0.082 0.163 0.163
7 Citrix Netscaler 0:00:00 0.018 0.050 0.099 0.200 0.201
8 Citrix Netscaler 0:00:00 0.024 0.261 0.082 0.162 0.162
9 Linux 0:00:00 0.027 0.049 0.011 0.057 0.057
10 Linux 0:00:00 0.027 0.199 0.090 0.197 0.197

See full table

The Online Certificate Status Protocol (OCSP) is an alternative method to Certificate Revocation Lists (CRLs) for obtaining the revocation status of an individual SSL certificate. Fast and reliable OCSP responders are essential for both Certificate Authorities (CAs) and their customers — a slow OCSP response will introduce an additional delay before many browsers can start sending and receiving encrypted traffic over an HTTPS connection.

Starfield Technologies, a Go Daddy brand, had the most reliable OCSP responder last month with only a single failed request and an average connection time of 24ms. Starfield Technologies was founded in 2003 as the technology research branch of Go Daddy. Go Daddy customers have the option to choose which issuing organization to use when buying an SSL certificate. Although both Go Daddy and Starfield appear to share the same OCSP responder infrastructure, had five failed requests, however this was still fewer than StartCom, Symantec, and Trend Micro. Both Go Daddy and Starfield issue certificates in all three certificate assurance categories: Domain Validation (DV), Organisation Validation (OV), and Extended Validation (EV). Starfield is most prominent in the EV sector — more than 15% of all EV certificates issued within the group are issued by Starfield — but it remains only a small part of Go Daddy's SSL certificate business: Starfield accounts for just 10% of certificates issued.

StartCom had the shortest average connect time (11ms) of all monitored CAs last month after having moved its OCSP infrastructure at the end of February. StartCom, as well as Entrust, now delivers its OCSP responses via the Akamai CDN (Content Delivery Network), reducing the OCSP connection overhead to a minimum by serving content from as topologically close as possible to the client. GlobalSign is a CloudFlare evangelist, using CloudFlare's CDN platform for its OCSP and CRL infrastructure as well as their own corporate website.

Many of the monitored OCSP responders are served by Citrix Netscaler devices. Citrix Netscaler is a hardware appliance that provides, amongst other features, load balancing and firewall functions. The use of such load balancing technology is no surprise — a single certificate on a popular site that does not use OCSP stapling could generate a significant number of OCSP requests, causing a CA's responder to experience high volumes of traffic.

In many circumstances each connection to an HTTPS site could trigger multiple OCSP requests: a request for the server's certificate and one for each intermediate certificate. OCSP responses are typically valid for a week, so some caching is possible. Caching can reduce both the burden on OCSP responders and increase the perceived performance of HTTPS websites to users, but is limited to repeat visits. OCSP Stapling is designed to improve performance by allowing the web site's server to “staple” the OCSP response to the TLS handshake, removing the need for the client to connect to the CA's OCSP responder.

Netcraft measures and makes available the OCSP and CRL end point response times of all the major Certificate Authorities (CAs). The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

UGG blog used to fleece HSBC customers

Hot on the heels of recent WordPress attacks, Netcraft has found a phishing attack which uses a script hosted on the official UGG blog at UGG — famous for its sheepskin boots — hosts its WordPress blog with Media Temple but its blog also contains a malicious PHP script which fleeces HSBC customers out of their bank account details. It is difficult to tell whether this attack is connected with the recent increase in brute-force password guessing attacks on WordPress or whether the location of the malicious script is unconnected.

The attack uses a phishing email with an attached HTML document designed to look like a genuine HSBC website. The HTML attachment contains a form which asks the victim for his date of birth, security number, account number, sort code and full name. The entered details are submitted to the server hosting the UGG blog, where the details are harvested by a PHP script hidden in the blog's stylesheet directory; the victim is then redirected to the real HSBC website as if nothing untoward were afoot.

The phishing form is submitted to the script hidden on UGG's blog.

WordPress is by far the most popular blogging platform and content management system on the internet: Netcraft's April 2013 Publishing Applications survey found more than 25 million WordPress sites. Given its popularity, it is not surprising that is often targeted by fraudsters. The predictable location of the administrative interface and the widespread use of the default "admin" username lends itself to simple brute-force password guessing attacks as have been reported recently.

Certificate revocation and the performance of OCSP

Certificate revocation is a critical aspect of maintaining the security of the third-party Certificate Authority (CA) infrastructure which underpins secure communication on the internet using SSL/TLS. A certificate may be worth revoking when it has had its private key compromised, the owner of the certificate no longer controls the domain for which it was issued, or the certificate was mistakenly signed. Without the ability to revoke certificates, a CA has no direct means of marking a certificate as untrusted before the expiry of the certificate, which could be several years away. In particularly urgent cases a browser vendor may have the ability to block certain individual certificates, trusted roots, or intermediate certificates, but this is rarely performed and is not suitable for lower-risk issues where revocation is necessary but not urgent.

There are two main technologies for browsers to check the revocation status of a particular certificate: using the Online Certificate Status Protocol (OCSP) or looking up the certificate in a Certificate Revocation List (CRL). OCSP provides real-time revocation information about an individual certificate from an issuing CA, unlike CRLs which provide a list of revoked certificates and may be received by clients less frequently.

The graph below shows a comparison of the time taken for the TLS handshake, both with and without OCSP checking enabled. The data was collected using packet traces taken while using Firefox 20 on Linux from an IP address in the UK. Measurements were taken three times (each time with a fresh cache) after discarding an initial request.

The relationship between whether OCSP checking is enabled and the time taken to complete the TLS handshake is not straightforward. In order for the browser to display the "green bar" to distinguish an Extended Validation (EV) certificate, OCSP requests must be made for every certificate in the chain whereas in many browsers, if an OCSP request is made at all, intermediate certificates are not checked. The increased time taken for the TLS handshake when using an EV certificate can be attributed to Firefox's sequential OCSP checking behaviour. However, where an OCSP check can be performed within the round-trip time to the server — for example, if the OCSP responder is served via a content delivery network or CDN — the check does not dramatically affect the time taken for the TLS handshake. When both the web server and the OCSP responder are topologically close to the client, as is the case with, the short round-trip time to the server isn't sufficient to mask the the time taken to receive OCSP responses for both the web site's certificate and the intermediate certificate presented. The slight difference between Paypal and GlobalSign's performance can at least partially be attributed to the additional OCSP request made for GlobalSign: GlobalSign's certificate chain requires three OCSP requests whereas Paypal's requires just two.

Reliability of RapidSSL's OCSP responder — December 2012

Netcraft has extracted around 40 OCSP responder URLs from certificates seen in the Netcraft SSL server survey, and has been monitoring them since late November 2012. The performance and reliability of the services varies significantly: Symantec's VeriSign OCSP responder has had consistently solid reliability, only a handful of connections failed over a 4 month period; whereas, in the same period more than 6% of requests to one of StartCom's responders failed. The reliability and performance of StartCom's OCSP responders have improved significantly since the end of February 2013 when it switched to using Akamai. Geotrust, another Symantec brand, did not have as strong a performance as either Thawte or VeriSign — all three of GeoTrust’s OCSP servers were down for between 48 and 104 minutes in a single event. Performance and reliability is measured from 11 points spread around Europe and North America: outages require at least one failed response from all measurement nodes within the 15-minute measurement interval.

Shift in reliability and performance for StartCom — late February 2013

For those browsers performing a synchronous OCSP request during the TLS handshake, the performance of the OCSP responder is often crucial. Any delay in responding to the request may noticeably slow down the handshake. For example, comparing GlobalSign's CloudFlare-accelerated OCSP responder with Entrust's, you find that GlobalSign's responder is significantly faster than Entrust's which uses Akamai's CDN. However, despite GlobalSign's performance advantage, its reliability has been affected by a number of CloudFlare outages — since Netcraft began monitoring OCSP, GlobalSign's responders have had at least 45 minutes of downtime whereas Entrust has had none.

GlobalSign (blue) and Entrust (green) OCSP responder performance.

OCSP responses can be stapled to a response from a web server when negotiating the TLS handshake to avoid the need for the browser to make a secondary request to a third party server. CloudFlare has claimed that enabling OCSP stapling has led to a 30% speed improvement for HTTPS sites. OCSP stapling support is present in newer versions of nginx — an increasingly popular open source web server — as a result of a development project sponsored by GlobalSign, DigiCert, and Comodo. OCSP stapling is not supported in the most popular version of Apache, 2.2.x, nor is it supported in current versions of Firefox (although support is in the pipeline), so it must remain only part of the solution for the foreseeable future. Frustrated by some of the limitations of OCSP, some CAs have lent support to a proposed an alternative revocation method using short lived certificates.

Browser support for the both OCSP and CRLs is mixed: currently, Firefox does not automatically download the CRLs from trusted CAs, so Firefox users must rely on OCSP alone; Google uses a proprietary mechanism to distribute CRLs to users of Google Chrome which aggregates per-CA CRLs into a single update which is distributed using its automatic update channel. Many browsers default to a "soft-fail" approach, leaving users vulnerable to eavesdroppers able to block or tamper with OCSP traffic. For as long as the CAs running OCSP responders do not have a strong record for both the performance and the reliability of their OCSP responders, browsers will find it difficult to justify switching to synchronous "hard-fail" behaviour.

Updated 18/04/2013

Angry Birds impersonated to distribute malware

As part of Netcraft's ongoing work in providing anti-fraud and anti-phishing services, we have recently discovered a significant number of Russian language attacks targeting users of popular pieces of software, including well known brands such as Angry Birds. This type of attack can be particularly successful as it exploits a user's trust in a brand. Malicious downloads for Android phones are becoming an increasingly common attack vector.

Angry Birds is a video game franchise created by Rovio Entertainment. The franchise gained popularity on Apple's iOS platform, and has since become available on all popular mobile and desktop operating systems. With over 1 billion downloads, and over 250 million active users, the franchise has become iconic in the marketplace — the original game and its variants are frequently seen in top ten app lists, so is continually attracting new users.

Angry Birds is impersonated to push malware.

Distributing malware purporting to be genuine software isn't a new tactic — Angry Birds has been a victim of this before. In this case smartphone users were hit by premium rate phone scams.

However, lately we have seen an increase in attackers taking additional measures to prevent their sites being found and taken down by the anti-phishing community. Restricting access to a site by country is one tactic that is becoming increasingly common. This is usually achieved via IP filtering; however Netcraft has seen attacks restricting access based on Accept-Language and User-Agent headers — one particular type of attack purported to provide a browser update, varying the brand impersonated depending on the User-Agent submitted.

Many of the attacks Netcraft has observed have been primarily composed of Russian language content, and restricted to IP addresses located in Russian-speaking countries. On another site impersonating Angry Birds, we found that when accessed from a proxy based in Russia, malware was distributed; however when attempting to download the content through a different proxy (located in Australia in the below example) we were redirected to Google.

IP filtering, amongst other measures taken by fraudsters, makes identifying and classifying phishing sites more difficult both for anti-phishing vendors and for hosting companies responding to abuse notifications.

You can protect yourself against phishing sites by installing Netcraft's Anti-Phishing Extension and help protect the internet community by reporting potential phishing sites to Netcraft by email to or at Netcraft can also help protect both brand owners and hosting companies.