BBC websites back to normal, DDoS mitigation reverted

The BBC's websites are now back to normal, four days after being taken down by an effective DDoS attack on New Year's Eve.

The BBC mitigated the attack within a few hours by moving its main website onto the Akamai content delivery network, which restored access to its millions of users. However, during this mitigation period, some of the BBC's other websites – which were still hosted at the BBC – remained mostly unreachable.

The BBC's DDoS mitigation was only temporary, and last night it moved its main website off Akamai, back onto a netblock owned by the BBC. This move resulted in another short outage on 4th January, followed by several hours of slightly slower response times within the UK. By the 5th January, the response times had settled down to be almost comparable with when it was using Akamai.

The main BBC website experienced another short outage last night as it moved off the Akamai CDN.

The main BBC website experienced another short outage last night as it moved off the Akamai CDN.

However, as expected, response times from other countries are no longer as fast as they were when the BBC's main website was hosted on the Akamai CDN. Response times from the US are notably slower, but currently no worse than they were before the DDoS attacks on New Year's Eve.

Response times from the US are now much slower again (although international visitors would typically visit bbc.com rather than bbc.co.uk).

Response times from the US are now much slower again (although international visitors would typically visit bbc.com rather than bbc.co.uk).

During the period in which the BBC's main website was hosted on the Akamai CDN, its legacy News website at news.bbc.co.uk remained hosted at the BBC. This was mostly unavailable during this period, with most client connection attempts being reset.

news.bbc.co.uk is now functioning normally, too.

news.bbc.co.uk is now functioning normally, too.

This site's availability was restored to normal at the same time that the main BBC website moved off Akamai. This suggests that the connection resets were a deliberate attempt to mitigate basic DDoS attacks, rather than as a direct side effect of a sustained DDoS attack. However, this approach was not ideal – while some browsers (such as Chrome) would automatically retry the connection attempt (often successfully), other browsers would give up at the first failure.

BBC websites still suffering after DDoS attack

Since suffering a crippling DDoS attack on New Year's Eve, some BBC websites are still experiencing significant performance issues.

Around 07:00 UTC on 31 December 2015, the main BBC website at www.bbc.co.uk was knocked offline after being subjected to a distributed denial of service attack. For the following few hours, requests to the BBC website either eventually timed out, or were responded to with its 500 Internal Error test card page. A group called New World Hacking later claimed responsibility for the attack, which it carried out as a test of its capabilities.

Requests that did not time out were eventually met with the BBC test card error page.

Requests that did not time out were eventually met with the BBC test card error page.

The British Broadcasting Corporation is the public service broadcaster of the United Kingdom, and the outage had a significant impact on its user base: The BBC's news, sport, weather and iPlayer TV and radio catchup services are all delivered via www.bbc.co.uk.

Performance chart for www.bbc.co.uk, showing the primary outage period.

Performance chart for www.bbc.co.uk, showing the primary outage period.

At the time of the attack, www.bbc.co.uk was served from a netblock owned by the BBC. It seems that service was restored by migrating the site onto the Akamai content delivery network, after which there were no apparent outages.

OS Server Last seen IP address Netblock Owner
Linux nginx 3-Jan-2016 88.221.48.170 Akamai
Linux nginx 2-Jan-2016 95.101.129.88 Akamai Technologies
Linux nginx 31-Dec-2015 95.101.129.106 Akamai Technologies
Linux nginx 30-Dec-2015 212.58.244.70 BBC
Linux nginx 29-Dec-2015 212.58.246.54 BBC
Linux nginx 28-Dec-2015 212.58.244.71 BBC

Moving www.bbc.co.uk onto the Akamai CDN also resulted in some significant performance benefits, particularly from locations outside of the UK. For example, prior to the attack, most requests from Netcraft's New York performance collector took around 0.4-0.6 seconds to receive a response, whereas after the site had migrated to Akamai, all requests were served in well under 0.1 seconds. These performance benefits are typical when using a globally distributed CDN, as cached content can be delivered from an edge server within the client's own country, rather than from a remote server that can only be reached via transatlantic cables.

Performance chart for www.bbc.co.uk from  New York, highlighting the improved response times and successful attack  mitigation after switching to Akamai.

Performance chart for www.bbc.co.uk from New York, highlighting the improved response times and successful attack mitigation after switching to Akamai.

However, not all of the BBC's websites have migrated to Akamai, and some of these are still exhibiting connectivity issues in the aftermath of the attack. For example, search.bbc.co.uk and news.bbc.co.uk are still hosted directly at the BBC, and these are still experiencing problems today.

The BBC's News service is currently found at www.bbc.co.uk/news, but up until a few years ago it used to be served from its own dedicated hostname, news.bbc.co.uk. This legacy hostname is still used by some webpages today, but mostly redirects visitors to the new site at www.bbc.co.uk/news. This conveniently collates all of the BBC's main online services under the same hostname, but at the expense of introducing a single point of failure. If each service were still to be found under a different hostname and on different servers, it might have offered further resilience to the initial attack.

The performance chart for news.bbc.co.uk shows massive outages long after the DDoS attack on New Year's Eve.

The performance chart for news.bbc.co.uk shows massive outages long after the DDoS attack on New Year's Eve.

As shown above, news.bbc.co.uk was also affected by the DDoS attack which took down the main BBC website, but eventually came back online later that day without having to relocate the website. However, the following morning (New Year's Day), it started to experience significant connectivity problems.

Most requests to news.bbc.co.uk are still failing.

Most requests to news.bbc.co.uk are still failing. Some browsers, such as Chrome, may automatically retry the request.

It is unclear whether this indicates a separate ongoing attack, or an attempt at mitigating such attacks, but nonetheless, it is likely to affect lots of users: Many old news articles are still served directly from news.bbc.co.uk, and some users habitually reach the news website by typing news.bbc.co.uk into their browsers. Some regularly updated pages also continue to be served from news.bbc.co.uk, such as horse racing results.

Most Reliable Hosting Company Sites in December 2015

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 EveryCity SmartOS 0:00:00 0.000 0.093 0.064 0.128 0.128
2 Lightcrest unknown 0:00:00 0.004 0.276 0.006 0.023 0.027
3 One.com Linux 0:00:00 0.004 0.203 0.037 0.105 0.105
4 Memset Linux 0:00:00 0.004 0.153 0.064 0.157 0.245
5 ServerStack Linux 0:00:00 0.004 0.125 0.067 0.135 0.135
6 Netcetera Linux 0:00:00 0.004 0.075 0.084 0.171 0.171
7 Codero Citrix Netscaler 0:00:00 0.004 0.177 0.092 0.189 0.381
8 GoDaddy.com Inc Linux 0:00:00 0.008 0.264 0.007 0.018 0.018
9 Datapipe Linux 0:00:00 0.008 0.145 0.012 0.024 0.031
10 Swishmail FreeBSD 0:00:00 0.008 0.144 0.062 0.125 0.168

See full table

EveryCity had the most reliable hosting company site in December 2015. Despite moving into new offices, its website was the only one to respond to all of Netcraft's requests. EveryCity has maintained its 100% uptime record throughout 2015, and has made it into the top ten 11 times during the year. It also had the most reliable hosting company site in May.

In second place in December was Lightcrest, which also appeared in the top ten in November. It experienced only one failed request, with an impressively fast average connection time of 6 milliseconds. Lightcrest operates its cloud services using its own Kahu Compute Fabric infrastructure, without outsourcing any components to third-party cloud providers.

In third place – also with a single failed request, but with a slower average connection time – was One.com. Established in 2002, One.com now has over 270 employees with companies registered in Denmark, India and Dubai.

Six of December's top ten hosting company sites ran on Linux operating systems, while Swishmail used FreeBSD, Codero used a Citrix Netscaler device, and EveryCity used SmartOS. The latter is a community fork of OpenSolaris, featuring the ZFS file system, DTrace dynamic tracing, kernel-based virtual machines and Solaris Zones operating system-level virtualisation.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

Most Reliable Hosting Company Sites in November 2015

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 One.com Linux 0:00:00 0.000 0.211g 0.035 0.100 0.100
2 GoDaddy.com Inc Linux 0:00:00 0.009 0.258 0.008 0.019 0.020
3 Datapipe Linux 0:00:00 0.009 0.145 0.012 0.025 0.031
4 XILO Communications Ltd. Linux 0:00:00 0.009 0.213 0.063 0.126 0.126
5 EveryCity SmartOS 0:00:00 0.009 0.094 0.065 0.131 0.131
6 Anexia Linux 0:00:00 0.009 0.191 0.083 0.169 0.169
7 Hivelocity Linux 0:00:00 0.009 0.174 0.087 0.174 0.174
8 Lightcrest unknown 0:00:00 0.013 0.278 0.007 0.019 0.023
9 INetU Linux 0:00:00 0.013 0.146 0.066 0.131 0.131
10 ServerStack Linux 0:00:00 0.013 0.130 0.066 0.132 0.132

See full table

One.com topped the table after successfully responding to all of Netcraft's requests in November. It last appeared in the top 10 in June 2015 when it placed ninth. The Denmark-based company offers shared hosting packages — all of which include unlimited traffic — from data centres operated by Interxion.

After having the most reliable hosting company site for the past three months, GoDaddy narrowly missed the top spot in November. Although five other sites also had two failed requests, GoDaddy's site is placed second based on its faster average connection time of 8ms.

Also with just two failed requests, Datapipe placed third in November. Datapipe has appeared in all but one of the top 10 lists in 2015 and has maintained a 100% uptime record for more than nine years. For the seventh time, Datapipe was recognised as one of New Jersey's 50 fastest growing companies, after a year that included the acquistion of GoGrid and DualSpark.

The top 10 is once again dominated by websites using Linux, which is used by eight of the top 10 sites. EveryCity, the only site that has been in the top 10 for every month of this year so far, uses SmartOS. Microsoft Windows is absent from the list for the fifth consecutive month.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

Most Reliable Hosting Company Sites in October 2015

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 GoDaddy.com Inc Linux 0:00:00 0.000 0.255 0.007 0.019 0.019
2 Swishmail FreeBSD 0:00:00 0.000 0.153 0.062 0.123 0.166
3 XILO Communications Ltd. Linux 0:00:00 0.000 0.224 0.063 0.126 0.126
4 Datapipe Linux 0:00:00 0.004 0.156 0.012 0.025 0.032
5 Qube Managed Services Linux 0:00:00 0.004 0.148 0.043 0.088 0.088
6 EveryCity SmartOS 0:00:00 0.004 0.092 0.066 0.133 0.133
7 Anexia Linux 0:00:00 0.004 0.187 0.085 0.171 0.172
8 Hivelocity Hosting Linux 0:00:00 0.004 0.185 0.091 0.181 0.181
9 Bigstep Linux 0:00:00 0.013 0.160 0.060 0.122 0.122
10 Memset Linux 0:00:00 0.013 0.157 0.063 0.160 0.246

See full table

For the third month in a row, GoDaddy has had the most reliable hosting company site. It responded to every availability request made by Netcraft throughout October, with an average connection time of just 7 milliseconds. This is the sixth time this year that GoDaddy has featured in the top ten.

On 29 October, GoDaddy announced a new 'best-in-class' partner offer with AdAgility for customers of its GoDaddy Pro programme. This additional feature allows web designers and developers to take greater control over the advertising content on their sites.

Swishmail has risen to second place for reliability this month. This is now its third appearance in the top ten since January, with the last appearance—in July—placing it tenth. Like GoDaddy, Swishmail responded to all of the availability requests made by Netcraft throughout October; however, the average connection time sat higher at 62 milliseconds.

XILO Communications Ltd. saw a return to the top three most reliable hosting companies this month for the first time since January. Like GoDaddy and Swishmail, 100% of the availability requests made by Netcraft received a response. With an average connection time of 63 milliseconds, this makes it the eighth time this year that XILO Communications Ltd. has made it into the top ten.

Yet again, the table this month is dominated by Linux, which is used by eight of the top ten sites. EveryCity uses the SmartOS community fork of OpenSolaris, while Swishmail uses the FreeBSD operating system. This is now the fourth month in a row where Microsoft Windows has been completely absent from the top ten.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

Most Reliable Hosting Company Sites in September 2015

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 GoDaddy.com Inc Linux 0:00:00 0.000 0.256 0.008 0.019 0.019
2 Datapipe Linux 0:00:00 0.004 0.164 0.012 0.025 0.032
3 Qube Managed Services Linux 0:00:00 0.004 0.150 0.044 0.089 0.089
4 XILO Communications Ltd. Linux 0:00:00 0.009 0.224 0.064 0.128 0.129
5 Memset Linux 0:00:00 0.013 0.159 0.065 0.158 0.248
6 EveryCity SmartOS 0:00:00 0.018 0.100 0.067 0.133 0.134
7 Anexia Linux 0:00:00 0.018 0.191 0.086 0.174 0.174
8 Pickaweb Linux 0:00:00 0.026 0.305 0.010 0.168 0.169
9 Hyve Managed Hosting Linux 0:00:00 0.031 0.110 0.063 0.127 0.128
10 Pair Networks FreeBSD 0:00:00 0.031 0.241 0.070 0.141 0.141

See full table

For the second month in a row, GoDaddy had the most reliable hosting company website. Throughout September, it was the only company to respond to all of Netcraft's requests, with an average connection time of only 8 milliseconds. This is the fifth time this year that GoDaddy has been in the top ten.

GoDaddy recently added new features to its GoDaddy Pro programme, which has grown to more than 50,000 members since its launch in June. Aimed at web designers and developers, it allows members to manage clients and clients' products, offering discounts of at least 30%.

Datapipe had the second most reliable hosting company website in September, with just a single failed request. Datapipe came first in July, and August was the only month so far this year when it was absent from the top ten.

On 9 September 2015, Datapipe announced that it had acquired DualSpark – an Amazon Web Services assessment, automation and migration company, which was founded by former AWS managers. Datapipe hopes to use DualSpark's DevOps and automation expertise to bring increased levels of support to its cloud clients.

Qube Managed Services ranked third in September, also with only one failed request, but with a marginally longer average connection time than Datapipe. Qube's site has appeared in the top ten for all but two months so far in 2015, making this its seventh appearance.

Linux dominates the chart this month, being used by eight of the top ten sites, and all of the top five. EveryCity uses the SmartOS community fork of OpenSolaris, while Pair Networks uses the FreeBSD operating system. This is the third month in a row where Microsoft Windows has been completely absent from the top ten.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.