The popular is.gd URL shortening service was reportedly unavailable for a few hours this morning, effectively breaking thousands of shortlinks posted to Twitter and other social networking sites. During the outage, the site's public-facing load balancer responded to PING requests, but was refusing HTTP connections to port 80.
is.gd is one of the most popular URL shortening services in current use – it has shortened 334 million URLs to date, which have been accessed more than 11 billion times. The service is wholly owned by UK hosting company Memset, which hosts the site on their own servers. Since December, Memset has also provided a shorter v.gd service, but this has only attracted 61 thousand URLs so far.
Memset told Netcraft that today's fault was caused by the failure of some virtual machines in the frontend cloud, which is responsible for accepting HTTP requests from the load balancer. These have been restored and the site is now back up and running with improved monitoring processes.
is.gd is primarily maintained by its creator, Richard West, a freelance developer and technologist. Memset proudly describe it as an "ethical" URL shortener; in particular, they have pledged to support is.gd as a free service indefinitely, will never place third-party adverts on the site and claim to be one of the most proactive URL shorteners in preventing spam and misuse.
Other sites hosted by Memset, including its own main presence at memset.com, were unaffected during the is.gd outage.
Rank Company site OS Outage
DNS Connect First
Total 1 Datapipe FreeBSD 0.000 0.061 0.015 0.031 0.042 2 iWeb Technologies Linux 0.000 0.077 0.038 0.077 0.077 3 www.codero.com Linux 0.000 0.129 0.040 0.291 0.484 4 Multacom FreeBSD 0.000 0.156 0.068 0.138 0.402 5 INetU unknown 0:00:00 0.004 0.049 0.021 0.063 0.135 6 Kattare Internet Services Linux 0:00:00 0.008 0.152 0.083 0.170 0.342 7 www.cwcs.co.uk Linux 0:00:00 0.008 0.248 0.115 0.464 0.871 8 New York Internet FreeBSD 0:00:00 0.013 0.061 0.020 0.052 0.130 9 Swishmail FreeBSD 0:00:00 0.013 0.051 0.021 0.043 0.117 10 Rackspace F5 Big-IP 0:00:00 0.013 0.067 0.024 0.049 0.049
Four of the hosting companies sampled this month experienced no failed requests at all to their websites, and these four are therefore ranked by average connection time. Top of the table is Datapipe, which has been in the top ten in ten of the last twelve months, and in the top three seven times in the same period. The company offers a variety of services including colocation, cloud computing and hosted servers from datacentres in the U.S., the U.K. and China.
Second this month is iWeb Technologies, which has also appeared in the top ten regularly over the last year. iWeb have been offering hosting services for over ten years with a range of products which includes shared web hosting, managed hosting and dedicated servers.
www.codero.com ranked third this month. The company is a relative newcomer to the hosting industry, having begun life as a division of aplus.net in July 2009. It now offers dedicated and managed hosting services aimed specifically at small businesses.
In terms of operating system, the top ten hosting companies are evenly split between Linux and FreeBSD this month with four of each and one company running F5 Big-IP.
Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.
From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.
Information on the measurement process and current measurements is available.
A new local crime and policing website for England and Wales was launched late last night at police.uk. The revamped site provides instant access to street-level crime maps and data – or at least, it did until curious members of the public woke up this morning.
In what could arguably be described as a media-driven DDoS, the new site has received a lot of publicity on the internet, radio and television today. As a result, a huge number of visitors appears to have swamped the police.uk site with traffic, causing it to break. Search results are currently returning error messages, or a blank page with a 503 Service Unavailable response header.
One response worryingly suggests there are no police in London:
The new police.uk site has been developed by advertising agency Rock Kitchen Harris, who also developed the original CrimeMapper site for all 43 English and Welsh police forces in 2009. The launch was announced today on their website, where they said:
"We not only designed, built and manage the site we also arranged the hosting using a mix of servers, with the public website using scaleable cloud hosting."
Despite the use of scaleable cloud hosting (in this case, Amazon EC2), the site does not appear to be holding out too well. Amazon's EC2 hosting service does provide a facility called Auto Scaling, which deals with traffic spikes by automatically increasing capacity, but it is not clear whether RKH have enabled this feature. WikiLeaks notably used Amazon EC2 when the Iraq War Logs and Cablegate sites went live, both of which coped well with the initial large volume of traffic.
Netcraft was unable to speak to anyone in the web team at RKH, as they are, understandably, "a bit tied up at the moment", but it was confirmed that the current problem is a result of too much traffic.
Following the recent uprising in Tunisia, thousands of demonstrators took to the streets in Egypt yesterday to demand an end to President Hosni Mubarak's rule. The online collective known as Anonymous has joined in the protests by orchestrating distributed denial of service attacks against key Egyptian websites.
Operation: Egypt began its recruitment campaign 3 days ago, inviting participants to join the #OpEgypt channel on its IRC network. As with the previous attacks against PayPal, MasterCard and Visa, volunteers are being sought to install and run the Low Orbit Ion Cannon (LOIC) software, which can automatically bombard a website with a large volume of traffic.
Last night, Twitter confirmed that it had been blocked in Egypt:
Some of the earlier DDoS attacks carried out by Anonymous had used Twitter feeds to announce targets to the automated attack software. IRC appears to be the primary control point now, with the current target set to www.mcit.gov.eg – The Egyptian Ministry of Communications and Information Technology.
It is not clear how many people are involved in this attack, but our performance data for www.mcit.gov.eg shows the website is currently up and responding to HTTP requests from around the world.
This could suggest that the number of volunteers taking part in the attacks has continued to decrease over the past few months. In an interview earlier this month, Anonymous member Sven Slootweg said that he did not expect many more DDoS attacks as the impact is limited.
To muster up more attackers, the IRC channel also invites users to use a web-based version of the LOIC attack tool, which can even be used from mobile phones. Further discussions, including which targets to attack, are being carried out on the interactive multi-user PiratePad site:
In the January 2011 survey we received responses from 273,301,445 sites.
nginx once again saw the largest increase in market share amongst the top web servers, gaining 0.88 percentage points or 3.59M hostnames. nginx now controls 7.50% of hostnames in the survey, though only 6.12% within the top 1 million sites. 2.4M of the additional nginx hostnames are hosted by Ecatel, which appears to have migrated from Apache to nginx this month. nginx's growth in active sites is more modest, adding 429k active sites, leaving nginx with a 8.23% share.
Lighttpd was the only other web server to increase its share of hostnames this month, increasing by 558k hostnames, 531k of which are hosted by Secure Hosting Limited.
Apache remains the dominant force, commanding a 59.13% share and an increase of 10.1M hostnames. The majority of the Apache growth was in the United States and Germany, together adding 9.9M hostnames. The growth in the United States is concentrated in AmeriNOC and Softlayer, together accounting for 7.34M hostnames. Germany's growth, on the other hand, is concentrated at a single hosting company, United Internet AG, adding domain holding pages for fhe3rz.net.
Microsoft made modest gains this month, adding 669k hostnames despite losing 1.2 percentage points in market share.Total Sites Across All Domains
August 1995 - January 2011
Market Share for Top Servers Across All Domains
August 1995 - January 2011
Developer December 2010 Percent January 2011 Percent Change Apache 151,516,152 59.35% 161,591,445 59.13% -0.23 Microsoft 56,723,544 22.22% 57,392,351 21.00% -1.22 nginx 16,910,205 6.62% 20,504,634 7.50% 0.88 14,933,865 5.85% 15,112,532 5.53% -0.32 lighttpd 1,308,935 0.51% 1,866,872 0.68% 0.17
Popular BitTorrent site The Pirate Bay is one of many joining in with today's "blackout" protests against a new media law approved in Hungary. The law allows Hungarian publications to be fined for violating public order, which could require journalists to reveal their sources.
The blackout campaign is being publicised through blackout4hungary.net, which encourages other sites to take part in the protest by adding black banners or stylesheets to their sites. The blackout4hungary.net site appears to have taken the blackout concept a step further and is currently offline, although this may just be a consequence of the large amount of traffic being driven to the site through hyperlinks and hotlinked stylesheets.
The Pirate Bay is currently hosted in Germany and uses the lighttpd web server. The site has nearly 5 million registered users and claims to be "the world's most resilient BitTorrent"; however, the site's availability has been somewhat choppy since Tuesday.