1. Egypt back online, but some sites kept offline

    Renesys earlier confirmed that Egyptian internet providers had returned to the internet just before 09:30 UTC; however, a few important sites mysteriously went back offline a short while later. www.mcit.gov.eg came online for a brief period, but then went offline again less than an hour later:

    Before Egypt shut down internet access, the online collective Anonymous had been carrying out a distributed denial of service attack against this site; however, that attack did not appear to succeed at the time. Meanwhile, www.egypt.gov.eg has been online solidly since Egypt returned to the internet, whereas www.moiegypt.gov.eg has been coming and going:

    This site was also attacked as part of an online protest by Anonymous, which resulted in some short outages on 26th January. A tweet from AnonymousIRC suggests that this site may be being kept offline by a second DDoS attack:

    We are continuing to monitor the performance of several Egyptian sites at http://uptime.netcraft.com/perf/reports/performance/wikileaks

    Posted by Paul Mutton on 2nd February, 2011 in Around the Net

  2. is.gd URL shortener suffers downtime

    The popular is.gd URL shortening service was reportedly unavailable for a few hours this morning, effectively breaking thousands of shortlinks posted to Twitter and other social networking sites. During the outage, the site's public-facing load balancer responded to PING requests, but was refusing HTTP connections to port 80.

    is.gd is one of the most popular URL shortening services in current use – it has shortened 334 million URLs to date, which have been accessed more than 11 billion times. The service is wholly owned by UK hosting company Memset, which hosts the site on their own servers. Since December, Memset has also provided a shorter v.gd service, but this has only attracted 61 thousand URLs so far.

    Memset told Netcraft that today's fault was caused by the failure of some virtual machines in the frontend cloud, which is responsible for accepting HTTP requests from the load balancer. These have been restored and the site is now back up and running with improved monitoring processes.

    is.gd is primarily maintained by its creator, Richard West, a freelance developer and technologist. Memset proudly describe it as an "ethical" URL shortener; in particular, they have pledged to support is.gd as a free service indefinitely, will never place third-party adverts on the site and claim to be one of the most proactive URL shorteners in preventing spam and misuse.

    Other sites hosted by Memset, including its own main presence at memset.com, were unaffected during the is.gd outage.

    Posted by Paul Mutton on 2nd February, 2011 in Around the Net

  3. Most Reliable Hosting Company Sites in January 2011

    Rank Company site OS Outage
    hh:mm:ss
    Failed
    Req%
    DNS Connect First
    byte
    Total
    1 Datapipe FreeBSD 0.000 0.061 0.015 0.031 0.042
    2 iWeb Technologies Linux 0.000 0.077 0.038 0.077 0.077
    3 www.codero.com Linux 0.000 0.129 0.040 0.291 0.484
    4 Multacom FreeBSD 0.000 0.156 0.068 0.138 0.402
    5 INetU unknown 0:00:00 0.004 0.049 0.021 0.063 0.135
    6 Kattare Internet Services Linux 0:00:00 0.008 0.152 0.083 0.170 0.342
    7 www.cwcs.co.uk Linux 0:00:00 0.008 0.248 0.115 0.464 0.871
    8 New York Internet FreeBSD 0:00:00 0.013 0.061 0.020 0.052 0.130
    9 Swishmail FreeBSD 0:00:00 0.013 0.051 0.021 0.043 0.117
    10 Rackspace F5 Big-IP 0:00:00 0.013 0.067 0.024 0.049 0.049

    See full table

    Four of the hosting companies sampled this month experienced no failed requests at all to their websites, and these four are therefore ranked by average connection time. Top of the table is Datapipe, which has been in the top ten in ten of the last twelve months, and in the top three seven times in the same period. The company offers a variety of services including colocation, cloud computing and hosted servers from datacentres in the U.S., the U.K. and China.

    Second this month is iWeb Technologies, which has also appeared in the top ten regularly over the last year. iWeb have been offering hosting services for over ten years with a range of products which includes shared web hosting, managed hosting and dedicated servers.

    www.codero.com ranked third this month. The company is a relative newcomer to the hosting industry, having begun life as a division of aplus.net in July 2009. It now offers dedicated and managed hosting services aimed specifically at small businesses.

    In terms of operating system, the top ten hosting companies are evenly split between Linux and FreeBSD this month with four of each and one company running F5 Big-IP.

    Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

    From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

    Information on the measurement process and current measurements is available.

    Posted by Jennifer Cownie on 1st February, 2011 in Hosting, Performance

  4. Police.uk fails to cope with public demand

    A new local crime and policing website for England and Wales was launched late last night at police.uk. The revamped site provides instant access to street-level crime maps and data – or at least, it did until curious members of the public woke up this morning.

    In what could arguably be described as a media-driven DDoS, the new site has received a lot of publicity on the internet, radio and television today. As a result, a huge number of visitors appears to have swamped the police.uk site with traffic, causing it to break. Search results are currently returning error messages, or a blank page with a 503 Service Unavailable response header.

    One response worryingly suggests there are no police in London:

    The new police.uk site has been developed by advertising agency Rock Kitchen Harris, who also developed the original CrimeMapper site for all 43 English and Welsh police forces in 2009. The launch was announced today on their website, where they said:

    "We not only designed, built and manage the site we also arranged the hosting using a mix of servers, with the public website using scaleable cloud hosting."

    Despite the use of scaleable cloud hosting (in this case, Amazon EC2), the site does not appear to be holding out too well. Amazon's EC2 hosting service does provide a facility called Auto Scaling, which deals with traffic spikes by automatically increasing capacity, but it is not clear whether RKH have enabled this feature. WikiLeaks notably used Amazon EC2 when the Iraq War Logs and Cablegate sites went live, both of which coped well with the initial large volume of traffic.

    Netcraft was unable to speak to anyone in the web team at RKH, as they are, understandably, "a bit tied up at the moment", but it was confirmed that the current problem is a result of too much traffic.

    Posted by Paul Mutton on 1st February, 2011 in Around the Net

  5. Anonymous attacks websites in Egypt

    Following the recent uprising in Tunisia, thousands of demonstrators took to the streets in Egypt yesterday to demand an end to President Hosni Mubarak's rule. The online collective known as Anonymous has joined in the protests by orchestrating distributed denial of service attacks against key Egyptian websites.

    Operation: Egypt began its recruitment campaign 3 days ago, inviting participants to join the #OpEgypt channel on its IRC network. As with the previous attacks against PayPal, MasterCard and Visa, volunteers are being sought to install and run the Low Orbit Ion Cannon (LOIC) software, which can automatically bombard a website with a large volume of traffic.

    Last night, Twitter confirmed that it had been blocked in Egypt:

    Some of the earlier DDoS attacks carried out by Anonymous had used Twitter feeds to announce targets to the automated attack software. IRC appears to be the primary control point now, with the current target set to www.mcit.gov.eg – The Egyptian Ministry of Communications and Information Technology.

    It is not clear how many people are involved in this attack, but our performance data for www.mcit.gov.eg shows the website is currently up and responding to HTTP requests from around the world.

    This could suggest that the number of volunteers taking part in the attacks has continued to decrease over the past few months. In an interview earlier this month, Anonymous member Sven Slootweg said that he did not expect many more DDoS attacks as the impact is limited.

    To muster up more attackers, the IRC channel also invites users to use a web-based version of the LOIC attack tool, which can even be used from mobile phones. Further discussions, including which targets to attack, are being carried out on the interactive multi-user PiratePad site:

    This also lists www.moiegypt.gov.eg as a main target, which has experienced some short outages (performance graphs).

    Posted by Paul Mutton on 26th January, 2011 in Around the Net

  6. January 2011 Web Server Survey

    In the January 2011 survey we received responses from 273,301,445 sites.

    nginx once again saw the largest increase in market share amongst the top web servers, gaining 0.88 percentage points or 3.59M hostnames. nginx now controls 7.50% of hostnames in the survey, though only 6.12% within the top 1 million sites. 2.4M of the additional nginx hostnames are hosted by Ecatel, which appears to have migrated from Apache to nginx this month. nginx's growth in active sites is more modest, adding 429k active sites, leaving nginx with a 8.23% share.

    Lighttpd was the only other web server to increase its share of hostnames this month, increasing by 558k hostnames, 531k of which are hosted by Secure Hosting Limited.

    Apache remains the dominant force, commanding a 59.13% share and an increase of 10.1M hostnames. The majority of the Apache growth was in the United States and Germany, together adding 9.9M hostnames. The growth in the United States is concentrated in AmeriNOC and Softlayer, together accounting for 7.34M hostnames. Germany's growth, on the other hand, is concentrated at a single hosting company, United Internet AG, adding domain holding pages for fhe3rz.net.

    Microsoft made modest gains this month, adding 669k hostnames despite losing 1.2 percentage points in market share.

    Total Sites Across All Domains
    August 1995 - January 2011

    Total Sites Across All Domains, August 1995 - January 2011


    Market Share for Top Servers Across All Domains
    August 1995 - January 2011

    Graph of market share for top servers across all domains, August 1995 - January 2011


    DeveloperDecember 2010PercentJanuary 2011PercentChange
    Apache151,516,15259.35%161,591,44559.13%-0.23
    Microsoft56,723,54422.22%57,392,35121.00%-1.22
    nginx16,910,2056.62%20,504,6347.50%0.88
    Google14,933,8655.85%15,112,5325.53%-0.32
    lighttpd1,308,9350.51%1,866,8720.68%0.17
    (more...)

    Posted by Netcraft on 12th January, 2011 in Web Server Survey