Cloud hosting startup DigitalOcean has grown extraordinarily over the past six months. In December 2012, DigitalOcean had just over 100 web-facing computers; in June 2013, Netcraft found more than 7,000. DigitalOcean provides SSD-backed virtual computers which are available by the hour. The cheapest droplet, as it refers to virtual computers, costs less than a cent per hour, about a third of the price of Amazon's cheapest option. DigitalOcean claims to be able to provision a new droplet within 55 seconds in one of three locations: New York, Amsterdam (available since January 2012), and San Francisco (April 2013).
Hosting provider December 2012 June 2013 Growth Growth (%) Amazon 134,117 165,438 +31,321 +23.35% Alibaba 6,649 17,347 +10,699 +160.91% Hetzner 75,880 84,896 +9,016 +11.88% DigitalOcean 138 7,134 +6,996 +5084.64% OVH 90,305 96,558 +6,253 +6.92% Shore Network Tech (Linode) 54,051 57,701 +3,650 +6.75%
Fastest growing hosting providers by web-facing computers, December 2012 to June 2013.
Over the last six months DigitalOcean had the fourth largest growth in web-facing computers with only Amazon, Alibaba, and Hetzner ahead of it. DigitalOcean's more than 50-fold growth makes it the 72nd largest hosting provider in the world by web-facing computers, up from 549th in December and 102nd last month. DigitalOcean had the second largest growth of web-facing computers last month — it was one of only five hosting providers to grow by more than 1,000 web-facing computers — and it contributed 10% of the total growth worldwide.
Previous hosting provider Net movement to DigitalOcean New sites +6,211 Rackspace +1,475 Shore Network Tech (Linode) +1,028 Amazon +626 Softlayer +263
Sites (hostnames) switching to DigitalOcean from notable providers, May 2013 to June 2013
As shown in the table above, websites are migrating to DigitalOcean from its better known competitors; last month almost 1,500 websites moved from Rackspace and 1,000 from Shore Network Tech (Linode). NewsBlur, a news aggregation website, is one of the busiest sites hosted at DigitalOcean. Several websites belonging to the Ruby on Rails project including guides.rubyonrails.org and api.rubyonrails.org are now hosted at DigitalOcean after recently moving from Linode.
Operating System share by web-facing computer at DigitalOcean in June 2013
DigitalOcean provide a number of template images in order to create a droplet, including five Linux distributions: Ubuntu, CentOS, Debian, Arch Linux and Fedora. Of the web-facing computers presenting a distribution-specific server banner, Ubuntu is by far the most popular: over 70% of web-facing computers with an identifiable distribution are using the Debian-derived distribution. Microsoft Windows is conspicuous by its absence; DigitalOcean has postponed plans to support Windows citing complexities including licensing and security concerns.
DigitalOcean — 2012 graduate of the Tech Stars startup accelerator — has had difficulty getting access to a sufficient quantity of IPv4 addresses in Amsterdam which meant that it was uneconomic to provide smaller droplet sizes. In May 2013, DigitalOcean announced the availability of further IP addresses for smaller droplet sizes in Europe, re-enabling the creation of 512MB and 1GB droplets.
Netcraft provides information on the Internet infrastructure, including the hosting industry, and web content technologies. For information on the cloud computing industry visit www.netcraft.com.
In the June 2013 survey we received responses from 672,985,183 sites, 148k more than last month.
Both Microsoft and Google grew slightly this month, gaining 0.5 percentage points of market share. Microsoft's web server, IIS, now serves 17.22% of the world's websites, down from a historic high of 37% which it reached in October 2007. Microsoft IIS's market share amongst secure websites (HTTPS) is significantly higher: it serves 39% of the secure websites found by Netcraft and is in 2nd place behind Apache. Apache's lead over Microsoft in the secure website market is only slight: it is ahead by just two percentage points and doesn't hold an absolute majority as it does for non-secure websites (HTTP).
Despite its market share dipping slightly, Apache is still significantly ahead of its position just two months ago due to Go Daddy's switch last month to Apache Traffic Server. Within the Million Busiest Sites, Apache bucked its recent downward trend this month: 7,300 more websites than last month are using Apache, including DigiCert's website which switched from nginx to Apache 2.4.5 (2.4.4 is the latest stable release).
nginx's growth within the Million Busiest Sites remains strong, 5,400 more busy websites now use the web server since last month's survey including The Verge which switched from Apache. Across all web sites, however, nginx lost almost 1% of market share and 6.4M websites caused by a large network of websites at namecheap.com failing to respond during the survey.
In early May 2013, nginx released a patch for a high severity security vulnerability which could allow an attacker to execute arbitrary code. Several attacks exploiting the vulnerability in the chunked transfer size calculation have been demonstrated including a proof of concept and an automated metasploit module. Almost 2M websites — or around 2% of all websites using nginx — presented a server banner corresponding to a vulnerable version (1.3.9+ and 1.4.0). The vast majority of nginx websites do not report the version in the server banner; however, the two most popular versions reported are 1.2.1 (released in June 2012) and 1.0.15 (released in April 2012) which do not have this vulnerability but may have others if left unpatched.
nginx is the most commonly used web server at Amazon: it is used on 41% of the 12M websites hosted using EC2 or S3. Last month Netcraft reported Amazon had 158k web-facing computers and has been the largest hosting provider by the number of web-facing computers since September 2012. After nginx, Apache is the next most common web server, 24.7% of websites use it, followed by Microsoft with 14%. Only 1% presented the AmazonS3 server banner, which can be used to host entire static websites in addition to simply static files.
Developer May 2013 Percent June 2013 Percent Change Apache 359,441,468 53.42% 358,974,045 53.34% -0.08 Microsoft 112,303,412 16.69% 115,920,681 17.22% 0.53 nginx 104,411,087 15.52% 97,991,191 14.56% -0.96 23,029,260 3.42% 26,036,616 3.87% 0.45
In September 2012 Netcraft reported that Amazon had become the largest hosting company in the world based on the number of web-facing computers. In the last eight months, the e-commerce company's tally of web-facing computers has grown by more than a third, reaching 158k. The number of websites hosted on these computers has also increased, from 6.8M in September 2012 to 11.6M in May 2013, a 71% increase.
Although Amazon’s main business is still online retail, Amazon Web Services (AWS), its cloud computing division, has been growing in significance. In Amazon's first quarter of 2013 the Other category (which still includes AWS along with other non-retail activity) was just under 5.0% of its revenue, up from 3.2% at the same point in 2011. The first publicly available AWS service was launched in 2004, but it was not until 2006 that Amazon launched its two core services S3 (data storage) and EC2 (per-hour rental of virtual computer instances). Since then, Amazon has been increasing the number of services provided: in 2012 alone, 159 new services and features were released.
Including its retail infrastructure, the number of web-facing computers at Amazon has grown more than thirty-fold in four years: in May 2009, Netcraft found 4,600 Amazon-controlled web-facing computers; in May 2013, Netcraft found 158k web-facing computers on 164k IP addresses. Netcraft estimates the number of computers behind a group of IP addresses by using a variety of heuristics based on the TCP/IP characteristics seen in the HTTP responses gathered. Hosted on those computers, there are more than 11.6M websites (or hostnames) which corresponds to 2.1M websites with unique content (active sites). Despite being the largest hosting provider by number of web-facing computers, it is dwarfed by Go Daddy, the largest hosting provider when considering the number of websites hosted. Go Daddy has 37M websites on just 23k web-facing computers: the high ratio of websites to web-facing computers may be indicative of Go Daddy's role as a registrar, for which it has a large network of holding pages, and its inexpensive shared hosting platform.
EC2 - Elastic Compute Cloud
EC2, provides on-demand virtual-computer instances billed per hour and is currently available from all nine AWS regions. Each region may correspond to multiple physical data centres which are structured into "Availability Zones". The two largest regions, US East (Northern Virginia) and EU West (Ireland), account for more than three-quarters of all EC2 usage as measured by Netcraft. Sydney, the newest AWS region, now accounts for just under 1% of all measured web-facing computers using AWS, having almost tripled in size in the past four months. In total, more than 156k instances power at least one hostname on 3M domains across the internet.
Launched in 2011, the GovCloud (US) region is specifically intended for more sensitive applications that require additional security and compliance with US regulations. As of May 2013, Netcraft found just 27 web-facing computers within the government cloud, some of which power www.grdregistry.org and www.govdashboard.com. Given its intended role, it would not be surprising if a large proportion of the computers used in the region are not web-facing.
Metric (EC2 Total) February 2013 March 2013 April 2013 May 2013 Growth (4 month) Web-facing Computers/Instances 141,960 145,648 152,041 156,225 10% IP Addresses 144,625 148,837 155,712 160,884 11.2% Domains 2,788,685 2,810,906 2,996,147 3,061,178 9.8% Hostnames 9,489,496 9,938,480 10,649,545 10,925,661 15.1%
Many uses of EC2 such as batch data-processing will not be directly measurably over the internet: Netcraft measures publicly visible computers with corresponding DNS entries and which respond to HTTP requests. Netcraft's Web Server Survey is run at Amazon from the Northern Virginia region, so the region may be over-reported due to services like latency based multi region routing which provide differing responses depending on topological location.
Geographic distribution of computers per EC2 region in May 2013
Data Centre (EC2 - Web Facing Computers) February 2013 March 2013 April 2013 May 2013 Growth (4 month) Asia Pacific (Singapore) 6,576 6,805 6,998 7,290 10.9% Asia Pacific (Sydney) 499 739 1,129 1,427 186% Asia Pacific (Tokyo) 7,342 7,595 8,065 8,601 17.1% EU West (Ireland) 23,778 24,635 25,326 25,942 9.1% South America (Sao Paulo) 2,115 2,263 2,396 2,655 25.6% US East (Northern Virginia) 87,094 88,543 92,426 93,537 7.4% US West (Northern California) 9,325 9,478 9,715 9,695 4% US West (Oregon) 5,217 5,573 5,965 7,051 35.2% GovCloud (Oregon) 14 17 21 27 92.9%
S3 - Simple Storage Service
S3 provides an online file storage service which can be managed programmatically via Amazon's API. Files are logically grouped into containers called buckets which can be made public and accessible over HTTP but default to being private. As with EC2, Netcraft cannot track private use of S3 but is able to survey websites using S3 publicly to serve static files and even entire websites.
Metric (S3 Total) February 2013 March 2013 April 2013 May 2013 Growth (4 month) Domains 41,782 42,561 45,721 48,636 16.4% Hostnames 124,454 127,370 132,962 138,588 11.4%
In May 2013, a total of 139k hostnames were found to be hosted directly on S3, either using a subdomain of s3.amazonaws.com or using a custom CNAME pointing to S3. Of these, 24.7k hostnames, or over 18.5k domains, point to an S3 bucket configured to serve an entire website, as does mediahackers.org. Many more websites are not hosted entirely on S3, but make use of the service to serve static files such as images, stylesheets, or file downloads.
One of the most widely referenced S3 hostnames is used for twitter badges bucket, which was once a common method to display twitter icons on a third-party website. Tumblr, a popular blogging platform recently acquired by Yahoo!, also makes use of S3 to host static media.
CloudFront is a Content Delivery Network which can be used to serve both dynamic and static content from 28 edge locations which are topologically closer to a site's visitors. Caching content reduces the bandwidth and performance requirements on the website's own servers and, by being topologically close to visitors, the latency associated with each HTTP request can be improved.
In the May 2013 survey, more than 63k hostnames were served via CloudFront, more than 60% of which point to an S3 bucket. Amazon uses CloudFront on its own websites, including imdb.com, and also uses it for serving images on Amazon.com. Other than Amazon itself, CloudFront users include: the Toronto Star, a Canadian newspaper, and Pirifrom, the makers of utility program CCleaner, are two of the most visited sites using CloudFront amongst users of the Netcraft Toolbar.
Metric (CloudFront Total) February 2013 March 2013 April 2013 May 2013 Growth (4 month) Domains 22,920 24,079 25,264 26,221 14.4% Hostnames 55,578 57,817 60,475 63,203 13.7%
The number of CloudFront-dedicated IP addresses and computers cannot be easily measured as different results are obtained depending on the location of the request.
Route 53, is a managed Domain Name System (DNS) hosting service. Route 53, named for the TCP and UDP port used for the protocol, hosts DNS records which map from human-readable hostnames to IP addresses. Integrated with the rest of AWS, it allows programmatic access to change DNS records in response to changes elsewhere in a customer's infrastructure. As with CloudFront, Amazon have servers providing this service in edge locations outside of its 9 EC2 regions; Route 53 is available from 28 separate locations.
Metric (Route 53 Total) February 2013 March 2013 April 2013 May 2013 Growth (4 month) Domains 136,698 146,635 161,619 169,111 23.7% Hostnames 3,493,986 3,662,195 3,831,910 4,068,053 16.4%
Over the past four months there has been a steady growth in the number of websites using Route 53 to host their DNS records: it now serves DNS records for 169k domains. Busy sites making use of this service include pinterest.com, a social photo-sharing website which is a heavy user of Amazon's infrastructure; MediaFire, a file uploading and sharing service; and ow.ly a URL shortener.
Heroku is Platform as a Service (PaaS) provider owned by Salesforce. Whilst not operated by Amazon, it makes heavy use of AWS services, especially EC2. Heroku provides an abstracted managed environment for web developers to deploy applications in a number of different languages. In May 2013, Heroku was serving 70K domains directly (not behind a CDN) across 4,786 computers.
Metric (Heroku total) April 2013 May 2013 Growth (2 month) Computers 4,293 4,786 11.5% IP Addresses 4,408 4,972 12.8% Domains 65,821 69,781 6% Hostnames 1,094,578 1,102,663 0.7%
Heroku, as demonstrated in the results from Netcraft's survey, has been available almost exclusively from the Northern Virginia EC2 region. In April, Heroku announced availability of its service in Europe from the AWS EU West region based in Ireland. Only a limited number of Heroku customers have had access to this region during a private beta phase which explains the currently low uptake: only 1% of the computers attributed to Heroku were in the region.
IP Addresses April 2013 May 2013 US East (Northern Virginia) 4,374 4,915 EU West (Ireland) 33 56
Netcraft provides information on the Internet infrastructure, including the hosting industry, and web content technologies. For information on the cloud computing industry including Microsoft Azure, Rackspace Cloud, and Google App Engine, please contact firstname.lastname@example.org.
In the May 2013 survey we received responses from 672,837,096 sites, which is 23.8M more than last month.
Apache had the largest growth this month, gaining 28.3M websites and increasing its market share by 2.41 percentage points to 53.4%. The majority of this growth was attributable to Apache Traffic Server (ATS), which gained 28M websites and increased its market share from 0.03% to 4.2%. Nearly all of the Apache Traffic Server growth occurred at Go Daddy — 75% of websites hosted by Go Daddy now use ATS and Go Daddy now hosts 99% of all sites using this server software.
Originally created as a commercial product by Inktomi in 1997, Apache Traffic Server is an extensible multi-threaded event-driven caching proxy server which is claimed to scale well on modern multi-core systems. Yahoo! acquired Inktomi in 2005, and in November 2009, the project was donated to the Apache Software Foundation.
The vast majority of the ATS served websites at Go Daddy were previously served by Microsoft IIS, resulting in the rather noticeable loss of 3.26 percentage points of market share. Microsoft IIS's market share is now 16.7%. Despite the loss at Go Daddy it gained more new sites than any competitor this month, with 43% of all new websites being served on Microsoft IIS, while accounting for only 30% of expired websites (this includes inactive blogs, as well as sites which no longer exist).
nginx reached a new milestone this month: it is now used by more than 100M websites, and within the Million Busiest Websites has overtaken Microsoft IIS to take second place with a market share of 13.5%. Overall, nginx's market share now stands at 15.5%, just 1.2 percentage points behind Microsoft, helped by a growth of 8.3M sites this month.
The latest stable version, nginx 1.4.0, was released last week, integrating OCSP stapling and experimental SPDY draft 2 support. nginx is used extensively by the WordPress.com blog hosting service, whose owners – Automattic – sponsored development of the ngx_http_spdy_module. Development of OCSP stapling support was sponsored by Comodo, DigiCert, and GlobalSign.
Developer April 2013 Percent May 2013 Percent Change Apache 331,112,893 51.01% 359,441,468 53.42% 2.41 Microsoft 129,516,421 19.95% 112,303,412 16.69% -3.26 nginx 96,115,847 14.81% 104,411,087 15.52% 0.71 22,707,568 3.50% 23,029,260 3.42% -0.08
In the April 2013 survey we received responses from 649,072,682 sites, 17.6M more than last month.
This month, market leader Apache lost 9.9M sites, or 3 percentage points of market share. A major contributor to this loss was the movement of a large affiliate referral network consisting of around 8M sites now being served by nginx. Apache is now used by just over 51% of websites, which is still substantially more than its closest competitor Microsoft IIS. IIS gained 1.95 percentage points of market share this month (an increase of 15.8M hostnames) bringing its market share to almost 20%. Meanwhile, nginx saw an overall growth of 10.6M sites this month, with the largest nginx hosting company, Hetzner Online AG, contributing an additional 1.6M sites.
In terms of active sites the survey was less volatile. Apache still experienced an overall loss, however much smaller at just 288k active sites. The biggest increase came from nginx, and was unrelated to their large hostname gain described earlier, with Peer1 Networks gaining 1.5M nginx active sites.
North Korea's drew the world's attention to its web presence by accusing the United States and its allies of "intensive and persistent virus attacks" on servers operated by the North Korean regime. The Korean Central News Agency's press release goes on to assert that:
"It is nobody's secret that the U.S. and south Korean puppet regime are massively bolstering up cyber forces in a bid to intensify the subversive activities and sabotages against the DPRK [Democratic People's Republic of Korea]."
There is only a very small number of North Korean sites accessible from outside of the country; however, these sites do make use of several modern and popular web technologies from around the globe. The Rodong Sinmun newspaper's site uses PHP and CentOS 5, and hosts an HTTPS service with an expired self-signed certificate. More controversially, The Korean Central News Agency's official website uses Java, Flash and jQuery and is hosted using Apache 2.2.3 on a server running Red Hat Enterprise Linux 5, a commercial Linux distribution which is owned, distributed and supported by American multinational Red Hat, Inc. Red Hat Enterprise Linux is subject to U.S. export controls, which specifically prohibit its use in North Korea. As a result, this installation is likely unlicensed and so may not receive security updates.
Meanwhile in South Korea, the Government of Korea, an SSL certificate authority (CA) trusted by Microsoft has revoked the last of more than 100 unusual SSL certificates each of which could have allowed its owner to act as a trusted CA. With the ability conferred by the cA bit being set in the Basic Constraints extension, a forged certificate signed using the mis-issued certificate could be trusted for any site by users of some SSL implementations. Any such certificate could be used to perform man-in-the-middle attacks on users of third-party websites in order to view the contents of any intercepted encrypted traffic. There is an additional property which is usually required for a certificate to be considered a valid intermediate — ‘Certificate Signing’ should be set as a permissible Key Usage — but some implementations may ignore this extra requirement. None of the Korean certificates found had the necessary flags set in this additional extension, so most implementations would not trust such forged certificates.
The certificates found appear to have been issued to South Korean academic institutions without the intention of them being able to sign additional certificates. These certificates have been in the Netcraft SSL Server Survey for some time but no longer pose a risk: all of the certificates concerned have either been revoked or have expired. The most recent revocation was on January 31st 2013 for a certificate issued in late 2011, showing it was at risk of misuse for more than a year.
Developer March 2013 Percent April 2013 Percent Change Apache 341,021,574 54.00% 331,112,893 51.01% -2.99 Microsoft 113,712,293 18.01% 129,516,421 19.95% 1.95 nginx 85,467,555 13.53% 96,115,847 14.81% 1.27 22,605,646 3.58% 22,707,568 3.50% -0.08
In the March 2013 survey we received responses from 631,521,198 sites.
Microsoft showed a noticeable gain of 9M sites this month, increasing its market share by 1.42 percentage points to 18.01%. Much of this growth was seen at Go Daddy, which alone gained 2.6M sites powered by Microsoft web servers. Microsoft also fared well amongst the top million busiest sites, where its market share grew by 0.74 percentage points to 13.60%, increasing its narrow lead over its closest rival, nginx.
More than 130,000 of the sites in Netcraft's survey are hosted on Microsoft's Windows Azure cloud platform, but not all are running Microsoft web servers: Since Microsoft Open Technologies launched its VM Depot preview in January, more than 100 ready-to-use images have been added to the community-driven catalogue of virtual machines. This makes it easier for customers to deploy Linux-based images running preconfigured applications and frameworks such as WordPress, Joomla!, Drupal, Django and MongoDB. The majority of these images are based on Ubuntu Linux, and many of them use Apache and PHP to serve their content.
nginx also saw a reasonable gain in market share this month, with an additional 4.4M sites taking its share up by 0.68 percentage points to 13.53%. The most recent development release of nginx (1.3.13) introduced support for proxying WebSocket connections – an HTML5 technology which provides full-duplex communications between a browser and a web application over a single TCP connection. WebSockets are supported by all modern desktop browser software, for which the protocol specification defines two URI schemes:
ws:for unencrypted connections, and
wss:for secure ones. The development of WebSocket support in nginx was sponsored by CloudBees and Apcera, who will both be making use of the new feature in their own services.
nginx performed less well amongst the top million sites, where it had looked set to overtake Microsoft this month. Instead, a net loss of 910 nginx sites resulted in its share falling to 12.72%.
Developer February 2013 Percent March 2013 Percent Change Apache 344,915,105 54.68% 341,021,574 54.00% -0.68 Microsoft 104,647,425 16.59% 113,712,293 18.01% 1.42 nginx 81,074,694 12.85% 85,467,555 13.53% 0.68 22,717,984 3.60% 22,605,646 3.58% -0.02