May 2015 Web Server Survey

In the May 2015 survey we received responses from 857,927,160 sites and 5,281,889 computers. The number of sites detected increased by nearly 9M this month after two consecutive months of losses. The number of web facing computers also increased by 54k.

Microsoft had the largest growth of web sites, gaining more than one percentage point of market share. Apache, the current leader with a market share of 39.26%, remained stable. Nginx, however, experienced the largest loss amongst major web server vendors and consequently saw a small loss in market share.

Nginx is performing well within the million busiest sites, and when counting the number of web facing computers using the web server — being responsible for the largest growth in each category. Nginx gained just over 2k of the million busiest sites, giving it a market share of 21.64%. Almost 16% of nginx's market share (or 3.4% of the top million sites) in the top million busiest sites is due to CloudFlare. While nginx is used to serve requests at CloudFlare, they may be proxied to backend servers running other web servers. Nginx contributed over half of the total net gain of web facing computers this month, with an increase of 27,500 computers.

DigitalOcean recently became the second largest hosting company by the number of web facing computers, overtaking OVH. Apache, with 48% of publicly visible DigitalOcean computers, is the leading web server, closely followed by nginx with 45.5%.

As the remaining IPv4 address space dwindles, companies are more often resorting to the emerging transfer market to acquire additional IPv4 ranges — for example, DigitalOcean recently appeared as one of the leading inbound receivers of IPv4 addresses from transfers within the RIPE NCC Service Region. With prices expected to increase as IPv4 becomes a more valuable asset, the adoption of IPv6 could accelerate. Netcraft currently finds 740k IPv6 addresses, an increase of 25% compared to the same time last year.

Total number of websites

Web server market share

DeveloperApril 2015PercentMay 2015PercentChange
Apache333,285,74139.25%336,813,95939.26%0.00
Microsoft236,288,84327.83%247,784,66828.88%1.05
nginx126,274,77814.87%123,697,64514.42%-0.45
Google20,051,4332.36%20,103,0682.34%-0.02
Continue reading

Counting SSL certificates

The SSL/TLS protocol — used to protect sensitive communication across the internet — combines encryption with authentication, providing a private connection to the intended recipient. To achieve this, SSL certificates bind together a cryptographic key and a domain name, and are digitally-signed by a trusted certificate authority (CA). Commercial CAs compete to sell certificates to the general public and account for the bulk of the SSL certificates seen on the internet.

Netcraft's SSL Server Survey has been running since 1996 and has tracked the evolution of this marketplace from its inception — there are now more than one thousand times more certificates on the web now than in 1996. As CAs issue certificates, and most charge (or not charge) accordingly, the number of certificates issued becomes the natural unit of measurement. Our survey therefore counts valid, trusted SSL certificates used on public-facing web servers, counting each certificate once, even if used on multiple websites.

certs

Two types of certificates make the distinction between counting sites and certificates most apparent: multi-domain certificates and wildcard certificates. These two types now account for almost a quarter of all certificates found.

  • Multi-domain certificates (or UCC certificates) use the Subject Alternative Name extension to specify additional hostnames for which this certificate is valid — CloudFlare uses this technique heavily, having dozens of unrelated sites share the same certificate.
  • Wildcard certificates are valid for all possible subdomains of a domain, for example *.netcraft.com would be valid for www.netcraft.com, host-a.netcraft.com, host-b.netcraft.com, etc. Our methodology counts a wildcard certificate once, no matter the number of sites for which it is valid.

Netcraft also counts certificates used by subdomains. For example, if foo.example.com, bar.example.com and baz.example.com are all using different SSL certificates, Netcraft will count all three certificates that have been issued.

Although the global SSL ecosystem is competitive, it is dominated by a handful of major CAs — three certificate authorities (Symantec, Comodo and GoDaddy) account for three-quarters of all issued SSL certificates on public-facing web servers. The top spot has been held by Symantec (or VeriSign before it was purchased by Symantec) ever since the survey began, with it currently accounting for just under a third of all certificates. To illustrate the effect of differing methodologies, amongst the million busiest sites Symantec issued 44% of the valid, trusted certificates in use — significantly more than its overall market share.

However, nothing ever stays still forever — Let's Encrypt could shake up the market for SSL certificates later on this year by offering free certificates with a simplified installation process. Whilst free certificates and automated tools are nothing new, the open approach and the backing of Mozilla, IdenTrust, the EFF, and Akamai could change the SSL ecosystem forever.

Beyond counting certificate numbers, Netcraft's SSL Survey also tracks the list and reseller prices of the most popular certificate authorities. This provides another useful market share metric, as it allows us to estimate the total monthly and annual revenue of each certificate authority attributable to public SSL issuance.

As each type of certificate — multi-domain, wildcard, or Extended Validation for example — is available at a distinct price point, the estimated revenue of a CA can vary significantly, despite initially appearing similarly sized by the total number of certificates. For example, GlobalSign comes in third-place when considering its estimated annual revenue (by list price) in 2014, despite accounting for approximately 6% of all currently valid publicly-visible SSL certificates.

For additional information or details on how to purchase Netcraft’s SSL Server Survey please contact us at sales@netcraft.com or visit our web site.

Most Reliable Hosting Company Sites in April 2015

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 Datapipe Linux 0:00:00 0.000 0.094 0.012 0.025 0.033
2 GoDaddy.com Inc Linux 0:00:00 0.004 0.106 0.009 0.170 0.170
3 ServerStack Linux 0:00:00 0.009 0.076 0.069 0.136 0.136
4 Bigstep Linux 0:00:00 0.013 0.117 0.064 0.130 0.130
5 Netcetera Windows Server 2012 0:00:00 0.017 0.061 0.084 0.168 0.169
6 www.dinahosting.com Linux 0:00:00 0.017 0.182 0.088 0.177 0.177
7 Kattare Internet Services Linux 0:00:00 0.022 0.181 0.119 0.268 0.576
8 EveryCity SmartOS 0:00:00 0.026 0.088 0.065 0.131 0.131
9 Codero Citrix Netscaler 0:00:00 0.026 0.175 0.096 0.200 0.396
10 Hyve Managed Hosting Linux 0:00:00 0.030 0.229 0.066 0.130 0.132

See full table

Datapipe had the most reliable hosting company website in April, responding successfully to all of Netcraft's requests; Datapipe has now featured in the top ten for eight consecutive months, and has maintained 100% uptime over the last nine years. In April, Datapipe announced that it had become one of the first AWS Managed Service Provider Partners, certifying that Datapipe "[offers] proactive monitoring, automation, and management of their customer's AWS environment".

With just a single failed request, GoDaddy came in second place. GoDaddy, the world's largest registrar, recently listed on the New York Stock Exchange, bought the GDDY.com domain name to match its stock ticker symbol, and will announce its first quarter 2015 results on 12th May.

ServerStack came in third place in April, narrowly missing second place with just a single failed request separating it from GoDaddy. ServerStack provides managed hosting services to enterprises, targeting companies spending more than $10,000 per month on hosting. Its data centres are based in San Jose, New Jersey and Amsterdam.

Once again, Linux was the most popular operating system for hosting company sites, powering 7 of the top 10 websites. Windows Server 2012, SmartOS and Citrix Netscaler all appear once each.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

DigitalOcean becomes the second largest hosting company in the world

DigitalOcean has grown to become the second-largest hosting company in the world in terms of web-facing computers, and shows no signs of slowing down.

The virtual private server provider has shown phenomenal growth over the past two-and-a-half years. First seen in our December 2012 survey, DigitalOcean today hosts more than 163,000 web-facing computers, according to Netcraft's May 2015 Hosting Provider Server Count. This gives it a small lead over French company OVH, which has been pushed down into third place.

Amazing growth at DigitalOcean

Amazing growth at DigitalOcean

DigitalOcean's only remaining challenge will be to usurp Amazon Web Services, which has been the largest hosting company since September 2012. However, it could be quite some time until we see DigitalOcean threatening to gain this ultimate victory: Although DigitalOcean started growing at a faster rate than Amazon towards the end of 2013, Amazon still has more than twice as many web-facing computers than DigitalOcean today.

Nonetheless, DigitalOcean seems committed to growing as fast as it can. Since October 2014, when we reported that DigitalOcean had become the fourth largest hosting company, DigitalOcean has introduced several new features to attract developers to its platform. Its metadata service enables Droplets (virtual private servers) to query information about themselves and bootstrap new servers, and a new DigitalOcean DNS service brought more scalability and reliability to creating and resolving DNS entries, allowing near-instantaneous propagation of domain names.

Other companies are also helping to fuel growth at DigitalOcean. Mesosphere created an automated provisioning tool which lets customers use DigitalOcean's resources to create self-healing environments that offer fault tolerance and scalability with minimal configuration. Mesosphere's API makes it possible to manage thousands of Droplets as if they were a single computer, and with DigitalOcean's low pricing models and SSD-only storage, it's understandable how this arrangement can appeal to particularly power-hungry developers.

In January, DigitalOcean introduced its first non-Linux operating system, FreeBSD. Although less commonly used these days, FreeBSD has garnered a reputation for reliability and it was not unusual to see web-facing FreeBSD servers with literally years of uptime in the past. In April, DigitalOcean launched the second version of its API, which lets developers programmatically control their Droplets and resources within the DigitalOcean cloud by sending simple HTTP requests.

DigitalOcean added a new Frankfurt region in April 2015.

DigitalOcean added a new Frankfurt region in April 2015.

More recently, DigitalOcean introduced a new European hosting region in Frankfurt, Germany. This is placed on the German Commercial Internet Exchange (DE-CIX), which is the largest internet exchange point worldwide by peak traffic, allowing Droplets hosted in this region to offer good connectivity to neighbouring countries. (An earlier announcement of an underwater Atlantis datacenter sadly turned out to be an April Fool's joke, despite the obvious benefits of free cooling).

Even so, Amazon still clearly dwarfs DigitalOcean in terms of variety of features and value-added services. Notably, Amazon offers a larger variety of operating systems on its EC2 cloud instances (including Microsoft Windows), and its global infrastructure is spread much wider. For example, EC2 instances can be hosted in America, Ireland, Germany, Singapore, Japan, Australia, Brazil, China or even within an isolated GloudGov US region, which allows US government agencies to move sensitive workloads into the cloud whilst fulfilling specific regulatory and compliance requirements. As well as these EC2 regions, Amazon also offers additional AWS Edge Locations to be used by its CloudFront content delivery network and its Route 53 DNS service.

Yet, as well as its low pricing, part of the appeal of using DigitalOcean could lie within its relative simplicity compared with Amazon's bewilderingly vast array of AWS services (AppStream, CloudFormation, ElastiCache, Glacier, Kinesis, Cognito, Simple Workflow Service, SimpleDB, SQS and Data Pipeline to name but a few). Signing up and provisioning a new Droplet on DigitalOcean is remarkably quick and easy, and likely fulfils the needs of many users. DigitalOcean's consistent and strong growth serves as testament to this, and will make the next year very interesting for the two at the top.

Instagram forgets to renew its SSL certificate

Instagram's SSL certificate expired at midday GMT on Thursday 30th April 2015 and was not replaced for more than an hour, leaving visitors unable to access the site without seeing browser warnings.

Browser warnings caused by Instagram's expired SSL certificate.

Browser warnings caused by Instagram's expired SSL certificate.

The expired DigiCert-issued certificate that was being served from https://instagram.com/ has now been replaced with a different certificate, valid until 15th October 2015.

Users who ignore the warnings from their browser could be at risk of man-in-the-middle attacks, where a correctly-positioned attacker can surreptitiously steal usernames, passwords and session cookies without the victim's knowledge.

Although the HTTP version of the site redirects to HTTPS, instagram.com does not currently make use of HTTP Strict Transport Security — an HTTP header that permits a site to specify that future visits must be over HTTPS. As a result, customers can bypass the warning message, placing them at risk of man-in-the-middle attacks.

If HSTS had been in use, visitors would correctly not be able to bypass the error message, protecting them from man-in-the-middle attacks, but leaving them without the ability to connect to instagram.com. As HSTS does not protect the user on their first visit, website owners can request to have their HSTS rules embedded into the browser via Chrome's preload list.

instagram-cert-error

The SSL error message in Google Chrome can be bypassed for instagram.com (which does not use HSTS).

paypal-cert-error

In simulating an attack on www.paypal.com (which does use HSTS), Chrome's SSL error message cannot be bypassed.

instagram.com is the 310th most popular website amongst users of the Netcraft Toolbar. The Instagram app does not appear to be affected, as it makes use of a different server at i.instagram.com, which uses a valid certificate.

The SSL certificate used by instragram.com expired at midday UTC

The SSL certificate used by instagram.com expired at midday UTC

Hostinger hosts over 90% of all Steam phishing sites

Netcraft blocked more than 1,400 Steam phishing URLs last month, spread across 331 different websites. Surprisingly, more than 90% of these sites were hosted by just one company: Hostinger.

With more than 125 million active accounts, Steam continues to make an attractive target for fraudsters. The number of phishing attacks targeting Steam rose significantly last month, even though the fraudsters behind these attacks have had to change their tactics a few times. Last year, a popular ruse was to use Steam's own chat client to trick victims into visiting look-alike domain names similar to the genuine steamcommunity.com. This modus operandi continued into 2015, but became less effective after Steam started to remove suspicious links from chat messages.

Consequently, many Steam phishers have abandoned the idea of registering their own look-alike domains (only two were blocked last month), and are instead using subdomains provided by free hosting services such as Hostinger. These allow the fraudsters to host Steam phishing sites with addresses like steamcommuniity.hol.es, steampoweredssuport.esy.es and steamcomcoomity.16mb.com – not quite as convincing as the hostnames used in previous attacks, although the deliberate misspellings are similar.

A Steam phishing site hosted at steamcomcoomity.16mb.com

A Steam phishing site hosted by Hostinger at steamcomcoomity.16mb.com

Lithuania-based Hostinger provides many different second-level domains under which its customers can host a website, and the most common ones used in these attacks were esy.es, besaba.com, 16mb.com, wc.lt, hol.es and pe.hu.

Hostinger displays this content on each of its free hosting  domains. Hostinger covers its costs by offering paid upgrades for those who need  more resources.

Hostinger displays this content on each of its free hosting domains. Hostinger covers its costs by offering paid upgrades for those who need more resources.

Free hosting providers are an obvious choice for fraudsters who wish to carry out phishing attacks without leaving a financial trail. Hostinger's offerings look particularly conducive for phishing, as they do not display ads on their customers' sites, and they provide support for PHP (nearly all phishing kits are written in PHP).

Nonetheless, the incredible popularity of Hostinger within the Steam phishing arena is rather unusual. While Hostinger was used to host over 90% of all Steam phishing URLs, it hosted only 0.6% of all other phishing attacks that were blocked during March.

This preference of using Hostinger could suggest that the fraudsters behind most of these Steam phishing attacks are working together or copying each others' methodologies. In addition, there are examples of phishing sites that have remained up for long periods of time, which makes it an attractive hosting location for phishers. The hostname steamcomcoomity.16mb.com (shown in the earlier screenshot) has been serving a Steam phishing site from Hostinger's infrastructure since last year and is still serving it at the time of writing.

Netcraft provides a Phishing Alerts service for hosting providers and domain registrars who are unwittingly providing facilities for phishing. Brand owners can also use Netcraft's Takedown service to identify phishing attacks against them and get fraudulent sites shut down.