Most Reliable Hosting Company Sites in September 2019

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 New York Internet (NYI) FreeBSD 0:00:00 0.000 0.521 0.053 0.107 0.107
2 EveryCity SmartOS 0:00:00 0.000 0.224 0.074 0.148 0.148
3 ServerStack Linux 0:00:00 0.000 0.214 0.084 0.167 0.167
4 Hyve Managed Hosting Linux 0:00:00 0.000 0.148 0.084 0.168 0.168
5 Pair Networks Linux 0:00:00 0.000 0.318 0.093 0.186 0.186
6 www.dinahosting.com Linux 0:00:00 0.000 0.271 0.095 0.190 0.190
7 www.choopa.com Linux 0:00:00 0.005 0.239 0.005 0.024 0.024
8 Rackspace Linux 0:00:00 0.005 2.108 0.005 0.014 0.014
9 CWCS Managed Hosting Linux 0:00:00 0.005 0.292 0.080 0.163 0.163
10 Hivelocity Linux 0:00:00 0.005 0.243 0.108 0.214 0.214

See full table

New York Internet (NYI) had the most reliable hosting company website in September 2019, responding to all of Netcraft's requests. This is the fourth time NYI has appeared in the top 10 in 2019. NYI offers bare metal, cloud and colocation services in its US data centers, and recently expanded into the Chicago market with the acquisition of a new data center.

The next five hosting company sites also responded to all of our requests, and are separated in the ranking by their average connection time. Second-placed EveryCity has had 99.998% uptime since Netcraft started monitoring 5 years ago and is the only site Netcraft is monitoring that is powered by SmartOS.

Third place was close with ServerStack claiming the position with an average connection time of 83.6ms—just half a millisecond faster than Hyve Managed Hosting in fourth place. ServerStack has now appeared in the top 10 for five consecutive months.

In sixth place, Choopa.com failed to respond to a single request but had the fastest average connection time of 5.2ms. This was closely followed by Rackspace, with an average connection time of 5.5ms.

FreeBSD and SmartOS appear in first and second place in September, with New York Internet (NYI) and EveryCity. The eight hosting company sites that complete the top 10 are powered by Linux.

Netcraft measures and makes available the response times of around twenty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

September 2019 Web Server Survey

In the September 2019 survey we received responses from 1,291,178,101 sites across 241,131,705 unique domain names and 9,068,313 web-facing computers. This reflects a gain of 19 million sites, 1.69 million domains and 119,000 computers.

All major vendors gained sites this month. The largest gain was for nginx with an increase of 20.6 million sites, followed by Microsoft (+2.9 million), Google (+2.1 million) and Apache (+462,000). This extends nginx's lead as the largest web server vendor by number of sites; it gained 1.12 percentage points taking it to a 32.7% market share. nginx also showed the largest gains in number of unique domains and web-facing computers.

The largest gain within the top million sites this month was by LiteSpeed, which also saw gains in hostnames, domains, and web-facing computers. The September survey saw 1,422 more sites within the top million using this light-weight Apache alternative, an 8.0% increase. This was accompanied by increases of 480,000 sites (+2.6%), 326,000 domains (+9.4%) and 1,665 web-facing computers (+8.1%).

There are losses in market share for both Apache and nginx as the largest server vendors by number of active sites. Apache lost 22,000 active sites while nginx gained 915,000; due to large gains elsewhere this amounted to Apache losing 0.94pp and nginx losing 0.11pp. Google gained 800,000 active sites and 0.16pp of market share to retake third place from Cloudflare; Cloudflare gained 591,000 sites. The largest increase of active sites was in sites running openresty with an increase of 1.04 million.

Web Server Releases

Apache 2.4.41 was released on August 14th bringing several security fixes. This is the first release of Apache 2.4 since 2.4.39 was released on April 1st.

OpenLiteSpeed released a major new feature in version 1.6.0 on September 10th adding support for QUIC and HTTP/3 as well as a new one-click build tool and support for more platforms.

Both OpenResty and Tengine released versions incorporating the nginx patches that fix the HTTP/2 related security issues discussed in last month's blog. OpenResty version 1.15.8.2 was released on September 8th and Tengine 2.3.2 released on August 20th.

Total number of websites

Web server market share

DeveloperAugust 2019PercentSeptember 2019PercentChange
nginx401,454,02931.56%422,048,24332.69%1.12
Apache374,277,24329.43%374,739,32129.02%-0.40
Microsoft187,109,42314.71%189,991,31214.71%0.00
Google30,969,2592.43%33,058,9302.56%0.13
Continue reading

Most Reliable Hosting Company Sites in August 2019

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 GoDaddy.com Inc Linux 0:00:00 0.000 0.405 0.004 0.022 0.023
2 CWCS Managed Hosting Linux 0:00:00 0.000 0.291 0.079 0.162 0.162
3 Hyve Managed Hosting Linux 0:00:00 0.000 0.157 0.084 0.168 0.168
4 Pair Networks Linux 0:00:00 0.000 0.324 0.092 0.185 0.185
5 www.dinahosting.com Linux 0:00:00 0.000 0.268 0.094 0.189 0.189
6 Aruba Windows Server 2012 0:00:00 0.000 0.275 0.105 0.213 0.213
7 Rackspace Linux 0:00:00 0.005 1.154 0.004 0.011 0.012
8 New York Internet (NYI) FreeBSD 0:00:00 0.005 0.537 0.054 0.107 0.107
9 ServerStack Linux 0:00:00 0.005 0.221 0.083 0.166 0.166
10 Multacom Linux 0:00:00 0.005 0.255 0.115 0.232 0.232

See full table

GoDaddy had the most reliable hosting company site in August 2019, with no failed requests and the fastest average connection time of 4ms. This is the second time in three months that GoDaddy has had the most reliable hosting company site. GoDaddy provides a wide range of hosting and domain registration services with 9 global data centres.

The top six hosting company sites each responded to all of Netcraft's requests and are separated by their average connection time. CWCS appears in second place in August and has been in the top 10 for three consecutive months. Italy-based hosting company Aruba appeared in sixth place. Aruba uses renewable energy, including hydro-electric and solar energy, to provide power to its green data centres, ensuring these data centres are energy efficient with zero net carbon impact.

Despite the decline of FreeBSD, it continues to appear in the top 10, as it powers the hosting company site for New York Internet (NYI). Windows Server 2012 also makes an appearance, but Linux continues to dominate, powering eight of the top 10 sites.

Netcraft measures and makes available the response times of around twenty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

Uniqlo and The Guardian among thousands of sites loading malicious code from S3

Updated 05/09/2019: Fast Retailing Co has stated that the credit card fields were contained within an iframe, which meant they would not be collected by this generic skimmer. However, the remainder of the personal information provided by customers would have still been vulnerable if at least one non-credit card field happened to match a regular expression designed to find credit card numbers. Fast Retailing has stated it has “verified its order history database records for last several years and confirmed that there are no inputs in existing orders matching a regular expression designed to find credit card numbers in any non-credit card fields. While the malicious code would have been executed by visitors, based on the information available to Fast Retailing it is unlikely that customers who successfully placed an order would have had their personal data stolen”.

Uniqlo's website was infected with a shopping site skimmer for more than a week in May this year, following the addition of malicious JavaScript. The injected code was designed to silently 'skim' part of the checkout form and send a copy of the customer's details to the criminals under certain conditions. In this case, the attack was not successful as the credit card details were not vulnerable — Uniqlo's Australian site uses an iframe-based credit card form which means it was isolated from the malicious JavaScript.

Thousands more sites have also been compromised in recent months via the same underlying vulnerability that allowed criminals to alter the behaviour of the Uniqlo website — unsecured Amazon S3 buckets. The criminals took a shotgun approach to compromising as many files as possible. They got lucky with a bucket containing JavaScript files used on Uniqlo's site, one of the most visited shopping sites on the internet.

Skimmer on Uniqlo's website

We detected that Uniqlo's Australian online shop was running malicious JavaScript on 18th May 2019. While the skimmer was active, a copy of any data that was entered during the checkout process on Uniqlo’s Australian site would have been silently sent to a dropsite operated by criminals if it matched a regular expression designed to find credit card numbers.

Personal data entered into Uniqlo Australia's checkout page would have been stolen
Personal data entered into Uniqlo Australia's checkout page may have been stolen

E-commerce is responsible for nearly 10% of Uniqlo Japan's sales and Uniqlo's parent company Fast Retailing Co is one of the world's largest and most successful retailers, worth $62 billion. Uniqlo is the most-visited online shop on which we have found a skimmer to date. This is the second attack to which Uniqlo has fallen victim in recent times; in May it was announced 460,000 users of the shopping site may have had their details stolen following a credential stuffing attack.

The criminals altered the website's behaviour by adding obfuscated JavaScript code to the all of the resources Uniqlo hosts within its S3 bucket, hoping that at least one would be loaded by the website. By deobfuscating the code, we can reveal the data it captured and to where the stolen data would have been transmitted.

The code captured every input on the page accessible to the scriptThe code captured every input on the page accessible to the script

The code was designed to capture all of the data entered by customers into the checkout form. However, customers would not have had their credit card details stolen by the skimmer, as this part of the checkout form is loaded in an isolated iframe or is processed externally via Paypal. If the injected code did not find any other customer details where at least one field matched a regular expression designed to find credit card numbers, none of the data would be stolen.

Unlike the skimming code used in the attacks against Cleor and British Airways, this JavaScript code is very generic and is designed to function on multiple websites without modification. It harvests all form fields (by looking for input, select, and textarea elements) whether or not they are part of a specific checkout form.

Stolen credentials were sent to cdn-c.com The captured data is transmitted to cdn-c[.]com The captured data is transmitted to cdn-c[.]com

At the time we discovered the attack, the Last-Modified header from the infected JavaScript files within the S3 bucket suggested that they had been harbouring malicious code since at least 13th May.

Uniqlo Australia was Uniqlo's only online shop that appeared to be affected by this attack. We alerted Uniqlo to the compromise and the malicious code was removed from the affected files on 21st May.

Unsecured S3 buckets

This type of attack — in which criminals target less-secure parts of an organisation's supply network — are known as supply chain attacks. This is not the first time supply chain attacks have been used to insert malicious JavaScript into websites. However, we have not identified the exploitation of unsecured S3 buckets to inject code intended to steal personal data entered into a website until recently.

Amazon provides customers with the ability to configure the permissions on their S3 storage with Access Control Lists (ACLs). Using ACLs, users can specify who may view, edit, delete and upload files. In Uniqlo's case, the ACL was misconfigured, allowing any user to modify any of the files within the bucket:

    {
        "Grantee": {
            "Type": "Group",
            "URI": "http://acs.amazonaws.com/groups/global/AllUsers"
        },
        "Permission": "FULL_CONTROL"
    },

The criminals took advantage of the lax permissions to add malicious code to every JavaScript file found in the S3 bucket. Uniqlo altered the permissions on the bucket after we provided them with the details of the incident.

Misconfigured permissions on S3 buckets have been the centre of a number of data leaks in the past few years with the NSA and GoDaddy among those affected.

A not-so-unique attack

The Guardian and HuffPost have also loaded compromised resources on their websites, though no customers were affected as the malicious code was loaded in an iframe. The malicious code is intended to work in resources loaded in <script> tags — when the criminals compromise other file types, the malicious code often does not work as intended. If the criminals had been targeting The Guardian, they could have inserted a very convincing phishing site into the article.

The Guardian's website served card stealing code The Guardian's website served card stealing code

Software vendors Picreel and Translation Exchange, both of whom provide resources that are loaded on their customers' sites were also compromised. By adding malicious code to just these two buckets, the criminals infected over a thousand sites.

Criminal infrastructure

In all of these cases, the criminals have used the same attack vector and malicious skimming code. We have so far seen a total of six different dropsites that receive credentials from sites compromised in this way:

Domain Registered Registrar IP Address Country Hosting Company
ww1-filecloud[.]com 2019-01-30 WebNIC 45.114.8.162 Hong Kong Cloudie Limited
font-assets[.]com 2019-04-22 Shinjiru 179.43.144.137 Panama Private Layer
cdn-c[.]com 2019-05-13 Namecheap 94.177.123.154 Russia QHoster
cdn-imgcloud[.]com 2019-05-16 Shinjiru 45.114.8.160 Hong Kong Cloudie Limited
js-cloudhost[.]com 2019-05-17 Ilovewww 45.114.8.163 Hong Kong Cloudie Limited
wix-cloud[.]com 2019-05-17 Shinjiru 94.177.123.158 Russia QHoster

This suggests that these attacks are carried out by a single criminal group as part of the same campaign. It is common for criminals to carry out campaigns with more than one dropsite as it makes it harder for their operation to be detected and stopped.

Protect yourself

It is very difficult even for the most tech-savvy consumers to spot a JavaScript skimmer when browsing, making skimmers an invisible threat to online shopping safety. Netcraft's browser extensions and Android app provide protection against online threats, including shopping site skimmers, other forms of malicious JavaScript and phishing.

Companies with customers within the EU that fail to adequately protect personal information can face severe penalties; since GDPR was implemented in 2018, fines of €20 million or up to 4% of annual global turnover can be issued to non-compliant companies. British Airways currently faces a £183m fine from the ICO following a similar attack against its customers. A number of other high-profile shopping sites have recently fallen victim, including Misfit Wearables and ARCTIC.

Netcraft offers a range of services, including web application security testing, to protect organisations and their customers against malicious JavaScript and other forms of attack.

Netcraft Extension adds protection against malicious JavaScript

Netcraft has updated its browser extension to add protection against malicious JavaScript, including shopping site skimmers and web miners.

The Netcraft Extension now protects against malicious JavaScript The Netcraft Extension now protects against malicious JavaScript

Shopping site skimmers are malicious JavaScript programs that steal your payment card information when you checkout on a compromised online store, and send it back to a fraudster to use later. These attacks have affected a large number of online shops, and are invisible to even the most vigilant shoppers as there is no visual change to the page.

Web miners are malicious JavaScript programs inserted by fraudsters into a website that let them steal your CPU power to mine for cryptocurrency using your browser without your consent. Browsing a website with a web miner can often slow down your computer by consuming its resources.

Netcraft proactively scans for new shopping site skimmers, web miners, and other malicious JavaScript on the web. In the past six months, we have found malicious JavaScript affecting over 70,000 sites on over 29,000 distinct IPs. We have identified these attacks on high-profile sites such as Cleor (a large French jewellery retailer), Misfit (a wearable devices brand), and Arctic (a PC components and accessories retailer), as well as on the personalized merchandise stores for dozens of sports clubs, including Southampton's and Swansea's.

A well-hidden shopping site skimmer A well-hidden shopping site skimmer

By using the Netcraft Extension you can get protection from these types of attack as soon as we detect them. It was first made available for Internet Explorer in 2004, Firefox in 2005, Chrome in 2012, Opera in 2013, and Microsoft Edge earlier this year.

Select your browser to download the Netcraft Extension now:

Firefox chrome_64x64 opera_64x64 edge-logo

If you already have the Netcraft Extension installed your browser will update it automatically.

The Netcraft app for Android also provides protection against these attacks, while the Netcraft app for iOS provides protection against phishing attacks.

google-play-badge Amazon Badge Apple App Store Badge

August 2019 Web Server Survey

In the August 2019 survey we received responses from 1,271,920,923 sites across 239,441,736 unique domain names and 8,948,887 web-facing computers. This reflects a large loss of 124 million sites, but a gain of 1.30 million domains and 10,700 computers.

All major vendors lost active sites this month, and of those, only Google made a gain in sites (+1.58 million). Microsoft lost the largest number of active sites (-2.03 million), while nginx lost the most sites (-81.4 million, -16.9%) but remains in the lead with a 31.6% share of all sites.

Despite losing so many sites, nginx showed the strongest growth in unique domains, web-facing computers, and among the top million sites. This bears more significance than the more unpredictable changes in the site counts, which are prone to fluctuations month-on-month as link farms, spam networks and other low-value web content comes and goes.

With a gain of 58,500 web-facing computers, nginx now has more than 31% of the computer market share – just 5.39 percentage points behind Apache – while Microsoft has lost 65,000 computers. As is evident in the graphs, counting web-facing computers provides the most stable metric and makes long term trends easy to spot. In particular, the clear and consistent rise in nginx's market share and the steady decline of Apache makes it hard not to imagine nginx taking the market lead from Apache by early next year.

The number of top-million websites powered by nginx has increased by 1,292, while Apache's count fell by 3,101. Apache maintains the lead in this market, but is now only 5.92 percentage points ahead of nginx. Apache also continues to lead in terms of unique domains, despite losing 784,000 this month. It has a similar lead over nginx, which is now only 5.32 percentage points behind Apache after gaining 753,000 domains.

Microsoft lost counts in almost all metrics this month, apart from where it gained 166,000 domains, although this still resulted in a small drop in its domain market share. The sites market is the only one where its share did not fall, despite losing 16.6 million sites.

Netflix finds nginx vulnerabilities

nginx 1.61.1 stable and nginx 1.17.3 mainline were released on 13th August, in order to address three HTTP/2 security issues that could cause excessive memory consumption and CPU usage. All versions between 1.9.5 – 1.17.2 are affected, but only if HTTP/2 is enabled. These security issues were discovered by Jonathan Looney at Netflix, which chose to use nginx when developing its own globally distributed content delivery network, known as Netflix Open Connect.

The content delivery network consists of Open Connect Appliances, which run the FreeBSD operating system and use nginx to stream audio and video directly to Netflix customers. Most of this content is served from appliances hosted by ISPs, rather than across the internet, which leads to better performance whilst vastly reducing the amount of peered traffic when huge numbers of customers worldwide stream a popular show at the same time. Thousands of ISPs have enthusiastically participated in this program because it is free to connect to the Open Connect network, and it prevents Netflix traffic from taking up a significant amount of an ISP's internet capacity.

FreeBSD is dying?!

Netflix chose FreeBSD for its balance of stability and features (as did Netcraft once upon a time), but it is becoming an increasingly less common frontend operating system on the web as a whole. Only 60,200 (0.67%) web-facing computers are running FreeBSD today. To put this into perspective, more than twice as many servers are still running Windows Server 2003, even though it has not been supported for several years.

Linux is by far the most commonly used operating system for web-facing computers. It is installed on 6.64 million (74.2%) servers, and at least 1.05 million of these can be positively identified as running the Ubuntu distribution.

Naturally, the choice of operating system depends to some extent on what type of web server will be running on it, and vice versa. For example, it is no surprise that most instances of Microsoft IIS can be found running on Windows Server, and most instances of Windows Server are used to run Microsoft IIS; but it is clear that the Linux operating system is especially favoured for some web servers. Between 92% and 96% of all web-facing computers that use each of nginx, Apache, Litespeed and lighttpd can be found running Linux.

AWS ELB overtakes Beaver

The awselb (Amazon Web Services Elastic Load Balancing) web server was found on 69,800 web-facing computers this month, overtaking Beaver to become the fourth most commonly used frontend server by computers. Practically all of these machines appear to be running Linux, and are responsible for hosting 464,000 sites across 48,500 unique domains.

ELB achieves fault tolerance and scalability by automatically distributing incoming application traffic across multiple targets – and can even spread it across multiple AWS Availability Zones – so the 69,800 AWS ELB servers exposed to the internet are likely to be only the tip of the iceberg in terms of the AWS infrastructure used by each website.

Total number of websites

Web server market share

DeveloperJuly 2019PercentAugust 2019PercentChange
nginx482,877,27534.59%401,454,02931.56%-3.03
Apache387,366,82627.75%374,277,24329.43%1.68
Microsoft203,673,34414.59%187,109,42314.71%0.12
Google29,385,0652.11%30,969,2592.43%0.33
Continue reading