Netcraft Taxonomy of Internet Services Companies

To provide a comprehensive view of the web hosting industry, Netcraft has researched all of the hosting locations with at least twenty web facing computers found by our Web Server Survey. Of these eleven thousand hosting locations, around seven thousand provide hosting and connectivity services, the remaining being enterprises, government or educational institutions.


Netcraft has noted the services provided by each Internet Services company and the dataset includes these classifications, together with the numbers of computers found in our Hosting Provider Server Count segmented by operating system.


Field Description
Parent Company Parent company, if applicable.
Company Company name
Number of Computers The total number of Web Facing Computers found by our Hosting Provider Server Count segmented by operating system.
Websites A list of the company's own websites.
Country The main country of the company, based on their headquarters address.
Services Notes which of the following services is offered by the company:
  • Domain Registration
  • Paid Shared Hosting
  • Free Shared Hosting
  • Dedicated Hosting
  • Reseller Hosting
  • Colocation
  • Managed Services - includes packages, software configuration, firewall maintenance, monitoring.
  • Cloud/Grid Services
  • Virtual Private Servers
  • Ecommerce & Shopping carts
  • Streaming / Podcast Hosting
  • Application Hosting / Software As A Service
  • Bespoke Web development
  • Blogs
  • E-mail hosting
  • Webmail
  • Broadband
  • IPv6 Addressing
  • Leased Lines
  • Traditional Telco Services - e.g. telephone calling plans, line rental, fax, and mobile contracts.
  • VoIP - Voice Over IP
  • SSL Certificates
Provisioning Information on online ordering including accepted payment methods and expected set-up periods.
Data Centre Locations Countries in which the company has data centres.
Control Panel Software A list of the solutions available, e.g. CPanel and Plesk
Virtualization Software A list of the solutions available, e.g. HyperV, Xen, VMware, Parallels.
Partners A list of the company's publically advertised partners, for example, Cisco, Microsoft, Dell.
Main Business The company's main business area.
Language The primary language used by the company website.
Multi-Lingual Whether the company website is available in more than one language.
Address The address of the company headquarters.
Company Contact Details The main telephone number, fax number and e-mail address of the company.
Company URLS URLs to the following pages on the company website: Contact Us, About Us, Management, Partners.
Executive Contacts The executive contacts published by the company.
Social Networking URLs to Twitter, Facebook and LinkedIn pages for the company.
Stock Market Information URL to Google Finance page for the company, if publicly quoted.

The dataset is available in Excel format, making it simple to filter and sort the information, and allowing companies offering similar services to be compared.


Availability

The dataset is available on a company license basis. We are able to provide subsets of the data, for example, all hosting companies that offer cloud services in North America or VPS providers in Europe, or any other segmentation by geography or technology.

On demand, the classification could be extended to include smaller hosting companies and resellers.

Please contact us (sales@netcraft.com) for further information and costs.


Domain Registration Risk Service now available

The Domain Registration Risk Calculator is a tool for domain registrars to analyse the likelihood that new domains will be used for fraudulent activities. The service identifies domains which are deceptively similar to legitimate websites run by banks and other institutions commonly targeted by phishing attacks.

Since such registrations are often made using stolen credit cards, there are significant advantages to the registrar in refusing them.

Netcraft has blocked well over five million phishing attacks since 2005, and our phishing feed is used by all of the major web browsers, and also by leading anti-virus companies, domain registrars, registries, certificate authorities and hosting companies. Our extensive experience in identifying, validating and eliminating phishing sites has provided us with a wealth of knowledge of the tricks that are used by fraudsters to create a deceptive domain name. We analyse our database of over six thousand organisations which have been targeted by phishing attacks to extract a comprehensive set of homoglyphs that could be used to convert bona fide domains to fraudulent ones. Example transformations are the corresponding characters from an IDN alphabet, or ASCII character set substitutions such as replacing “o” (letter O) with “0” (zero), or replacing “l” (lower-case letter l) with “1” (digit one), or simply appending or prepending strings such as update or secure.

A Facebook phishing site, along with its Domain Registration Risk score

The service computes a registration risk score for a proposed domain, which gives a measure of the likelihood that this candidate domain may be used to host a phishing attack. We do this by using the results of two algorithms:

  • The first algorithm, Phish target score compares the candidate domain to each of the frequently-phished legitimate domains we have on record. This comparison is done on a per-character basis, and the score is formed by looking at the minimum set of edits required to map from one to the other.

    The algorithm recognises certain tricks commonly used in domain names to deceive victims, such as double letters (paaypal.com) or confusing characters or combinations of characters (paypa1.com). We also check against a list of deceptive prefixes and suffixes that are frequently used by phishing sites, including signin and verify.

    As well as using a set of fixed rules, this algorithm also retains the flexibility to match new mappings and edits that have not been seen before. Using the suggested cut-off of a minimum score of 5/10, this method identifies 278 (12.7%) out of the 2,191 phishing domains currently blocked by Netcraft.

  • The second algorithm, String entropy score, works entirely differently. Many phishing domains in our database are essentially random strings of alphanumeric digits, yet very few legitimate sites follow this pattern. The string entropy test looks to see if a domain looks like a combination of real dictionary words and plausible names, or whether it looks more like a randomised string. The higher the score, the more random a string appears to be.

    Although most dictionary strings score zero, the suggested cut-off is a minimum score of 5/10; any domain scoring higher than this is very likely to be random, but below this score false positives are increasingly likely.

    Using the suggested cut-off identifies 474 (21.6%) of the 2,191 identified phishing domains and these are substantially non-overlapping with those domains spotted by the first method.

These two methods work together to give sophisticated and largely independent indicators of the likelihood that a candidate domain may be used to host phishing attacks against a known legitimate target. Using the overall risk rating produced by combining the two scores would presently detect 742 (33.9%) of the 2,191 currently blocked phishing domains.


Example Domains

The domains in the table below have run phishing attacks and are shown together with their domain registration risk.

DomainTargetRegistration Risk
hsbc-hk.bizhsbchk.com10.00
activate-facebook-security-confirmation.tkfacebook.com10.00
xdzfhv.tk(none)9.98
cimbclicksonline.comcimbclicks.com.my9.10
jtlwm.com(none)8.94
taobao581.cntaobao.com8.84
halifaxinternational.orghalifax.co.uk8.67
skype-load.comskype.com8.49
natwestt.co.uknatwest.co.uk8.26
1tw1tter.comtwitter.com7.14
santadar.co.uksantander.co.uk6.93
htmail.co.ukhotmail.co.uk6.66
dhl-couriers.co.ukdhl.co.uk5.54
sbo6666.comsbo666.com5.64
alibabeexpress.comalibaba.com5.07

The Interface

A web-based interface to the system is available for evaluation purposes and ad-hoc queries. For automated processes and bulk queries an API is available to return domain registration risk information in JSON format. Bespoke formats can be made available on request.

Entering the domain securepaypa1.com into the test system produces the report shown below:


More Information

Please get in touch (sales@netcraft.com) if you would like to try out this service or for subscription information.


October 2012 Web Server Survey

In the October 2012 survey we received responses from 620,480,777 sites, an increase of 350K sites since last month's survey.

In spite of this, all major web server vendors lost hostnames this month with the exception of Microsoft – gaining around 3.5M – with a 0.58% rise in market share. This continues Apache's decline in market share, with a drop of 0.49 percentage points from last month.

In terms of active sites, all of the major vendors made losses. In terms of market share, both Microsoft and Apache made small gains, 0.5 percentage points for Apache compared with 0.1% for Microsoft.

In the top million busiest sites, nginx made modest gains in market share. Microsoft, Apache and Google all lost sites. This continues the downward trend on from last month for both Apache and Google, with Apache falling further back from the 60% threshold that it hit in August.

Although Apache dominates the overall market, where it is used by 58% of all websites, the secure server market paints a very different picture. Netcraft's SSL Survey found 2.3 million distinct, valid third-party certificates being used by HTTPS websites in October. Apache and Microsoft are almost neck and neck with 41.6% and 40.8% of the secure market share respectively, with the latter being noticeably higher than Microsoft's 15.7% share amongst HTTP sites.

Usage of nginx is also significantly different between the HTTPS and HTTP markets: although it is used by 11.9% of all sites in the Web Server Survey, it is used to serve only 2.3% of SSL certificates.

This month 516k hostnames moved from FC2 to Amazon, contributing to a net gain of 342k. FC2 is a provider of free ad-supported blogs and other web-based blogging tools. However, as many of these blogs are inactive, this change did not lead to an overall increase in the number of active sites hosted by Amazon.

Nasdaq has announced that it plans to make use of Amazon Web Services (AWS) to power its new financial data management system, which retains data required for financial regulation. This can be seen as a sign that large organisations are becoming more willing to make use of third party cloud offerings to host their data systems, and that the benefits associated with cloud-based solutions (low cost and high scalability) are increasingly outweighing the risks (security and accountability concerns). This follows Amazon becoming the largest hosting location last month. The quantity of web-facing computers owned by Amazon increased a further 5% this month.

Developer September 2012 Percent October 2012 Percent Change
Apache 362,714,083 58.49% 359,875,516 58.00% -0.49
Microsoft 97,368,803 15.70% 101,005,285 16.28% 0.58
nginx 73,976,009 11.93% 73,243,944 11.80% -0.12
Google 21,576,233 3.48% 20,947,340 3.38% -0.10
Continue reading

Most Reliable Hosting Company Sites in September 2012

Rank Company site OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 Qube Managed Services Linux 0:00:00 0.003 0.196 0.096 0.194 0.194
2 Hosting 4 Less Linux 0:00:00 0.003 0.142 0.101 0.204 0.391
3 Kattare Internet Services Linux 0:00:00 0.007 0.186 0.064 0.129 0.263
4 New York Internet FreeBSD 0:00:00 0.007 0.172 0.079 0.160 0.486
5 XILO Communications Ltd. Linux 0:00:00 0.007 0.275 0.103 0.319 0.537
6 Datapipe FreeBSD 0:00:00 0.014 0.100 0.016 0.032 0.048
7 www.logicworks.net Linux 0:00:00 0.014 0.199 0.082 0.473 0.585
8 www.choopa.com Linux 0:00:00 0.014 0.207 0.088 0.180 0.245
9 ServInt Linux 0:00:00 0.014 0.362 0.091 0.185 0.338
10 www.netcetera.co.uk Windows Server 2008 0:00:00 0.014 0.083 0.102 0.206 0.512

See full table

Qube Managed Services was the most reliable hosting company in September, responding to 99.997% of all requests throughout the month. The London-based company specialises in VMware cloud services, colocation, backups, and also offers PCI-DSS compliant hosting on both virtual and dedicated platforms. Qube also has infrastructure in Zurich and New York, and was also the most reliable hosting company during August.

Hosting 4 Less was the second most reliable hosting company, responding to the same percentage of requests, but with a longer average connection time. Hosting 4 Less has been operating since 1998 and offers both dedicated and shared hosting on Linux or Windows platforms, all backed by a 99.9% uptime guarantee.

Linux was the most prevalent operating system amongst the 10 most reliable hosting companies in September; seven of these sites were hosted on Linux servers, including www.qubenet.net and www.hosting4less.com, while two used FreeBSD and one used Windows Server 2008.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

Phishing Alerts for Certificate Authorities

The internet community has been taught that one of the key steps in protecting their personal information on the internet is to ensure that it is entered only over an encrypted connection, perhaps by looking for the lock symbol in the browser address bar or web addresses beginning with https://. As a result, phishing attacks which make use of SSL certificates are especially dangerous as most users associate the presence of a valid SSL certificate with an increased level of assurance. Such attacks erode the reputation of Certificate Authorities and SSL certificates.

While the majority of phishing attacks run over HTTP, a significant number run on sites for which SSL certificates have been issued. In July 2012 alone, Netcraft found phishing attacks using a total of 505 unique valid SSL certificates from widely trusted issuers.

Although in some cases certificates have been issued specifically for the purposes of phishing the more common case is where well intentioned, bona fide certificate owners find that they are unwittingly providing facilities for phishing because their site has been compromised by an attacker.

Having access to timely, professionally validated alerts when phishing attacks occur is operationally efficient and responsible for certificate authorities, as well as an important part of preserving their company's reputation. It gives post issuance information on troublesome certificates and domains of which the certificate authority might otherwise be blissfully unaware.

Phishing Alerts are also a very valuable service for certificate holders, for whom it may be the first notification of a serious problem, giving them an opportunity to engage the attacker and wrest back control of their site before more harm is done.

Netcraft produces a continuously updated phishing feed that is very widely used. At least three separate third party studies have found it to be the most comprehensive feed available. The feed is used in all the major web browsers and it is also licensed by many of the leading anti-virus, content filtering, web-hosting and domain registration companies.

Phishing sites are submitted to the feed by the Netcraft Toolbar community. Reporters range from individuals submitting phishing mails that they have personally received, to specialist security researchers and several of the largest banks and financial payment systems. All submissions are carefully validated before being added to the feed. Well over five million unique phishing sites have been detected and blocked by Netcraft's community to date [September 2012].

GlobalSign commenced providing this service to all of its certificate owners in August 2012 (press release), and in the first month of the service around 70 distinct certificate owners were alerted to phishing attacks on sites where their certificates were deployed.

More information:

Please contact us (sales@netcraft.com) for pricing or further details about any of our services.

September 2012 Web Server Survey

In the September 2012 survey we received responses from 620,132,319 sites, a decrease of 8M sites since last month's survey.

A large portion of this drop was caused by a large network of linkfarmed domains disappearing from under the .com TLD, causing Apache numbers to suffer the most, with a loss of 10M sites. This resulted in a small drop in Apache's market share to 58%. Google also saw losses of 1M sites, but both Microsoft and nginx gained, with 840k and 1.5M new domains respectively.

Server headers for IIS 8.0 – the latest version of Microsoft's server software – were returned by 1,723 sites this month. This is an increase of 1,445 sites (+519%) over the six months since the public beta release of Windows Server 2012 in April, which uses IIS 8.0 as its default web server. However, only twelve of the million busiest sites were found to be using the software, seven of which are within Microsoft's own iis.net.

Amazon reached a significant milestone this month, with its strong and continued growth in the web hosting market now making it the world's largest hosting location by number of web-facing computers. The previous leader was China Telecom, which now has 116k web-facing computers against Amazon's 118k.

Netcraft's hosting provider server count uses a set of heuristics to identify individual computers, regardless of how many web-facing IP addresses each may have, or how many websites they serve.

Amazon has nearly doubled its count of web-facing computers within the past year, and this growth does not look set to slow down any time soon. The majority of these computers are located in the US (77%) and Ireland (13%), although smaller numbers of servers have started popping up in other locations within the past year, including the Netherlands, Singapore, Brazil, and Japan.

Although Amazon has the largest number of web-facing computers, these are used to host a relatively modest sum of 6.8M websites. 2.9M of these sites are served by nginx, which is closely followed by 2.3M served by Apache. A further 410k are served by Polyvore Web Server, which is used by sites within the Polyvore fashion social-commerce network. Only 2.4% (163k) of the sites hosted at Amazon are running Microsoft IIS.

Although Amazon's scalable, pay-as-you-go EC2 service supports Microsoft Windows, Linux is by far the most popular operating system to be found amongst all of its web-facing computers, including those used by CloudFront and S3. Nearly 97% of Amazon's web-facing computers were running Linux during September's survey.





DeveloperAugust 2012PercentSeptember 2012PercentChange
Apache373,069,75159.39%362,714,08358.49%-0.90
Microsoft96,529,58615.37%97,368,80315.70%0.33
nginx72,429,97611.53%73,976,00911.93%0.40
Google22,561,8543.59%21,576,2333.48%-0.11
Continue reading