In the November 2012 survey we received responses from 625,329,303 sites, a modest increase of 4.8 million sites since last month's survey.
Apache has continued its declining trend in market share, suffering the greatest loss this month of 0.77 percentage points. Although it still captures the majority of the market with a share of 57.23%, this has fallen from 65.00% since this time last year. Conversely, Microsoft enjoyed the largest gain this month, upping its share by 0.25 to 16.52% this month and by 1.07 over the past year.
In absolute terms, Apache lost 2.0 million sites this month, whereas Microsoft gained 2.3 million. nginx also showed significant growth, with an additional 1.2 million sites resulting in a small increase to its market share.
Studying only active sites, the changes in market share are reversed: Apache grew by 0.23 to 55.66%, while Microsoft fell by 0.83 to 11.53%. Within the million busiest sites, nginx was the only major developer to increase its market share, which now stands at 12.22%.
November's survey saw Tumblr suddenly become the tenth largest hosting company by number of sites. The microblogging service previously hosted its customers' sites with Softlayer, but now hosts 9.2 million sites on its own netblock. Softlayer was previously the largest hosting company in the world in terms of hostnames, but this move now places that title in the hands of Go Daddy, who host nearly 6 million sites more than Softlayer.
Many sites using the .tk country code top level domain (ccTLD) disappeared from the survey this month, as non-existent domain names under this ccTLD are no longer resolved by a wildcard DNS configuration. The .tk ccTLD belongs to Tokelau (a territory of New Zealand), and is run by Dot TK - a joint venture between the Government of Tokelau, the communication company Teletok, and BV Dot TK, which is a privately held company with offices in the Netherlands, the UK and the Isle of Man.
Dot TK offers free domain names under the .tk ccTLD, which is also used by its URL shortening service. The April 2012 report by the Anti-Phishing Working Group says that .tk domains were taken advantage of extensively by phishers, and ranked .tk as the top phishing TLD by phish per 10,000 domains in the second half of 2011.
To combat phishing and other fraud, Dot TK introduced an anti-abuse API to allow trusted partners to shut down sites using the .tk ccTLD. The report notes that this has resulted in lower-than-average uptimes for phishing sites, but has not prevented phishers from obtaining and using .tk domains in the first place.
Netcraft has produced a live table of the top 50 phishiest TLDs, based on the ratio of the number of phishing sites to the total number of sites hosted within each TLD. This ranks .tk as only the 22nd phishiest TLD today, while .to (Tonga) currently ranks as the phishiest. The .to ccTLD is run by the Tonga Network Information Center (Tonic), which is one of the few ccTLD operators that does not provide registration information in a WHOIS database. This fact alone must surely contribute to the appeal of using the .to TLD for phishing and other fraud.
Developer October 2012 Percent November 2012 Percent Change Apache 359,875,516 58.00% 357,865,215 57.23% -0.77 Microsoft 101,005,285 16.28% 103,333,170 16.52% 0.25 nginx 73,243,944 11.80% 74,437,764 11.90% 0.10 20,947,340 3.38% 21,090,410 3.37% -0.00
Monitor phishing within your top-level domains
While some registries still perceive phishing as a content issue for hosting companies and registrars, detailed knowledge of phishing activity within their Top Level Domain(s) is very beneficial for registries. It is a key data source for identifying problematic, negligent, or fraud-friendly registrars, and an essential tool for maintaining the reputation of a TLD.
It is common for hosting companies and domain registrars to unknowingly allow their infrastructure to be used for phishing. Even seemingly respectable companies may develop a reputation as a haven for fraud though some systematic deficiency in their working practices, such as a low level of resourcing for abuse related workflow (particularly outside core working hours and during weekends), or inexperienced or less capable staff being unable to recognise and act on fraudulent content.
The most prolific hosts of .net phishing sites, October 2012
Conversely, some criminal registrars and hosting companies specialise in hosting fraudulent content, and even go so far as to advertise their services as "bullet-proof". Bullet-proof hosting companies are typically based in jurisdictions where laws may be hard to apply, and being in an informed position to decline further business from these registrars may greatly aid operational efficiency.
Professionally validated feed, relied upon throughout the Industry
Netcraft's continuously updated, professionally validated phishing feed is used throughout the Internet Infrastructure industry. In addition to Internet registries, all of the main web browsers, along with major anti-virus companies, firewall vendors, SSL Certificate authorities, large hosting companies and domain registrars use Netcraft's feed to protect their user communities. Since Netcraft first launched its anti-phishing system in 2005, over 5.2 million unique phishing sites have been detected and blocked as of September 2012.
Reporting and Analysis
Reports can be refreshed hourly, and also trended over time periods of many months, with analysis by registrar, hosting company, name server, country or phishing target.
.net phishing sites by country, October 2012
When Netcraft validates a phishing report in your TLD, you can receive an alert and can also arrange for alerts to be passed through to registrars. Acting on these individual alerts will demonstrate that your top-level domains are not welcoming to fraud. Fraudsters adjust to these signals within a short period of time, and are themselves quite efficient at moving their operations away from parts of the DNS where they are clearly unwelcome.
A refreshable Excel spreadsheet includes details of the phishing sites under the .net TLD
Case Study - Nominet .uk
Nominet is the registry responsible for managing the .uk domain, which is one of the largest ccTLDs with over 10 million domains registered as of March 2012. Netcraft has provided Nominet with information on phishing using .uk domains since 2009, with alerts made available to individual registrars via an opt-in service.
Please contact us (firstname.lastname@example.org) for pricing or further details about any of our services.
To provide a comprehensive view of the web hosting industry, Netcraft has researched all of the hosting locations with at least twenty web facing computers found by our Web Server Survey. Of these eleven thousand hosting locations, around seven thousand provide hosting and connectivity services, the remaining being enterprises, government or educational institutions.
Netcraft has noted the services provided by each Internet Services company and the dataset includes these classifications, together with the numbers of computers found in our Hosting Provider Server Count segmented by operating system.
Field Description Parent Company Parent company, if applicable. Company Company name Number of Computers The total number of Web Facing Computers found by our Hosting Provider Server Count segmented by operating system. Websites A list of the company's own websites. Country The main country of the company, based on their headquarters address. Services Notes which of the following services is offered by the company:
- Domain Registration
- Paid Shared Hosting
- Free Shared Hosting
- Dedicated Hosting
- Reseller Hosting
- Managed Services - includes packages, software configuration, firewall maintenance, monitoring.
- Cloud/Grid Services
- Virtual Private Servers
- Ecommerce & Shopping carts
- Streaming / Podcast Hosting
- Application Hosting / Software As A Service
- Bespoke Web development
- E-mail hosting
- IPv6 Addressing
- Leased Lines
- Traditional Telco Services - e.g. telephone calling plans, line rental, fax, and mobile contracts.
- VoIP - Voice Over IP
- SSL Certificates
Provisioning Information on online ordering including accepted payment methods and expected set-up periods. Data Centre Locations Countries in which the company has data centres. Control Panel Software A list of the solutions available, e.g. CPanel and Plesk Virtualization Software A list of the solutions available, e.g. HyperV, Xen, VMware, Parallels. Partners A list of the company's publically advertised partners, for example, Cisco, Microsoft, Dell. Main Business The company's main business area. Language The primary language used by the company website. Multi-Lingual Whether the company website is available in more than one language. Address The address of the company headquarters. Company Contact Details The main telephone number, fax number and e-mail address of the company. Company URLS URLs to the following pages on the company website: Contact Us, About Us, Management, Partners. Executive Contacts The executive contacts published by the company. Social Networking URLs to Twitter, Facebook and LinkedIn pages for the company. Stock Market Information URL to Google Finance page for the company, if publicly quoted.
The dataset is available in Excel format, making it simple to filter and sort the information, and allowing companies offering similar services to be compared.
The dataset is available on a company license basis. We are able to provide subsets of the data, for example, all hosting companies that offer cloud services in North America or VPS providers in Europe, or any other segmentation by geography or technology.
On demand, the classification could be extended to include smaller hosting companies and resellers.
Please contact us (email@example.com) for further information and costs.
The Domain Registration Risk Calculator is a tool for domain registrars to analyse the likelihood that new domains will be used for fraudulent activities. The service identifies domains which are deceptively similar to legitimate websites run by banks and other institutions commonly targeted by phishing attacks.
Since such registrations are often made using stolen credit cards, there are significant advantages to the registrar in refusing them.
Netcraft has blocked well over five million phishing attacks since 2005, and our phishing feed is used by all of the major web browsers, and also by leading anti-virus companies, domain registrars, registries, certificate authorities and hosting companies. Our extensive experience in identifying, validating and eliminating phishing sites has provided us with a wealth of knowledge of the tricks that are used by fraudsters to create a deceptive domain name. We analyse our database of over six thousand organisations which have been targeted by phishing attacks to extract a comprehensive set of homoglyphs that could be used to convert bona fide domains to fraudulent ones. Example transformations are the corresponding characters from an IDN alphabet, or ASCII character set substitutions such as replacing “o” (letter O) with “0” (zero), or replacing “l” (lower-case letter l) with “1” (digit one), or simply appending or prepending strings such as update or secure.
A Facebook phishing site, along with its Domain Registration Risk score
The service computes a registration risk score for a proposed domain, which gives a measure of the likelihood that this candidate domain may be used to host a phishing attack. We do this by using the results of two algorithms:
The first algorithm, Phish target score compares the
candidate domain to each of the frequently-phished legitimate domains we have on
record. This comparison is done on a per-character basis, and the score is formed
by looking at the minimum set of edits required to map from one to the other.
The algorithm recognises certain tricks commonly used in domain names to deceive victims, such as double letters (paaypal.com) or confusing characters or combinations of characters (paypa1.com). We also check against a list of deceptive prefixes and suffixes that are frequently used by phishing sites, including signin and verify.
As well as using a set of fixed rules, this algorithm also retains the flexibility to match new mappings and edits that have not been seen before. Using the suggested cut-off of a minimum score of 5/10, this method identifies 278 (12.7%) out of the 2,191 phishing domains currently blocked by Netcraft.
The second algorithm, String entropy score, works entirely differently. Many phishing domains in our database are essentially random strings of alphanumeric digits, yet very few legitimate sites follow this pattern. The string entropy test looks to see if a domain looks like a combination of real dictionary words and plausible names, or whether it looks more like a randomised string. The higher the score, the more random a string appears to be.
Although most dictionary strings score zero, the suggested cut-off is a minimum score of 5/10; any domain scoring higher than this is very likely to be random, but below this score false positives are increasingly likely.
Using the suggested cut-off identifies 474 (21.6%) of the 2,191 identified phishing domains and these are substantially non-overlapping with those domains spotted by the first method.
These two methods work together to give sophisticated and largely independent indicators of the likelihood that a candidate domain may be used to host phishing attacks against a known legitimate target. Using the overall risk rating produced by combining the two scores would presently detect 742 (33.9%) of the 2,191 currently blocked phishing domains.
The domains in the table below have run phishing attacks and are shown together with their domain registration risk.
Domain Target Registration Risk hsbc-hk.biz hsbchk.com 10.00 activate-facebook-security-confirmation.tk facebook.com 10.00 xdzfhv.tk (none) 9.98 cimbclicksonline.com cimbclicks.com.my 9.10 jtlwm.com (none) 8.94 taobao581.cn taobao.com 8.84 halifaxinternational.org halifax.co.uk 8.67 skype-load.com skype.com 8.49 natwestt.co.uk natwest.co.uk 8.26 1tw1tter.com twitter.com 7.14 santadar.co.uk santander.co.uk 6.93 htmail.co.uk hotmail.co.uk 6.66 dhl-couriers.co.uk dhl.co.uk 5.54 sbo6666.com sbo666.com 5.64 alibabeexpress.com alibaba.com 5.07
A web-based interface to the system is available for evaluation purposes and ad-hoc queries. For automated processes and bulk queries an API is available to return domain registration risk information in JSON format. Bespoke formats can be made available on request.
Entering the domain securepaypa1.com into the test system produces the report shown below:
Please get in touch (firstname.lastname@example.org) if you would like to try out this service or for subscription information.
- The first algorithm, Phish target score compares the candidate domain to each of the frequently-phished legitimate domains we have on record. This comparison is done on a per-character basis, and the score is formed by looking at the minimum set of edits required to map from one to the other.
In the October 2012 survey we received responses from 620,480,777 sites, an increase of 350K sites since last month's survey.
In spite of this, all major web server vendors lost hostnames this month with the exception of Microsoft – gaining around 3.5M – with a 0.58% rise in market share. This continues Apache's decline in market share, with a drop of 0.49 percentage points from last month.
In terms of active sites, all of the major vendors made losses. In terms of market share, both Microsoft and Apache made small gains, 0.5 percentage points for Apache compared with 0.1% for Microsoft.
In the top million busiest sites, nginx made modest gains in market share. Microsoft, Apache and Google all lost sites. This continues the downward trend on from last month for both Apache and Google, with Apache falling further back from the 60% threshold that it hit in August.
Although Apache dominates the overall market, where it is used by 58% of all websites, the secure server market paints a very different picture. Netcraft's SSL Survey found 2.3 million distinct, valid third-party certificates being used by HTTPS websites in October. Apache and Microsoft are almost neck and neck with 41.6% and 40.8% of the secure market share respectively, with the latter being noticeably higher than Microsoft's 15.7% share amongst HTTP sites.
Usage of nginx is also significantly different between the HTTPS and HTTP markets: although it is used by 11.9% of all sites in the Web Server Survey, it is used to serve only 2.3% of SSL certificates.
This month 516k hostnames moved from FC2 to Amazon, contributing to a net gain of 342k. FC2 is a provider of free ad-supported blogs and other web-based blogging tools. However, as many of these blogs are inactive, this change did not lead to an overall increase in the number of active sites hosted by Amazon.
Nasdaq has announced that it plans to make use of Amazon Web Services (AWS) to power its new financial data management system, which retains data required for financial regulation. This can be seen as a sign that large organisations are becoming more willing to make use of third party cloud offerings to host their data systems, and that the benefits associated with cloud-based solutions (low cost and high scalability) are increasingly outweighing the risks (security and accountability concerns). This follows Amazon becoming the largest hosting location last month. The quantity of web-facing computers owned by Amazon increased a further 5% this month.
Developer September 2012 Percent October 2012 Percent Change Apache 362,714,083 58.49% 359,875,516 58.00% -0.49 Microsoft 97,368,803 15.70% 101,005,285 16.28% 0.58 nginx 73,976,009 11.93% 73,243,944 11.80% -0.12 21,576,233 3.48% 20,947,340 3.38% -0.10
Rank Company site OS Outage
DNS Connect First
Total 1 Qube Managed Services Linux 0:00:00 0.003 0.196 0.096 0.194 0.194 2 Hosting 4 Less Linux 0:00:00 0.003 0.142 0.101 0.204 0.391 3 Kattare Internet Services Linux 0:00:00 0.007 0.186 0.064 0.129 0.263 4 New York Internet FreeBSD 0:00:00 0.007 0.172 0.079 0.160 0.486 5 XILO Communications Ltd. Linux 0:00:00 0.007 0.275 0.103 0.319 0.537 6 Datapipe FreeBSD 0:00:00 0.014 0.100 0.016 0.032 0.048 7 www.logicworks.net Linux 0:00:00 0.014 0.199 0.082 0.473 0.585 8 www.choopa.com Linux 0:00:00 0.014 0.207 0.088 0.180 0.245 9 ServInt Linux 0:00:00 0.014 0.362 0.091 0.185 0.338 10 www.netcetera.co.uk Windows Server 2008 0:00:00 0.014 0.083 0.102 0.206 0.512
Qube Managed Services was the most reliable hosting company in September, responding to 99.997% of all requests throughout the month. The London-based company specialises in VMware cloud services, colocation, backups, and also offers PCI-DSS compliant hosting on both virtual and dedicated platforms. Qube also has infrastructure in Zurich and New York, and was also the most reliable hosting company during August.
Hosting 4 Less was the second most reliable hosting company, responding to the same percentage of requests, but with a longer average connection time. Hosting 4 Less has been operating since 1998 and offers both dedicated and shared hosting on Linux or Windows platforms, all backed by a 99.9% uptime guarantee.
Linux was the most prevalent operating system amongst the 10 most reliable hosting companies in September; seven of these sites were hosted on Linux servers, including www.qubenet.net and www.hosting4less.com, while two used FreeBSD and one used Windows Server 2008.
Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.
From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.
Information on the measurement process and current measurements is available.