August 2014 Web Server Survey

In the August 2014 survey we received responses from 992,177,228 sites — four million fewer than last month.

Despite losing more than six million hostnames and its lead over Apache shrinking to 2.13 percentage points, Microsoft managed to retain the top spot it snatched from Apache last month. Many of the lost hostnames that were using Microsoft IIS belonged to a single Chinese link farm; these sites typically last just a few months and so such volatility in the number of hostnames is no surprise. Apache was the only major server vendor to gain hostnames, adding more than 780,000.

Web Server Developer - Market Share of Computers

The number of web-facing computers, on the other hand, is less susceptible to fluctuations, representing the install base of each server vendor.

Web server market share for computers

DeveloperJuly 2014PercentAugust 2014PercentChange
Apache2,321,14147.92%2,338,92747.83%-0.09
Microsoft1,512,93331.24%1,515,67431.00%-0.24
nginx460,7959.51%478,7939.79%0.28

Whilst nginx lost more than six million hostnames — many from link farms of a similar nature to those using IIS albeit on a smaller scale — the number of web-facing computers running the open-source web server grew by almost 18,000. Although nginx had the largest growth this month, Apache was not far behind, growing by almost 17,800 web-facing computers. Despite this growth, Apache's share of web-facing computers fell slightly, though it retains its lead over Microsoft by a comfortable margin (47.8% vs 31%).

Although all of the major server vendors have experienced regular and steady growth in the web-facing computers metric, they have often been outpaced by nginx. Both Apache and IIS have seen a slow decline in market share as a result, losing 0.9 and 2.4 percentage points respectively over the last year. nginx's share of web-facing computers is now just shy of 10%, almost double its share in August 2012.

DigitalOcean opens London data centre

Cloud hosting provider DigitalOcean has seen tremendous growth over the past 18 months, and this month became the 5th largest hosting provider in terms of web-facing computers. More than half of all Digital Ocean droplets are running Apache, and a further 43% use nginx.

In July, DigitalOcean opened a new data centre in London, its third data centre in Europe. If the growth rate of their Singapore data centre is anything to go by, DigitalOcean could soon become one of the largest hosting providers in the UK.

Less than a month after opening their Singapore data centre in February 2014, DigitalOcean became the 8th largest provider in Singapore. Half a year later, DigitalOcean is now the second largest hosting provider in Singapore, with a total 4,900 computers, trailing only Amazon, which has more than 12,000 web-facing computers.

Explosive growth on .xyz

The biggest gain among the new gTLDs was on .xyz, which entered general availability on July 2nd. In just over a month, .xyz has grown by 225,000 hostnames, and is already the largest of the new gTLDs by both hostnames and domains. Almost all the growth appears to be the result of a Network Solutions promotion which offers a free matching .xyz domain with each .com domain purchased. Almost 200,000 of these new hostnames resolve to a single IP address, which shows a Network Solutions domain holding page.

Other large growths among the new gTLDs include .berlin which gained 85,500 hostnames, .club (+9,700), and .events (+5,500).

Microsoft using Brazilian IP address space in US

Microsoft recently opened an Azure data centre in São Paulo, making its first foray into South America. Shortly after the data centre was opened, some Azure users began noticing that their virtual machines were being assigned Brazilian IP addresses despite not using the new Brazilian data centre. Microsoft attributed this behaviour to its dwindling supply of available IPv4 address space, particularly in the United States. This month, almost 9,000 hostnames were found on 1,600 Brazilian IP addresses despite being hosted in one of Microsoft's other Azure data centres.

Total number of websites
 
Web server market share
 
Developer July 2014 Percent August 2014 Percent Change
Microsoft 373,869,026 37.53% 367,805,416 37.07% -0.46
Apache 345,921,550 34.73% 346,702,990 34.94% 0.22
nginx 141,041,852 14.16% 135,037,738 13.61% -0.55
Google 20,511,505 2.06% 20,076,890 2.02% -0.04
Continue reading

Most Reliable Hosting Company Sites in July 2014

Rank Performance Graph OS Outage hh:mm:ss Failed Req% DNS Connect First byte Total
1 Anexia Linux 0:00:00 0.011 0.273 0.093 0.396 0.673
2 Hyve Managed Hosting Linux 0:00:00 0.015 0.220 0.061 0.123 0.124
3 EveryCity SmartOS 0:00:00 0.015 0.070 0.066 0.132 0.132
4 Aspserveur Linux 0:00:00 0.015 0.294 0.086 0.413 0.757
5 Qube Managed Services Linux 0:00:00 0.019 0.096 0.036 0.073 0.073
6 ServerStack Linux 0:00:00 0.019 0.083 0.072 0.143 0.143
7 Datapipe FreeBSD 0:00:00 0.022 0.115 0.019 0.037 0.055
8 Host Europe Linux 0:00:00 0.022 0.126 0.073 0.156 0.157
9 GoDaddy.com Inc Linux 0:00:00 0.030 0.144 0.010 0.274 0.321
10 iWeb Linux 0:00:00 0.030 0.144 0.081 0.160 0.160

See full table

Anexia had the most reliable hosting company site in July, with three failed requests. This is Anexia's second appearance in the top 10 in 2014, and the first time it's topped the table since Netcraft starting monitoring its performance. The Europe-based IT service provider operates infrastructure in across the world, and has maintained a 100% uptime record over the past year. Last year, Anexia was ranked #136 in Deloitte's Technology Fast500 EMEA, which recognizes the top 500 fastest-growing technology companies in Europe, the Middle East & Africa.

Coming in close behind are Hyve Managed Hosting (second most reliable), EveryCity (third), and Aspserveur (fourth). As all three hosting companies had the same number of failed requests in July, the tie was broken by examining the average connection time.

This is Hyve Managed Hosting's third consecutive month in the top ten. Hyve have recently been appointed to G-Cloud V, the UK Government cloud computing initiative. It will offer a variety of cloud computing services to government departments, local authorities and public sector organisations across the country.

Serverstack ranked sixth with five failed requests, marking its third appearance in the top ten in 2014. Founded in 2004, Serverstack has since expanded to three datacenters in Amsterdam, New Jersey and San Jose. Serverstack offers a 100% uptime guarantee and has had a 100% uptime record over the past year.

Almost all the this month's top ten use Linux machines, with the exceptions of EveryCity's site in third place running SmartOS and Datapipe's site in seventh place running FreeBSD.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

July 2014 Web Server Survey

In the July 2014 survey we received responses from 996,106,380 sites.

Microsoft gained 22 million sites this month, helping to increase its market share by 1.18 percentage points. Combined with a 1.78 point loss in Apache's market share, Microsoft is now the new market leader with a 37.5% share of all sites.

In the early days of the web, hostnames were a good indication of actively managed content providing information and services to the Internet community. Today, hostnames are used for a wide range of activities, including holding pages produced at the point of customer acquisition by domain registration or hosting service companies, typo-squatting advertising providers, speculative domain registrants, and search-engine optimisation companies. Where wildcard DNS is used, the vast majority of the hostnames will not receive visitors, and the resources required to run the sites are minimal.

iis-share
The IIS market share growth in hostnames has not been reflected in our other metrics

Our active sites metric shows just how significant an influence automatically generated sites can have on the number of hostnames found by the survey for all server vendors. Apache remains the clear leader by number of active sites, and has been ever since we started using this additional metric in 2000. Over half of the world's active sites use Apache; a total of more than 91 million active sites, compared with Microsoft's 21 million.

Microsoft now leads in hostnames, but Apache is still far ahead in terms of active sites.

Microsoft’s most recent growth in hostnames since mid-2013 has, for the most part, been caused by a large number of Chinese linkfarms (泛站群). The sites in question provide advertising for gambling sites, online product listings, and normally make use of affiliate schemes. Yet they are hosted in the USA, on generic TLDs such as .com and .net to bypass China’s TLD and internet content provider (ICP) license requirements. Unusually, each linkfarm makes use of a reasonably large number of domains and IP addresses, presumably making them harder for search engines to evade. This would normally be cost prohibitive for this kind of activity, however hosting and domain packages can be found advertised on auction sites specifically for this purpose, with packages of (random/unspecified) .com domains available for as little as ¥17 (~ £2 / $3) each, guaranteed to remain yours for at least a month. It is not clear why IIS has been chosen for these sites, however it does have a considerably higher market share (for all of our metrics) in China compared to worldwide - for example 59% of domains hosted in China use IIS compared to just 29% worldwide.

In just over a year IIS has gained over 236 million hostnames (+172%) while only gaining 503k active sites (+2%). The number of web-facing computers running IIS websites has increased by just over 30k (+2%), compared to Apache’s 171k growth (+8%), and nginx’s 159k growth (+53%), resulting in a 2.4 percentage point loss in market share for IIS by this metric.

computer-share
 
 
Developer June 2014 Percent July 2014 Percent Change
Microsoft 352,208,487 36.35% 373,869,026 37.53% 1.18
Apache 353,672,431 36.50% 345,921,550 34.73% -1.78
nginx 133,763,494 13.81% 141,041,852 14.16% 0.35
Google 20,192,595 2.08% 20,511,505 2.06% -0.02
Continue reading

Bitcoin phishers get desperate with search engine ads

More than a week after we reported deceptive search engine ads being used in Bitcoin wallet attacks, fraudsters are still using Bing ads to trick Blockchain users into visiting phishing sites — but this time, the ads are using some crude social engineering ploys.

Searching for "blockchain" on bing.com currently displays the following pair of phishing ads at the top of the search results:

"Other ads are all phishing site" – click this one!
(Page requested at 12:15 BST, 2nd July 2014)

The first ad begs the user to "click this one" and warns that all other ads are phishing sites, but clicking on the ad actually sends the victim to a Blockchain phishing site, where he is prompted to enter his identifier and password. This phishing site is hosted in a subdirectory on a compromised website, which belongs to a web development outsourcing company in India.

Similarly, the second phishing ad warns that the other one is a phishing site; however, the fraudster behind this ad has made a mistake. When a victim clicks on this ad, it will try to send him to blockchain.lnfo (.LNFO). This link won't work because the .lnfo top-level domain does not exist, and probably never will, because as the fraudster has so perfectly demonstrated, it could easily be confused with .info.

As we saw in previous attacks, the green display URLs shown in these ads are carefully chosen by the fraudster to look similar to the real Blockchain website, which uses the blockchain.info domain. Neither of the display URLs accurately reflect the actual location reached after clicking on the ads. Also, the blue link text on the second ad uses an i-acute character in place of the "i" in Blockchain, presumably to make it harder to detect misuse of the Blockchain brand.

The fact that these phishing ads are trying to discredit each other suggests that there are multiple Bitcoin fraudsters competing for click-through traffic on sites which display Bing ads. These phishing ads also appear on other search engines which use the Yahoo Bing ad network, such as Yahoo and DuckDuckGo.

A phishing ad displayed on the privacy-conscious DuckDuckGo search engine.

Most Reliable Hosting Company Sites in June 2014

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 Datapipe FreeBSD 0:00:00 0.008 0.121 0.018 0.037 0.055
2 Netcetera Windows Server 2012 0:00:00 0.008 0.064 0.071 0.156 0.293
3 Pair Networks FreeBSD 0:00:00 0.008 0.223 0.081 0.165 0.560
4 Hosting 4 Less Linux 0:00:00 0.008 0.196 0.125 0.247 0.435
5 Hyve Managed Hosting Linux 0:00:00 0.012 0.241 0.063 0.125 0.128
6 Kattare Internet Services Linux 0:00:00 0.012 0.194 0.126 0.253 0.530
7 Logicworks Linux 0:00:00 0.019 0.146 0.075 0.154 0.314
8 krystal.co.uk Linux 0:00:00 0.019 0.140 0.091 0.178 0.178
9 Swishmail FreeBSD 0:00:00 0.023 0.135 0.073 0.146 0.194
10 Aspserveur Linux 0:00:00 0.031 0.309 0.087 0.439 0.791

See full table

Datapipe had the most reliable hosting company site in June, with only two isolated failed requests. This is Datapipe's third victory so far this year, and the company also achieved second place in May. Datapipe has accrued an outstanding 100% uptime record over the past eight years, and consistently exhibits very fast connections times, regularly being one of the fastest sites we monitor. The only other hosting company to have reached first place this year is Qube who did so three times, equalling Datapipe.

Netcetera came second in June, also with only two failed requests, giving it the most reliable Windows-based hosting company site. Netcetera has been in the hosting business since 1996 and offers a 99.9% uptime guarantee, although in practice its site actually reached 99.99% uptime over the past year and 99.96% over nine years.

Pair Networks had the third most reliable hosting company site in June. Like Datapipe, their website is served using FreeBSD. As well as hosting websites, Pair Networks recently hosted a Girl Develop It workshop in Pittsburgh, which is where their own custom-built data centres reside.

Netcetera had the only Windows-based hosting company site to appear in the top ten in June, while three sites used FreeBSD and the remaining six used Linux. Downtime is only recorded when all of Netcraft's performance monitors simultaneously record an outage, hence why it is still possible to achieve 100% uptime even if a site fails to respond to an individual performance monitor.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

Steam phishing attacks exploiting look-alike domain names

An ongoing series of phishing attacks against the Steam gaming community is making effective use of look-alike domains to trick users into surrendering their usernames and passwords. The fraudsters behind these attacks then attempt to bypass Steam's two-factor authentication with a malicious executable that is deceptively named SteamGuard.exe.

One of the many look-alike domains involved in the attacks against steamcommunity.com

Victims are being targeted through Steam's own chat client, giving fraudsters the opportunity to spear phish accounts which are known to contain valuable tradable items. Since the inception of Steam Trading, it has become easier to monetize stolen accounts by selling the victim's virtual items to other Steam users.

Fraudsters are using Steam's own chat client to lure victims to phishing sites.
These sites use deceptive domain names, designed to look similar to the real steamcommunity.com.

If a targeted Steam user is persuaded to click on one of these links, he will be taken to a fake Steam profile. The following example shows another of these fake profiles on a similar look-alike domain. Profiles used in these attacks may appear to offer rare or unusual tradable items, and the high level and displayed XP score lends some degree of trustworthiness to potential trades.

The fake profile offers some attractive items up for trade.

To further entice the victim into trading with the fraudster, the fake profile also includes fabricated feedback which enhances the fraudster's reputation as a fast and reliable trader.

However, the fraudster is not intending to trade any items with his victim — he instead wants to gain access to the victim's account, and then steal the victim's own tradable items. When the victim clicks on the "Add Friend" button, he will be presented with a spoofed login form on the look-alike domain that requests his Steam username and password:

The stolen username and password will not be of much use to the fraudster if the victim has enabled Steam Guard. This two-factor authentication mechanism is enabled by default if the victim has a verified email address and has restarted Steam at least twice since verifying the address. If Steam Guard is enabled, the fraudster will be unable to access his victim's Steam account without entering an access code which is emailed only to the victim.

Bypassing Steam's two-factor authentication (Steam Guard)

Older Steam phishing sites simply asked the victim for their access code, but this approach is no longer suitable for trade fraudsters: there is now a time-delay before the trading feature can be used from a new device, which gives the victim an opportunity to recover his compromised account before any items can be traded by the fraudster.

Steam phishing sites consequently evolved to ask their victims to upload a special ssfn file. This file is located in the victim's Steam folder and acts as an authentication key, so that after providing a valid access code, the user does not have to keep on requesting and entering a new access code every time they launch Steam. If this file is copied to the fraudster's computer, he will be able to bypass the two-factor authentication mechanism and gain access to the victim's account.

The Steam phishing sites used in these latest attacks have evolved further still. Rather than tricking the victim into uploading the ssfn file, the phishing sites now display the following dialog box which prompts the victim to install a "special tool":

Unsurprisingly, this special tool is actually malware designed to find and upload the victim's ssfn file to the fraudster. The SteamGuard.exe file used in this particular attack is hosted on Google Drive, and submits the victim's ssfn file to a hard-coded URL on the phishing site it was originally downloaded from.

After the fraudster has been furnished with the victim's username, password and ssfn file, he will be able to login to the account and begin trading immediately.

Constant stream of look-alike domains

Since the start of May, more than a hundred look-alike domains have been registered specifically for the purpose of Steam phishing. More than a third of these phishing sites have been hosted in Russia, and many of the domains have also been registered to individuals with Russian addresses and email addresses at yandex.ru, a free webmail service.

Some of the 100+ look-alike domain names that have been registered for Steam phishing since May.

Most of the domains used in these attacks have been registered under the .com top-level domain. One notable counterexample is steamcommunity.cm, which uses the country code top-level domain for Cameroon. As well as being used in spear phishing attacks via Steam's chat client, it is likely that this particular phishing site could also have also received typo-traffic from Steam users.

More generally, the .cm ccTLD offers tremendous typosquatting opportunities against any corresponding .com domain. The domain's operators received criticism in the past when it wildcarded the entire .cm domain. It no longer does this, but there is evidently nothing stopping fraudsters from registering a .com domain's corresponding .cm domain anyway.

Using an "unusual hat" as a lure to visit the steamcommunity.cm phishing site.

Monetizing stolen Steam accounts

Albrecht Neumann, a mathematics student in Germany, is an active Steam trader who has reported some of these phishing attacks to Netcraft. He suspects the fraudsters are automatically searching trading portals for people who are offering to sell expensive items, and are then sending messages to those users via Steam: Each time he "bumps" a thread in which he is offering expensive items, he gets up to five new friend requests.

Neumann told Netcraft that keys and earbuds are a primary target for trade fraudsters, as these items serve as a relatively stable currency in the Steam economy, and are easy to turn into real money. Earbuds are cosmetic items which can be worn by a player's in-game character, and were given away to Mac OS X users who played Team Fortress 2 during a limited time period in 2010; but now they can only be obtained through trading. Some users stockpile these items in the hope that their value might increase and earn them a profit further down the line. Such items are valuable by virtue of their rarity, and can often be sold for $30-$40 each, making some accounts worth thousands.

All of the domain names used in these attacks were very similar to the real steamcommunity.com domain. Netcraft's Fraud Detection service helps brand owners pre-emptively identify these types of fraudulent domain registrations. Some of the domains were registered months before the attacks actually took place, which would have allowed plenty of time to get them shut down before they were misused. Domain registrars are in a position to nip this in the bud even earlier — they can use Netcraft's Domain Registration Risk service to prevent their customers from registering domain names which are deceptively similar to well known phishing targets.