January 2015 Web Server Survey

In the January 2015 survey we received responses from 876,812,666 sites and 5,061,365 web-facing computers.

This is the lowest website count since last January, and the third month in a row which has seen a significant drop in the total number of websites. As was the case in the last two months, the loss was heavily concentrated at just a few hosting companies, and a single IP address that was previously hosting parked websites was responsible for over 50% of the drop.

Microsoft continues to be impacted most by the decline. Having overtaken Apache in the July 2014 survey their market share now stands at just 27.5%, giving Apache a lead of more than 12 percentage points.

Microsoft's decline seems far less dramatic when looking at the number of web-facing computers that use its server software. A net loss of 6,200 computers this month resulted in its computer share falling by only 0.28 percentage points, while Apache's went up by 0.18 to 47.5%.

These losses included many sites running on Microsoft IIS 6.0, which along with Windows Server 2003, will reach the end of its Extended Support period in July. Further abandonment of these platforms is therefore expected in the first half of this year, although Microsoft does offer custom support relationships which go beyond the Extended Support period.

Apache made an impressive gain of 22,000 web-facing computers this month. Half of this net growth can be attributed to the Russian social networking company V Kontakte, which hosts nearly 13,000 computers. Almost all of these were running nginx last month, but 11,000 have since defected to Apache, leaving less than 2,000 of V Kontakte's computers still using nginx.

OVH is still the second largest hosting company in terms of web-facing computers (although DigitalOcean is hot on its heels), but demand for its own relatively new .ovh top-level domain appears to be waning. Last month, we reported that the number of sites using the new .ovh TLD had shot up from 6,000 to 63,000. These sites were spread across just under 50,000 unique .ovh domains, and the number of domains grew by only 2,000 this month.

Only the first 50,000 .ovh domains were given away for free, while subsequent ones were charged at EUR 0.99. Despite being less than a third of the planned usual price of EUR 2.99, this shows how even a tiny cost can have a dramatic impact on slowing down the uptake in domain registrations.

Other new top-level domains which have shown early signs of strong hostname growth include .click, .restaurant, .help, .property, .top, .gifts, .quebec, .market and .ooo, each of which were almost non-existent last month but now number in their thousands.

The proliferation of new top level domains is evidently generating a lot of money for registrars and ICANN, but for some parties it has caused expenditure that was previously unnecessary. Take the new .hosting TLD for example: you would expect this domain to only be of interest to hosting companies, but US bank Wells Fargo has also registered some .hosting domains, including wellsfargo.hosting, wellsfargoadvisors.hosting and wellsfargohomemortgage.hosting. These domains are not used to serve any content, and instead redirect customers to Wells Fargo's main site at wellsfargo.com. The sole purpose of registering these domains appears to be to stop any other party from doing so, which protects the bank's brand and prevents the domains being used to host phishing sites.

In a similar move, Microsoft has also registered several .hosting domains including xbox.hosting, bing.hosting, windows.hosting, skype.hosting, kinect.hosting and dynamics.hosting. Browsing to any of these domains causes the user to be redirected to bing.com, which displays search results for the second-level string (i.e. "xbox", "windows", etc.).

Of course, with many other new TLDs continually popping up, brand protection becomes an increasingly costly exercise. Microsoft has also recently registered hundreds of other nonsensical domains which are used to redirect browsers to bing.com, such as lumia.ninja, lync.lawyer, xboxone.guitars, windowsphone.futbol, microsoft.airforce, azure.luxury, yammer.singles, xboxlive.codes, halo.tattoo, internetexplorer.fishing, and so on.

However, the race to register domain names is not always won by Microsoft — bing.click is a prime example of a domain that someone else got to first. This domain is currently offered for sale, highlighting the fact that it's not just ICANN and the registrars that stand to gain money from the influx of new TLDs.

Total number of websites

Web server market share

DeveloperDecember 2014PercentJanuary 2015PercentChange
Continue reading

Student Loans Company advice makes phishing easier

Anticipating a surge in phishing attacks over the festive period, the Student Loans Company warned students in Britain to be on the lookout for suspicious emails. Unfortunately, some of its anti-phishing advice could have backfired, potentially increasing the risk of students falling for phishing attacks.

Warning students to be on the lookout for fraudulent emails attempting to impersonate the SLC, it told The Telegraph and Money Saving Expert that any official correspondence would come from the email address notifications@slc.co.uk. However, this advice is rather dangerous because the slc.co.uk domain has not been configured to prevent spoof emails being sent from this address.

In particular, slc.co.uk does not have a Sender Policy Framework record. SPF rules can be used to describe who can send email on its behalf, and the lack of any policy means there are no restrictions on who can send emails appearing to come from notifications@slc.co.uk.

If students infer from the SLC's advice that all emails from notifications@slc.co.uk will be legitimate, then fraudsters will be able to carry out much more convincing phishing attacks simply by spoofing emails from this address.

The domain also lacks a DMARC record, which means the SLC cannot choose what happens to forged emails that appear to come from the slc.co.uk domain. If correctly configured, such emails could not only be blocked by some email providers, but SLC would also be able to view the contents of forged emails and receive statistics to see how many are being sent.

Preventing forged emails is an important part of mitigating phishing attacks, as many attacks are initiated via email. A typical phishing email will play on the victim's sense of urgency — for example, by warning the student that his next payment may be lost or delayed if he does not update his records at the fraudster's "secure" website that masquerades as a real SLC website.

Once the victim has been tricked into visiting the phishing site, he will be prompted to enter a multitude of information which can be used to steal the loan money as soon as it arrives in his bank account. Most student loan phishing sites blocked by Netcraft usually ask for far more information than a conventional online banking phishing attack would do, capturing not just the victim's bank account details and card number, but also details about the student's university course and term time address.

Frozen phish

Despite the SLC warning of an increase in phishing attacks, it is fortunate that the fraudsters instead put an unexpected freeze on their phishing activity over the Christmas holidays. In fact, not a single student loan phishing site has been blocked by Netcraft since before Christmas day.

During 2014, Netcraft blocked more than 180 phishing URLs that impersonated the Student Loans Company or the Student Finance England service (which is run by the SLC), while SLC's fraud team took down around 150 phishing sites. Over the past three years, it claims to have prevented almost £3 million being stolen.


The calm before the storm that didn't happen: Most student loan phishing attacks occurred at the start of the academic year (September), and all of those carried out in December took place before Christmas day. Despite the second loan instalments being sent out this week, no attacks have taken place since.

New students make particularly attractive targets for fraudsters, as many will have no previous experience at managing their own finances. Research by the British Bankers' Association suggests that one in six of those aged 18 to 25 could be vulnerable to money transfer scams; a higher proportion than any other age group.

Student loans in the UK typically consist of a tuition fee loan – which can be up to £9,000 per year and is paid directly to a student's university or college – plus a maintenance loan of up to £7,751, which is paid into the student's own bank account, making the latter component an obvious target for phishing fraudsters.

Organisations concerned about email impersonation attacks can use Netcraft's Fraud Detection service, which processes DMARC (Domain-based Message Authentication, Reporting and Conformance) reports on your behalf. These reports are sent by ISPs and e-mail receivers when they see any emails which claim to be from one of your own domains. A web interface shows the status of all of your own domains, any configuration changes required, and highlights unprotected domains being used by fraudsters attacking your customers. Netcraft can also provide real time alerts of phishing sites targeting your company, and our takedown service can be used to remove phishing sites.

Moonpig breach highlights need for app and API testing

A severe vulnerability in the API used by Moonpig's Android app has highlighted the need for organisations to apply greater scrutiny to the security of their apps and endpoints. Through its apps and website, the custom greetings card company sends out more than 12 million cards every year and turned over £53 million last year.

By enumerating an easily predictable sequence of user ID numbers, anyone could retrieve various information about millions of Moonpig customers, including names, addresses, and some credit card details. Because there was no authentication mechanism for the API, an attacker could also have placed orders on other customers' accounts.

Unlike with traditional web applications, much of what goes on beneath the glossy facade of an app is hidden from the user — but with the right tools and the right knowledge, it can be trivial to identify and exploit any vulnerabilities that might affect it. The Moonpig vulnerability exemplifies this, as the problem was not only easy to spot, but could be exploited simply by pasting a modified URL into a standard web browser.

The Moonpig vulnerability stemmed from the fact that the API trusted data sent from the app, without considering that it could have been altered or fabricated by a malicious party. This type of vulnerability fundamentally compromises the security of the application and the data it handles, and would likely be quickly identified in a third-party security test of the API.

The danger posed by this vulnerability was compounded by Moonpig's failure to react promptly — Moonpig purportedly knew about this issue 17 months ago after it was reported by one of its own customers. However, Moonpig failed to shut down or fix the vulnerable service until after the vulnerability was publicly disclosed last night.

Moonpig issued the following statement on its website today:

You may have seen reports this morning about our Apps and the security of customer details when shopping with Moonpig. We can assure our customers that all password and payment information is and has always been safe. The security of your shopping experience at Moonpig is extremely important to us and we are investigating the detail behind today's report as a priority. As a precaution, our Apps will be unavailable for a time whilst we conduct these investigations and we will work to resume a normal service as soon as possible. The desktop and mobile websites are unaffected.

Netcraft offers Mobile App Security Testing services and traditional Web Application Security Testing, both of which include testing of relevant APIs and other endpoints that may be commonly overlooked. Contact us at security-sales@netcraft.com to discuss your requirements.

Most Reliable Hosting Company Sites in December 2014

Rank Performance Graph OS Outage
DNS Connect First
1 Qube Managed Services Linux 0:00:00 0.007 0.097 0.037 0.075 0.075
2 Datapipe Linux 0:00:00 0.011 0.104 0.017 0.034 0.047
3 Anexia Linux 0:00:00 0.011 0.430 0.088 0.195 0.216
4 CWCS Linux 0:00:00 0.011 0.226 0.099 0.181 0.183
5 iWeb Linux 0:00:00 0.019 0.132 0.079 0.156 0.156
6 www.choopa.com Linux 0:00:00 0.022 0.150 0.073 0.157 0.203
7 ServerStack Linux 0:00:00 0.022 0.087 0.079 0.156 0.156
8 LeaseWeb Linux 0:00:00 0.026 0.170 0.026 0.058 0.058
9 Bigstep Linux 0:00:00 0.026 0.126 0.060 0.122 0.122
10 krystal.co.uk Linux 0:00:00 0.026 0.149 0.062 0.141 0.141

See full table

Qube Managed Services had the most reliable hosting company website in December, with just two failed requests. London-based Qube has been in the top 10 for all but one month in 2014, and reached first place six times. Qube offers a range of managed services, including colocation, cloud hosting and dedicated servers. It has data centres in London, New York and Zurich and focuses on providing reliable and high-quality solutions to its clients.

Datapipe follows closely in second place, with three failed requests. Datapipe, whose website has over 8 years of continuous uptime, was recently named a leader in hosted private cloud solutions by a Forrester Research report. Stratosphere, Datapipe's hosted private cloud solution, supports multiple hypervisors and has data centres in seven locations spread throughout the world.

Anexia, which also had three failed requests but a higher average connection time, came in third place this month. Anexia last entered the top ten in July 2014 when it took first place. It was recently named by Deloitte as one of the fastest-growing Austrian technology companies of 2014, marking the second year in a row that it has appeared in Deloitte's Technology Fast 500 EMEA list.

CWCS closely follows Anexia in fourth place, having the same number of failed requests but a slightly higher average connection time. UK-based CWCS last featured in the top ten in August 2013.

Linux remains the most popular choice of operating system, with every hosting company website in December's top ten served by Linux machines. Notably, Datapipe, whose website previously used FreeBSD, switched to Linux in November.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

Proxy auto-config attacks defeat 2-factor auth, hide using country specific content

Fraudsters have been using proxy auto-config (PAC) scripts to steal online banking credentials for several years, but as with most phishing techniques, it is inevitable for these attacks to evolve and become more effective. The latest spate of PAC attacks has achieved this by using geolocation technology to evade detection and select which targets to attack.


PAC attacks typically channel online banking traffic through rogue proxy servers, allowing fraudsters to gobble up unencrypted usernames and passwords when forms are submitted, or to hijack already-authenticated sessions by stealing session cookies. Being able to view and modify this traffic also allows two-factor authentication mechanisms such as one-time passwords to be easily defeated.

PAC scripts used in these attacks inevitably look suspicious, which highlights the fact that fraud is taking place. Consequently, it is in the fraudster's interest to stop these scripts being found by law enforcement agencies, or indeed anyone else who might be tasked with investigating or preventing the fraud.

The latest attacks use a PAC script which is hosted on a web server in the Netherlands. This server has been configured to refuse TCP connections from certain countries or locations, which could be sufficient to put an investigator off the scent – if the server simply does not appear to exist, they may not bother investigating further. Meanwhile, the remaining unblocked users will continue to fall victim to the PAC attack.

Where the server can be accessed, geolocation is also used to customise the contents of the PAC script. For example, a completely benign PAC script is returned to clients in Australia, which simply tells the victim's browser to connect directly to all websites; no proxying takes place:

Deobfuscated JavaScript from the benign PAC script

Deobfuscated JavaScript from the benign PAC script

Conversely, requesting the PAC script from Japan causes the following JavaScript to be returned:

PAC attack against Japanese banking customers (contents  deobfuscated for clarity)

PAC attack against Japanese banking customers (contents deobfuscated for clarity)

The FindProxyForURL function specifies which hostnames should be proxied through the fraudster's server. Anyone using this proxy script will be giving the fraudster an opportunity to observe or modify all unencrypted traffic flowing between his browser and each of the specified Japanese online banking websites.

If the victim browses to a site which does not match any of these patterns, his browser will not use the proxy and instead make a direct connection to be site. This serves to reduce the load on the fraudster's proxy server, as well as reducing the likelihood of the victim noticing something is awry. For example, if the victim performs a Google search for "what is my ip?", his browser will connect directly to google.com, causing Google to display the victim's own IP address rather than that of the fraudster's proxy.

Although online banking sites are the clear targets of these attacks, it is notable that many of these scripts, including the Japanese example, also target Facebook. The following PAC script is returned to clients in Switzerland, and proxies traffic destined for *.facebook.com, as well as several Swiss banking websites.


It was not apparent why Facebook is being targeted among these banks, but compromised Facebook accounts could be useful for propagating the malicious proxy scripts to other users. For example, users could be tricked into manually editing their proxy settings by following instructions posted from a trusted friend's compromised account, or other social engineering tricks to get the user to download and run malware.

This PAC attack is still active, with Japan and Switzerland being targeted by distinct malicious scripts. Most locations are unable to connect to the Dutch PAC script server, apart from Australia and Poland, which receive an identical benign script which does not proxy any web traffic.


Poor web application security can contribute significantly to the success of these proxy-based attacks. For instance, if the session cookies used on a bank's HTTPS website are not marked with the Secure attribute, then they will be transmitted unencrypted through the fraudster's proxy if the victim subsequently makes an HTTP request to the same hostname. Such attacks are much less likely to succeed if the targeted HTTPS site uses HTTP Strict Transport Security (HSTS) to prevent the connection being downgraded to HTTP.

Netcraft's Web Application Security Testing service can identify sites that are readily vulnerable to these types of attack. Banks and other organisations can also use Netcraft's takedown service to remove malicious proxy scripts and phishing sites from the internet, while infrastructure providers can use our phishing site feed to protect their users. For more information, please contact sales@netcraft.com.

December 2014 Web Server Survey

In the December 2014 survey we received responses from 915,780,262 sites and 5,034,578 web-facing computers.

This is the second month in a row where there has been a large drop in the total number of websites, giving this month the lowest count since January. As was the case in November, the loss has been concentrated at just a small number of hosting companies, with the ten largest drops accounting for over 52 million hostnames. The active sites and web facing computers metrics were not affected by the loss, with the sites involved being mostly advertising linkfarms, having very little unique content. The majority of these sites were running on Microsoft IIS, causing it to overtake Apache in the July 2014 survey. However the recent losses have resulted in its market share dropping to 29.8%, leaving it now over 10 percentage points behind Apache.

Despite losing more than six million hostnames this month, nginx outpaced all other major server vendors by gaining 22,300 web-facing computers. nginx is now used by nearly 11% of all web-facing computers – twice the share that it had two years ago.

Overall, the total number of web-facing computers in our survey increased by just over 40,000 this month, making nginx responsible for more than half the increase. Despite an increase of over 11,000 computers for Apache, and 1,700 for IIS, both continue to lose market share.

Thanks to continued strong growth at Amazon Web Services, Amazon is the largest hosting company by a considerable margin in terms of our web-facing computers metric (which includes web-facing virtual machines, providing that each has its own kernel and IP address). With nearly 300,000 web-facing computers in total, Amazon has just over twice as many as second-place OVH. In October, we reported that DigitalOcean had become the 4th largest hosting company in under 2 years, but it quickly reached third place in November and is continuing to close the gap on OVH.

Cloud growth

Both Amazon Web Services and Microsoft Azure expanded their cloud hosting footprints recently. Amazon opened a new European AWS region in Frankfurt, which augments its existing EU region in Ireland. Besides being able to host services closer to the center of Europe, the new region means that customers can now build multi-region applications with the assurance that their data will stay within the EU. The new Frankfurt region houses two EC2 availability zones and three AWS edge locations.

Microsoft's new Azure "geo" is in Australia, and consists of two geographically redundant regions in New South Wales and Victoria. This will help Microsoft to compete with Amazon's two EC2 availability zones in Sydney.

New TLDs

More new top-level domains showing strong growth in this month's survey include .nyc, which is targeted for use by New Yorkers, and .realtor, which is only allowed to be used by members of the National Association of Realtors or the Canadian Real Estate Association. These have grown from virtually nothing to a total of 40,000 and 80,000 sites respectively.

Ironically, one of this month's fastest growing new top-level domains started off as an April Fool's Day joke in 2009, when the founder of OVH announced the creation of the .ovh TLD – years before such things were actually possible. This joke resulted in over 22,000 requests to register .ovh domains within a few hours, demonstrating the potential demand for such domains. OVH eventually entered into a Registry Agreement with ICANN in January 2014, and the sunrise period for the .ovh TLD began in September. This month's survey saw the number of sites using .ovh domains grow from 6,000 to 63,000, likely due to the first 50,000 .ovh domains being given away for free, and with subsequent growth being fuelled by attractive pricing: new .ovh domains can currently be registered for only EUR 0.99 per year and renewed for EUR 1.99.

Total number of websites

Web server market share

DeveloperNovember 2014PercentDecember 2014PercentChange
Continue reading