February 2015 Web Server Survey

In the February 2015 survey we received responses from 883,419,935 sites and 5,135,229 web-facing computers.

Microsoft showed the largest growth in terms of hostnames, with an additional 12 million sites taking its total up to 253 million. This has increased Microsoft's market share to 28.7%, but Apache continues to lead with a 38.8% share, despite a loss of 5.9 million sites.

Web-facing computer growth was fairly even across the board, with the top three server vendors all showing similar gains. nginx made the largest gain of just under 22,000 computers, while Microsoft and Apache each gained just over 20,000. This has resulted in nginx's market share growing slightly to 11.3%, but Apache maintains its comfortable lead with a 47.2% share, while Microsoft's stays at 29.9%.

Despite its impending lack of support, the number of hostnames using Microsoft IIS 6.0 grew by more than 5% this month; however, the number of web-facing computers using this platform fell by 2%. This version of IIS was released more than 10 years ago, alongside Windows Server 2003, both of which will reach the end of their Extended Support periods in July.

Several of the new generic top-level domains continue to show surprising growth. The number of sites using the .xyz TLD nearly doubled this month, and now totals more than 10 million. Strong growth was also seen by the .red TLD, which grew by nearly 3,000% to reach a total of 850,000. Other new colour-based gTLDs to have appeared in Netcraft's survey recently include .blue, .pink and .black; these are all run by Afilias, which also acts as the domain registry for other well-established TLDs such as .info, .mobi.

The .paris geographic TLD has shown a promising start by already reaching a total of 13,000 sites, outpacing growth seen by other new GeoTLDs which reached general availability around the same time. The .paris GeoTLD became available to all on 2 December 2014 and proclaims itself to be the most affordable address in Paris. The most visited .paris website is currently www.toureiffel.paris, which is where visitors will end up if they attempt to visit the Eiffel Tower's previous website at www.tour-eiffel.fr.

In January, Google added support for the Google Domains beta directly into Blogger, making it easier for users to purchase custom domain names for their blogs. Google has been an ICANN accredited domain registrar since 2005, allowing it to sell domain names under the most popular top-level domains such as .com, .net and .org, but it is also in the process of making a much larger range of new gTLDs available to the public under its role as a registry.

Google Registry is operated by Charleston Road Registry Inc, which is a wholly-owned subsidiary of Google. So far, it has launched three new TLDs: .みんな (which means "everyone" in Japanese), .soy (Spanish for "I am"), and most recently, .how. Google's other successful applications for gTLDs include .zip, .eat, .foo, .meme, and .new, but these are not yet available to register.

Google applied for more than 100 new gTLDs in total, costing it over $18M in ICANN application fees. Some of these applications were subsequently withdrawn, such as that for .and, which was not allowed as it corresponds to the ISO 3166-1 alpha-3 country code for Andorra. Many of the gTLDs that Google applied for also had other applicants competing for ownership, including Amazon in 21 cases.

Google and Amazon were the only applicants for the .dev gTLD, but Amazon withdrew its application after an assumed private deal or auction. Despite .dev being used by private domain names in some corporate development environments, the risk of name collisions was evidently deemed to be low enough to allow Google's application to succeed.

Total number of websites

Web server market share

DeveloperJanuary 2015PercentFebruary 2015PercentChange
Apache348,460,75339.74%342,480,92038.77%-0.97
Microsoft241,276,34727.52%253,484,22128.69%1.18
nginx128,083,92014.61%130,093,89914.73%0.12
Google20,209,6492.30%20,238,0572.29%-0.01
Continue reading

Amazon goes down in Europe

Some of Amazon's European retail sites and video streaming services went down last night, causing a flurry of complaints across social media. The affected sites included amazon.co.uk, amazon.de and amazon.fr.

p-3915.170

These outages are particularly notable, as Amazon has a considerable amount of experience hosting websites. It has one of the largest hosting infrastructures in the world, which is used not only by itself, but also by thousands of its Amazon Web Services customers.

Amazon is the world's largest hosting provider in terms of web-facing computers, accounting for more than 6% of the 5.1 million computers in Netcraft's February 2015 Web Server Survey. 52,000 of Amazon's web-facing computers are located in Ireland, which is where its European retail sites are hosted.

Amazon's presence in Ireland has grown astonishingly since Amazon Data Services Ireland opened the first of its three Irish EC2 Availability Zones in 2007. Remarkably, more than three-quarters of all web-facing computers in Ireland are now operated by Amazon, and these account for 2.7% of all web-facing computers in the Europe, Middle East and Africa region which it is designed to serve.

amazon-emea

Amazon's US site at www.amazon.com, which is hosted in the US, was not affected by last night's outages.

North Korean websites still barely reachable since Christmas

North Korea's presence on the internet has remained extremely patchy for more than a month, with little improvement since a suspected DDoS attack that took place just before Christmas.

The state-run Korean Central News Agency website at www.kcna.kp has been barely reachable since Christmas day. Only 13% of requests to the site succeeded during the past month, with the worst period being around the end of January when the site became completely unavailable for several days in a row from our network of performance monitors.

kcna-graph4

Although the articles on www.kcna.kp are written in multiple languages, the KCNA clearly acknowledges that North Korea has never been an ideal location to host material that is intended for global consumption — for greater dissemination, the agency continues to publish articles to a secondary site at www.kcna.co.jp, which is hosted at a much more reliable location in Japan.

Even so, both of these sites remain deliberately inaccessible from some parts of the world. Access to both has been blocked in South Korea, and addresses in New Zealand were blocked after scraping content to be used on the KCNA Watch website, which tracks North Korean media.

When they do succeed, most requests to www.kcna.kp are met with an HTTP 1.0 response, which renders as a blank page. These responses can take a few minutes to be received:

$ curl -i http://www.kcna.kp
HTTP/1.0 200 OK
Connection: Close
Pragma: no-cache
cache-control: no-cache
Refresh: 0.1
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/1999/REC-html401-19991224/strict.dtd">
<!-- <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd"> -->
<HTML>
<HEAD>
<META HTTP-EQUIV="Refresh" CONTENT="0.1">
<META HTTP-EQUIV="Pragma" CONTENT="no-cache">
<META HTTP-EQUIV="Expires" CONTENT="-1">
<TITLE></TITLE>
</HEAD>
<BODY><P></BODY>
</HTML>

Occasionally, www.kcna.kp will return its proper content in a HTTP 1.1 response which uses JavaScript to redirect the browser to http://www.kcna.kp/kcna.user.home.retrieveHomeInfoList.kcmsf, but this page — as well as all of the images and scripts it uses — suffers from similar performance issues, making the site practically unusable from many locations outside of North Korea.

When it is accessible, this is what kcna.kp looks like.

When it is accessible, this is what kcna.kp looks like.

Roughly half of the small number of websites hosted in North Korea, including www.kcna.kp, use Apache 2.2.15 running on the Linux-based Red Star 3.0 operating system. The Korea Computer Center (which also administers the .kp top-level domain) released this version of Red Star in 2013, but it was not until the end of last year that the rest of the world gained hands-on experience with it after an ISO image of the installation disk was distributed via bittorrent.

korelcfund.org2.kp

One of the sites using Apache 2.2.15 and Red Star 3.0 is the Korea Elderly Care Fund website at www.korelcfund.org.kp, which seemed to disappear completely for a few weeks after Christmas.

The rest of North Korea's websites are served by Apache running on CentOS, which is a free operating system derived from the sources of Red Hat Enterprise Linux. Websites using this platform in North Korea include the Korea National Insurance Corp site at www.knic.com.kp and the Committee for Cultural Relations with Foreign Countries at www.friend.com.kp, which ironically failed to respond to 84% of requests from our network of performance monitors.

Two years ago, Netcraft noted that kcna.kp used to run on Apache 2.2.3 with Red Hat Enterprise Linux 5. As this Linux distribution is owned, distributed and supported by an American multinational company, it is subject to U.S. export controls, which specifically prohibit its use in North Korea. As a result, this installation was likely unlicensed and so may not have received security updates, and would certainly not have received any official support.

North Korea normally has a very small presence on the internet, even when everything is working properly. Before the alleged attacks, Netcraft's Web Server Survey found 916 million websites around the globe, but only 24 of these sites were hosted in North Korea. To put that in perspective, you would have more chance of winning the UK's National Lottery jackpot than you would of randomly picking a North Korean website out of our survey.

Despite having an estimated population of 25 million people, North Korea has relatively few IP addresses of its own - just 1,024 in total. A third of the websites hosted in North Korea are served from a single IP address within this block, so a successful DDoS attack against this address is likely to take out several sites at once.

Hosted on an IP address assigned to North Korea, cqztjx.com is plastered with adverts for online gambling services.

Hosted on an IP address assigned to North Korea, cqztjx.com is plastered with adverts for online gambling services.

In addition to North Korea's 1,024 native IP addresses, a block of 256 IP addresses in the range 5.132.126.0 – 5.132.126.255 has also been assigned to an End User in North Korea. These addresses appear to be used solely for hosting online gambling websites on virtual private servers. This block is marked as ASSIGNED PA, which means it is not permanently allocated to North Korea; the range will be lost if the local issuing internet registry, Outside Heaven, terminates its services.

inetnum:        5.132.126.0 - 5.132.126.255
netname:        OUTSIDEHEAVEN_MUTI-IP_VPS
descr:          OUTSIDEHEAVEN_MUTI-IP_VPS infrastructure
country:        KP
admin-c:        OHS18-RIPE
tech-c:         OHS18-RIPE
status:         ASSIGNED PA

North Korea's other additional assigned network block at 77.94.35.0 – 77.94.35.255 does not currently appear to be used for hosting websites.

Most Reliable Hosting Company Sites in January 2015

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 Netcetera Windows Server 2012 0:00:00 0.004 0.061 0.082 0.162 0.162
2 Qube Managed Services Linux 0:00:00 0.015 0.100 0.037 0.076 0.076
3 XILO Communications Ltd. Linux 0:00:00 0.022 0.205 0.065 0.130 0.130
4 EveryCity SmartOS 0:00:00 0.022 0.080 0.066 0.133 0.133
5 Logicworks Linux 0:00:00 0.034 0.129 0.070 0.146 0.337
6 INetU Windows Server 2008 0:00:00 0.037 0.126 0.076 0.205 0.436
7 Host Europe Linux 0:00:00 0.041 0.121 0.070 0.168 0.169
8 New York Internet FreeBSD 0:00:00 0.045 0.220 0.036 0.075 0.200
9 Hivelocity Hosting Linux 0:00:00 0.049 0.147 0.111 0.221 0.221
10 Datapipe Linux 0:00:00 0.052 0.103 0.016 0.033 0.045

See full table

Netcetera had the most reliable hosting company site in January, with only a single failed request. Netcetera offers an SLA-backed 99.9% uptime guarantee on its services, and exceeds this promise on its own site, with an uptime of 99.97% over the previous year and 99.96% over ten years. The company offers dedicated, managed and colocation solutions based out of a data centre in the Isle of Man. Amongst other features, the data centre is carbon neutral, achieved via a combination of carbon offsetting and energy-saving technologies.

Qube Managed Services placed second in January, carrying over its excellent performance from 2014 where it placed in the top ten on eleven occasions, coming first four times. London-based Qube offers managed services out of data centres in London, New York and Zurich.

In third place, with 100% uptime and six failed requests, is XILO Communications. XILO offers services from shared hosting to dedicated servers out of its Maidenhead data centre in the south of England, as well as a corporate broadband service.

Linux remains a popular choice in terms of operating system, with six hosting companies sites served from Linux machines; Windows Server 2012, Windows Server 2008, SmartOS, and FreeBSD each make a single appearance.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

January 2015 Web Server Survey

In the January 2015 survey we received responses from 876,812,666 sites and 5,061,365 web-facing computers.

This is the lowest website count since last January, and the third month in a row which has seen a significant drop in the total number of websites. As was the case in the last two months, the loss was heavily concentrated at just a few hosting companies, and a single IP address that was previously hosting parked websites was responsible for over 50% of the drop.

Microsoft continues to be impacted most by the decline. Having overtaken Apache in the July 2014 survey their market share now stands at just 27.5%, giving Apache a lead of more than 12 percentage points.

Microsoft's decline seems far less dramatic when looking at the number of web-facing computers that use its server software. A net loss of 6,200 computers this month resulted in its computer share falling by only 0.28 percentage points, while Apache's went up by 0.18 to 47.5%.

These losses included many sites running on Microsoft IIS 6.0, which along with Windows Server 2003, will reach the end of its Extended Support period in July. Further abandonment of these platforms is therefore expected in the first half of this year, although Microsoft does offer custom support relationships which go beyond the Extended Support period.

Apache made an impressive gain of 22,000 web-facing computers this month. Half of this net growth can be attributed to the Russian social networking company V Kontakte, which hosts nearly 13,000 computers. Almost all of these were running nginx last month, but 11,000 have since defected to Apache, leaving less than 2,000 of V Kontakte's computers still using nginx.

OVH is still the second largest hosting company in terms of web-facing computers (although DigitalOcean is hot on its heels), but demand for its own relatively new .ovh top-level domain appears to be waning. Last month, we reported that the number of sites using the new .ovh TLD had shot up from 6,000 to 63,000. These sites were spread across just under 50,000 unique .ovh domains, and the number of domains grew by only 2,000 this month.

Only the first 50,000 .ovh domains were given away for free, while subsequent ones were charged at EUR 0.99. Despite being less than a third of the planned usual price of EUR 2.99, this shows how even a tiny cost can have a dramatic impact on slowing down the uptake in domain registrations.

Other new top-level domains which have shown early signs of strong hostname growth include .click, .restaurant, .help, .property, .top, .gifts, .quebec, .market and .ooo, each of which were almost non-existent last month but now number in their thousands.

The proliferation of new top level domains is evidently generating a lot of money for registrars and ICANN, but for some parties it has caused expenditure that was previously unnecessary. Take the new .hosting TLD for example: you would expect this domain to only be of interest to hosting companies, but US bank Wells Fargo has also registered some .hosting domains, including wellsfargo.hosting, wellsfargoadvisors.hosting and wellsfargohomemortgage.hosting. These domains are not used to serve any content, and instead redirect customers to Wells Fargo's main site at wellsfargo.com. The sole purpose of registering these domains appears to be to stop any other party from doing so, which protects the bank's brand and prevents the domains being used to host phishing sites.

In a similar move, Microsoft has also registered several .hosting domains including xbox.hosting, bing.hosting, windows.hosting, skype.hosting, kinect.hosting and dynamics.hosting. Browsing to any of these domains causes the user to be redirected to bing.com, which displays search results for the second-level string (i.e. "xbox", "windows", etc.).

Of course, with many other new TLDs continually popping up, brand protection becomes an increasingly costly exercise. Microsoft has also recently registered hundreds of other nonsensical domains which are used to redirect browsers to bing.com, such as lumia.ninja, lync.lawyer, xboxone.guitars, windowsphone.futbol, microsoft.airforce, azure.luxury, yammer.singles, xboxlive.codes, halo.tattoo, internetexplorer.fishing, and so on.

However, the race to register domain names is not always won by Microsoft — bing.click is a prime example of a domain that someone else got to first. This domain is currently offered for sale, highlighting the fact that it's not just ICANN and the registrars that stand to gain money from the influx of new TLDs.

Total number of websites

Web server market share

DeveloperDecember 2014PercentJanuary 2015PercentChange
Apache358,159,40539.11%348,460,75339.74%0.63
Microsoft272,967,29429.81%241,276,34727.52%-2.29
nginx132,467,76314.47%128,083,92014.61%0.14
Google20,011,2602.19%20,209,6492.30%0.12
Continue reading

Student Loans Company advice makes phishing easier

Anticipating a surge in phishing attacks over the festive period, the Student Loans Company warned students in Britain to be on the lookout for suspicious emails. Unfortunately, some of its anti-phishing advice could have backfired, potentially increasing the risk of students falling for phishing attacks.

Warning students to be on the lookout for fraudulent emails attempting to impersonate the SLC, it told The Telegraph and Money Saving Expert that any official correspondence would come from the email address notifications@slc.co.uk. However, this advice is rather dangerous because the slc.co.uk domain has not been configured to prevent spoof emails being sent from this address.

In particular, slc.co.uk does not have a Sender Policy Framework record. SPF rules can be used to describe who can send email on its behalf, and the lack of any policy means there are no restrictions on who can send emails appearing to come from notifications@slc.co.uk.

If students infer from the SLC's advice that all emails from notifications@slc.co.uk will be legitimate, then fraudsters will be able to carry out much more convincing phishing attacks simply by spoofing emails from this address.

The domain also lacks a DMARC record, which means the SLC cannot choose what happens to forged emails that appear to come from the slc.co.uk domain. If correctly configured, such emails could not only be blocked by some email providers, but SLC would also be able to view the contents of forged emails and receive statistics to see how many are being sent.

Preventing forged emails is an important part of mitigating phishing attacks, as many attacks are initiated via email. A typical phishing email will play on the victim's sense of urgency — for example, by warning the student that his next payment may be lost or delayed if he does not update his records at the fraudster's "secure" website that masquerades as a real SLC website.

Once the victim has been tricked into visiting the phishing site, he will be prompted to enter a multitude of information which can be used to steal the loan money as soon as it arrives in his bank account. Most student loan phishing sites blocked by Netcraft usually ask for far more information than a conventional online banking phishing attack would do, capturing not just the victim's bank account details and card number, but also details about the student's university course and term time address.

Frozen phish

Despite the SLC warning of an increase in phishing attacks, it is fortunate that the fraudsters instead put an unexpected freeze on their phishing activity over the Christmas holidays. In fact, not a single student loan phishing site has been blocked by Netcraft since before Christmas day.

During 2014, Netcraft blocked more than 180 phishing URLs that impersonated the Student Loans Company or the Student Finance England service (which is run by the SLC), while SLC's fraud team took down around 150 phishing sites. Over the past three years, it claims to have prevented almost £3 million being stolen.

student-loan-phishing

The calm before the storm that didn't happen: Most student loan phishing attacks occurred at the start of the academic year (September), and all of those carried out in December took place before Christmas day. Despite the second loan instalments being sent out this week, no attacks have taken place since.

New students make particularly attractive targets for fraudsters, as many will have no previous experience at managing their own finances. Research by the British Bankers' Association suggests that one in six of those aged 18 to 25 could be vulnerable to money transfer scams; a higher proportion than any other age group.

Student loans in the UK typically consist of a tuition fee loan – which can be up to £9,000 per year and is paid directly to a student's university or college – plus a maintenance loan of up to £7,751, which is paid into the student's own bank account, making the latter component an obvious target for phishing fraudsters.

Organisations concerned about email impersonation attacks can use Netcraft's Fraud Detection service, which processes DMARC (Domain-based Message Authentication, Reporting and Conformance) reports on your behalf. These reports are sent by ISPs and e-mail receivers when they see any emails which claim to be from one of your own domains. A web interface shows the status of all of your own domains, any configuration changes required, and highlights unprotected domains being used by fraudsters attacking your customers. Netcraft can also provide real time alerts of phishing sites targeting your company, and our takedown service can be used to remove phishing sites.