June 2016 Web Server Survey

In the June 2016 survey we received responses from 1,045,534,808 sites and 5,951,685 web-facing computers. This reflects an increase of 12 million sites, along with a modest gain of 4,700 computers.

Apache regained the lead from Microsoft this month, with a large increase of 60 million sites taking its total up to 360 million, while Microsoft lost 24 million. Microsoft enjoyed a brief foray at the top in April and May, thanks to a proliferation of link farming sites, but now stands 18 million sites behind Apache.

Apache's net growth included only 14 million new websites – the remainder consisted of existing websites that switched to Apache after previously using other web server software. Most notably, 52 million sites switched from Tengine to Apache, while 12 million switched from Microsoft. The number of websites using Tengine fell by more than 60%, largely as a result of the migration to Apache. Most of the sites involved in this switch were hosted by OVH in Canada, and not only changed server vendor, but also moved to a different hosting company—Data Foundry—in the United States.

Although the number of Tengine websites fell to 29 million, the number of active sites using Tengine actually increased slightly to 1.8 million. In a similar vein, the number of active sites running Apache fell by 630,000, even though the total number of sites grew by more than 60 million.

Out of the largest vendors, Microsoft has the lowest proportion of active sites, with only 4.8% of its 340 million sites being active, while Tengine's proportion has crept up to 6.3%. Both are significantly lower than Apache's proportion of 22.3% and nginx's 21.6%.

Newer versions of web server software generally attract a much higher proportion of active sites. For example, 56% of sites running on Microsoft IIS 10 (which will be included with Windows Server 2016, but is already available for Windows 10) are classified as active, while 23% of sites running IIS 8.5 are active, along with 14% of IIS 8.0 sites. This proportion dwindles to only 2.8% by the time we consider IIS 6.0, which remains a very popular choice of server in China despite no longer being supported by Microsoft.

nginx also demonstrates this trend, with the largest number of its active sites running on the latest 1.10.x stable branch. More than half of all sites using this version are active.

Across all versions, nginx continues to muscle its way into the market with confidence. This month it gained almost a million active sites, along with an additional 31,000 web-facing computers, giving it the largest growth in these important metrics. Conversely, Apache lost 26,000 computers, while Microsoft lost 4,500.

nginx has also continued to grow its presence amongst the top million websites, where it is now used by 27.6% of sites. Apache continues to lead with a 43.7% share of this market, although its share has generally been on the decline since 2011. If current trends continue, nginx could possibly take the lead from Apache within the next couple of years.

Total number of websites

Web server market share

DeveloperMay 2016PercentJune 2016PercentChange
Apache300,447,47029.06%360,458,01834.48%5.41
Microsoft366,964,00935.50%342,605,66632.77%-2.73
nginx163,902,97115.85%169,316,54716.19%0.34
Google21,567,2522.09%21,662,6732.07%-0.01
Continue reading

Most Reliable Hosting Company Sites in May 2016

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 Datapipe Linux 0:00:00 0.004 0.142 0.012 0.024 0.030
2 XILO Communications Ltd. Linux 0:00:00 0.008 0.206 0.064 0.127 0.127
3 GoDaddy.com Inc Linux 0:00:00 0.013 0.226 0.016 0.037 0.037
4 Anexia Linux 0:00:00 0.013 0.180 0.088 0.186 0.186
5 One.com Linux 0:00:00 0.017 0.201 0.037 0.108 0.108
6 dinahosting Linux 0:00:00 0.017 0.230 0.083 0.165 0.165
7 ServerStack Linux 0:00:00 0.025 0.112 0.064 0.129 0.129
8 Aspserveur Linux 0:00:00 0.025 0.301 0.077 0.327 0.484
9 LeaseWeb Linux 0:00:00 0.029 0.303 0.028 0.055 0.055
10 Webair Internet Development Linux 0:00:00 0.029 0.156 0.054 0.108 0.109

See full table

May is the second month in a row that Datapipe has had the most reliable hosting company site. Its own site once again has lived up to its 100% Network Uptime Guarantee, as has been the case during over 10 years of monitoring by Netcraft. Datapipe has 25 data centres globally including locations in key technology and financial hubs: New York, Silicon Valley, London, and Hong Kong.

In second place in May was XILO Communications Ltd with just two failed requests. XILO’s website has maintained 100% uptime over the past two years, and 99.990% since October 2011. XILO offers services from shared hosting to dedicated servers and operate from the UK.

GoDaddy reached third place in May, marking its twelfth consecutive appearance in the top 10. Since May 2015, GoDaddy has had excellent connection times ranging from 5-16ms. GoDaddy recently enhanced its offering to small businesses by acquiring FreedomVoice, a cloud VOIP provider, with the aim of accelerating the delivery of communications services.

During May, Anexia ranked fourth in our top 10 and had just three failed requests. It was narrowly beaten by GoDaddy who had the same number of failed requests but a faster connection time. Anexia also placed impressively in the Inc. 5000 Europe, ranking as the 517th fastest growing company in Europe.

For the second time in 2016, all of the top 10 hosting company websites were powered by Linux. The last time a site using Windows appeared in the top 10 was June 2015

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

May 2016 Web Server Survey

In the May 2016 survey we received responses from 1,033,790,346 sites and 5,946,961 web-facing computers. This reflects a gain of 147,000 computers, coupled with a loss of 49 million sites.

While last month's survey recorded the largest number of sites ever, many of the Chinese sites running Microsoft IIS that appeared last month have since disappeared. Combined with other departures, Microsoft suffered a net loss of 75 million sites this month, which has played a major part in its market share falling by more than 5 percentage points to less than 36%. Nevertheless, it is still the most common server vendor by number of sites, with a total of nearly 370 million hosted on IIS servers.

Despite Microsoft's loss of 75 million sites, the number of active sites using IIS actually grew by 450,000, which is indicative of the low quality of the sites it lost. Most of the lost sites were engaged in link farming activity, with large numbers of these sites being served from relatively few computers. The loss of these sites therefore had little impact on the number of web-facing computers using Microsoft IIS, which grew by 14,000.

Microsoft's closest competitor, Apache, gained 8.4 million sites, with its increased market share of 29.1% putting it within 6.4 percentage points of Microsoft's leading share.

Although it has yet to reach the same level as Microsoft and Apache, nginx made the largest gains, growing by 21 million sites and increasing its market share by 2.6 points to 15.9%.

nginx also showed the strongest growth in the survey's other metrics: it gained nearly 7.5 million active sites, 74,100 web-facing computers, and increased its presence within the top million sites by 16,000. The most significant of these gains was nginx's active site count increasing by a whopping 27%, largely as a result of Tumblr sites now exhibiting the Server: nginx header (in previous months, most Tumblr sites did not reveal which server software they were using).

While Microsoft has shaken off many of its low-quality sites, Alibaba's nginx fork, Tengine, gained around 10 million. Most of the new sites served by Tengine this month make use of domains under the .science gTLD, which has proved popular with many Chinese link farms and webspam sites – most likely due to the sub-dollar registration costs. Tengine suffered a small net loss in active sites this month, which corroborates the low quality of the 10 million new sites.

Only 2.4% of the sites served by Tengine now qualify as active sites, which highlights just how many of them are used for displaying automatically generated content. Microsoft is still also fairly popular with link farm operators (particularly in China), with only 4.6% of its sites showing active content. In contrast, more than 26% of Apache sites, and nearly 22% of nginx sites feature active content.

Total number of websites

Web server market share

DeveloperApril 2016PercentMay 2016PercentChange
Microsoft441,470,89440.75%366,964,00935.50%-5.26
Apache292,043,54826.96%300,447,47029.06%2.10
nginx143,349,43913.23%163,902,97115.85%2.62
Google20,597,6051.90%21,567,2522.09%0.18
Continue reading

Bangladesh government exporting live phish

Bangladesh is one of the world's largest producers of fish; but lately, its government has also become an inadvertent exporter of phish.

Over the past week, several phishing sites have popped up on Bangladeshi government websites, under the .gov.bd second-level domain. These fraudulent sites have been used in phishing attacks against customers of Wells Fargo bank, Google, AOL, and other email providers.

One of the phishing sites currently using a .gov.bd domain is hosted on a website belonging to the Bandarban Technical Training Center in Bangladesh. This site imitates Google Docs in an attempt to steal victims' email credentials, whichever mail providers they use.

One of the phishing sites currently using a .gov.bd domain is hosted on a website belonging to the Bandarban Technical Training Center in Bangladesh. The fraudulent content imitates Google Docs in an attempt to steal victims' email credentials, whichever mail providers they use.

Domain name registrations under .gov.bd are restricted to government-related entities in Bangladesh, although it is unlikely that the government is directly responsible for these attacks. As with most phishing sites, the fraudulent content has probably been placed on these government sites by remote hackers; nonetheless, this would make the Bangladesh government at least responsible for poor security.

The vast majority of websites under .gov.bd are hosted within Bangladesh, but the apparently-compromised server involved in these attacks is one of a few that are hosted in the United Kingdom, on a static IP address used by the hosting company Nibs Solutions. No Bangladeshi servers are currently serving phishing sites from .gov.bd domains.

After more than a week since this spate of phishing attacks started appearing on UK-hosted .gov.bd sites, none of the fraudulent content has been removed. The presence of multiple live phishing sites on the affected server, and the fact that the previous compromises have not yet been cleaned up, suggests that whatever security vulnerabilities might have affected the server are yet to be resolved.

Detected just over a week ago, the oldest phishing site in this spate of attacks targets Wells Fargo customers and remains accessible today on the Jessore Technical Training Center website at jessorettc.gov.bd. This training center was established by the Government of the People's Republic of Bangladesh in 2004, hence its eligibility to use the .gov.bd domain.

Detected just over a week ago, the oldest phishing site in this spate of attacks targets Wells Fargo customers and remains accessible today on the Jessore Technical Training Center website at jessorettc.gov.bd. This training center was established by the Government of the People's Republic of Bangladesh in 2004, hence its eligibility to use the .gov.bd domain.

Bangladesh has a relatively small presence on the web, with just over 30,000 websites making use of the entire .bd country code top-level domain. However, the ratio of phishing incidents to sites is quite high at roughly 1 in 100.

Users of the Netcraft anti-phishing extension are already protected from these attacks, including the examples shown above, even though the fraudulent content has not yet been removed by the sites' administrators.

Most Reliable Hosting Company Sites in April 2016

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 Datapipe Linux 0:00:00 0.000 0.160 0.012 0.024 0.031
2 Qube Managed Services Linux 0:00:00 0.000 0.153 0.058 0.117 0.117
3 CWCS Linux 0:00:00 0.000 0.189 0.070 0.142 0.143
4 Pair Networks FreeBSD 0:00:00 0.004 0.246 0.070 0.143 0.143
5 GoDaddy.com Inc Linux 0:00:00 0.009 0.257 0.010 0.024 0.025
6 XILO Communications Ltd. Linux 0:00:00 0.009 0.222 0.063 0.127 0.127
7 Kattare Internet Services Citrix Netscaler 0:00:00 0.009 0.517 0.114 0.228 0.228
8 LeaseWeb Linux 0:00:00 0.013 0.351 0.029 0.055 0.055
9 Hyve Managed Hosting Linux 0:00:00 0.013 0.222 0.060 0.120 0.120
10 Aspserveur Linux 0:00:00 0.013 0.343 0.077 0.326 0.471

See full table

Datapipe had the most reliable hosting company site in April, responding to all of Netcraft's requests. Datapipe's performance has seen it appear in the top ten every month so far in 2016, continuing a streak which has placed the company in the top ten 11 times in the past 12 months. Datapipe provides hosting services out of a number of data centres in Europe, Asia and North America. In April, Datapipe announced that it would be partnering with Singapore's largest electronics retailer to ensure the scalability of the latter's online infrastructure.

Qube had the second most reliable hosting company site. As with Datapipe, Qube's site responded to all of Netcraft's requests, but was fractionally slower to do so. Qube's performance is also consistent: the company has appeared in the top ten 9 times over the last 12 months.

CWCS achieved third place in April, also with a 100% response rate, albeit with an average connect time that was marginally slower than both Datapipe and Qube's. CWCS provides shared and managed hosting solutions, with data centre facilities in England and North America, and counts organisations such as KPMG and the University of York amongst its clients.

Linux is once again the most popular choice of operating system with hosting companies: eight out of the top ten companies in April hosted their websites on Linux machines. Kattare Internet Services and Pair Networks were the only two exceptions, using a Citrix Netscaler device and FreeBSD respectively.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

Hook, like and sinker: Facebook serves up its own phish

Fraudsters are abusing Facebook's app platform to carry out some remarkably convincing phishing attacks against Facebook users.

A phishing site displayed on the real Facebook website.

A phishing site displayed on the real Facebook website.

Masquerading as a Facebook Page Verification form, this phishing attack leverages Facebook's own trusted TLS certificate that is valid for all facebook.com subdomains. This makes the page appear legitimate, even to many seasoned internet users; however, the verification form is actually served via an iframe from an external site hosted by HostGator. The external website also uses HTTPS to serve the fraudulent content, so no warnings are displayed by the browser.

The phishing attack does not require the victim to be already logged in.

The phishing attack does not require the victim to be already logged in.

This phishing attack works regardless of whether the victim is already logged in, so there is little chance of a victim being suspicious of being asked to log in twice in immediate succession.

The source code of the phishing content reveals that it sends the stolen credentials directly to the fraudster's website.

The source code of the phishing content reveals that it sends the stolen credentials directly to the fraudster's website.

To win over anyone who remains slightly suspicious, the phishing site always pretends that the first set of submitted credentials were incorrect. A suspicious user might deliberately submit an incorrect username and password in order to test whether the form is legitimate, and the following error message could make them believe that the credentials really are being checked by Facebook.

The phishing site always pretends the first submitted credentials are incorrect.

The phishing site always pretends the first submitted credentials are incorrect. Note that it now also asks for the victim's date of birth.

Those who were slightly suspicious might then believe it is safe to enter their real username and password. Anyone else who had already entered the correct credentials would probably just think they had made a mistake and try again. After the second attempt, the phishing site will act as if the correct credentials had been submitted:

On the second attempt, the phishing site will ask the victim to wait up to 24 hours.

On the second attempt, the phishing site will ask the victim to wait up to 24 hours.

The final response indicates that the victim will have to wait up to 24 hours for their submission to be approved. Without instant access to the content they were trying to view, the victim will probably carry on doing something else until they receive the promised email notification.

But of course, this email will never arrive. By this point, the fraudster already has the victim's credentials and is just using this tactic to buy himself some time. He can either use the stolen Facebook credentials himself, or sell them to others who might monetize them by posting spam or trying to trick victims' friends into helping them out of trouble by transferring money. If more victims are required, then the compromised accounts could also be used to propagate the attack to thousands of other Facebook users.

Some of Facebook's security settings.

Some of Facebook's security settings.

However, Facebook does provide some features that could make these attacks harder to pull off. For example, if login alerts are enabled, the victim will be notified that their account has been logged into from a different location – this might at least make the victim aware that something untoward is going on. Although not enabled by default, users can completely thwart this particular attack by activating Facebook's login approvals feature, which requires a security code to be entered when logging in from unknown browsers. Only the victim will know this code, and so the fraudster will not be able to log in.