June 2014 Web Server Survey

In the June 2014 survey we received responses from 968,882,453 sites, six million less than last month. The battle between Microsoft and Apache heated up this month, with Apache losing 13 million sites and Microsoft gaining 26 million. The resultant changes in market share have left Apache barely clinging onto the lead — Microsoft is now only 0.15 percentage points behind. This is the closest Microsoft has ever been, giving it a good chance of taking the lead for the first time next month. However, Apache continues to dominate in terms of active sites, i.e. sites which are actively managed by humans rather than being automatically generated for use in activities such as link farming and domain squatting. Under this metric, Apache's losses were less significant, still leaving it with more than half of the market share, and more than 36 percentage points ahead of its closest competitor, nginx. In terms of all websites, nginx suffered the second largest loss of 8.6 million sites. nginx is very often used as a reverse proxy, although other web servers can also fulfill this role. Apache's mod_proxy module allows it to be configured as either a forward or reverse proxy, and Microsoft IIS can be configured to act as a reverse proxy with the URL Rewrite and Application Request Routing modules. Microsoft Azure Web Sites can also achieve the same functionality once the proxy feature has been enabled. Tengine, which is based on nginx, also fell by three million sites this month. This web server software is used extensively by its originators, Taobao, and has been an open source project since 2011. Tengine supports all of the features of nginx 1.4.7, plus some additional features which are not present in the stable releases of nginx 1.4.x or 1.6.x, such as syslog and pipe support. However, the most recent mainline version (nginx 1.7.1), which was released on 27 May, does now allow the error_log and access_log directives to be logged to syslog.

IPv4 addresses nearing total exhaustion

On 20 May, ICANN announced that it had begun the process of allocating the remaining blocks of IPv4 addresses to the five Regional Internet Registries (RIRs). As the total number of available 32-bit IPv4 addresses dwindles, network operators are being encouraged to adopt the use of 128-bit IPv6 addresses, which will allow a significantly larger number of unique addresses: IPv4 can only provide 4.3 billion addresses, whereas IPv6 can provide nearly 8×1028 times as many. Unfortunately, adoption of IPv6 is proving to be a slow process. Only 3% of the hostnames in this month's survey can be resolved to IPv6 addresses, and the total number of IPv6 addresses used by websites has increased by only 18% over the past 12 months. ICANN's statement says the process of allocating the remaining blocks was triggered when Latin America and Caribbean Network Information Centre's (LACNIC) supply of IPv4 addresses dropped to below 8 million. Topically, this month's survey saw ICANN's website at www.icann.org change its Server banner from Apache to BigIP. For the past few years, it had either been "Apache" or "Apache/2.2.3 CentOS", although the operating system has consistently been identified as F5 BIG-IP throughout. Adobe's community forums at forums.adobe.com also switched to BigIP this month, from Apache-Coyote/1.1.
 
 
Developer May 2014 Percent June 2014 Percent Change
Apache 366,262,346 37.56% 353,672,431 36.50% -1.05
Microsoft 325,854,054 33.41% 352,208,487 36.35% 2.94
nginx 142,426,538 14.60% 133,763,494 13.81% -0.80
Google 20,685,165 2.12% 20,192,595 2.08% -0.04
Continue reading

Most Reliable Hosting Company Sites in May 2014

Rank Performance Graph OS Outage
hh:mm:ss
Failed
Req%
DNS Connect First
byte
Total
1 Qube Managed Services Linux 0:00:00 0.004 0.113 0.039 0.080 0.080
2 Datapipe FreeBSD 0:00:00 0.011 0.113 0.018 0.037 0.055
3 EveryCity SmartOS 0:00:00 0.015 0.100 0.066 0.133 0.133
4 Dinahosting Linux 0:00:00 0.015 0.259 0.091 0.182 0.182
5 Aspserveur Linux 0:00:00 0.019 0.289 0.085 0.413 0.750
6 Hosting 4 Less Linux 0:00:00 0.019 0.198 0.129 0.254 0.447
7 ServerStack Linux 0:00:00 0.022 0.085 0.076 0.151 0.151
8 Hyve Managed Hosting Linux 0:00:00 0.026 0.264 0.077 0.145 0.169
9 Pair Networks FreeBSD 0:00:00 0.026 0.231 0.082 0.167 0.571
10 Logicworks Linux 0:00:00 0.030 0.162 0.072 0.148 0.304

See full table

Qube had the most reliable hosting company site in May, with only one failed request. London-based Qube has performed remarkably well so far this year, fitting in with its vision to be the most reliable and trusted managed hosting company in the industry. As well as coming first three times so far this year, Qube also narrowly missed out on another first place in January.

With only three failed requests, Datapipe had the second most reliable hosting company site in May. Datapipe has also performed well this year, achieving first place results in both January and March; so far this year, only Qube and Datapipe have achieved first place. Over the past eight years, Datapipe has racked up an impressive 100% uptime record, and 99.994% since Netcraft started monitoring its website in June 2003 (downtime is only recorded when all of Netcraft's performance monitors simultaneously record an outage).

In third place, with four failed requests, was EveryCity, which has only been monitored by Netcraft since April. EveryCity started more than six years ago and its offices have been based near London's Tower Bridge ever since. Its primary datacenter is powered by 100% renewable energy and it offers various products and services, including public and private cloud hosting, dedicated servers, domain names, SSL certificate management, disaster recovery and content delivery.

The Linux operating system was used by seven of May's top ten hosting company websites, while two used FreeBSD. www.everycity.co.uk runs on SmartOS, which combines the ZFS file system, DTrace dynamic tracing, kernel-based virtual machines and Solaris Zones operating system-level virtualisation into a single operating system based on a community fork of OpenSolaris.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

National Crime Agency “urgent alert” site knocked offline

With only two weeks until the recently seized Gameover Zeus botnet is likely to be functioning again, the UK's National Crime Agency has published urgent advice on how to protect computers against the Gameover Zeus and CryptoLocker trojans.

Unfortunately, the page hosting this urgent advice is proving rather troublesome to view:

GetSafeOnline, Offline

When it can be viewed, the NCA's advice page at www.getsafeonline.org/nca/ outlines the threat and lists a set of tools which can be used to check for the presence of malware. The page also notes that the NCA "cannot over-stress the importance of taking these steps immediately" and "You must follow the advice on this page straight away".

With expectations of high traffic and the need for users to act immediately, it is surprising that this important information was not hosted on a platform which was capable of handling the load. Last night's tweet by @GetSafeOnline suggests that the performance issues are being caused by lots of traffic; there are no indications of an attack against the site.

Reverse DNS lookups for the www.getsafeonline.org hostname resolve to 170-203-253-62.static.virginm.net,
and the final hop in a traceroute is spc1-barn6-0-0-cust460.asfd.broadband.ntl.com

The FBI believes the Gameover Zeus trojan is responsible for more than one million computer infections, resulting in financial losses in the hundreds of millions of dollars. A Russian man believed to be involved in these attacks has been added to the FBI's Cyber's Most Wanted list.

The NCA announced the urgent alert on Facebook yesterday, prompting a stream of comments about the site not working

Since referring to the NCA's advice page in an article yesterday, the BBC's Dave Lee has mirrored the content on evernote so others can see it.

Ask.fm users being redirected to malware sites

Malicious adverts displayed on the Ask.fm website have been automatically redirecting users to malware sites, where they are prompted to install unwanted or malicious software under the pretense of Java and Flash Player updates.

This particular advert is benign and serves only as an example of the banner's placement

Ask.fm is a popular social network which allows its users to receive and answer anonymous questions, but both registered users and anonymous question askers are being put at risk by some of the adverts it displays: Merely viewing a user's profile on Ask.fm caused some users to be redirected to the following page, which claimed that an outdated Java plugin had been detected (even when Java had been disabled).

Rather than downloading a Java update, victims will instead end up installing a program which several anti-virus vendors identify as DomaIQ. This is an advertising platform used by adware and other malicious programs to display unwanted pop-up ads within Internet Explorer, Firefox and Google Chrome.

The rogue advert responsible for performing the redirection was initially served through ADTECH GmbH, which is a wholly-owned subsidiary of AOL. However, the trail does not end there – the framed content served by ADTECH subsequently requested several pages from AppNexus servers at ib.adnxs.com and ams1.ib.adnxs.com, before one of these pages initiated a request to a Java servlet on exchange.admailtiser.com. Finally, this servlet page caused the parent frame to be redirected from Ask.fm to the page on www.updriong.com, essentially taking the browser to a different website without requiring any user interaction.

After returning to the Ask.fm website, another rogue advert immediately redirected the browser to a fake Adobe Flash update site. Again, no user interaction was required – the chain of requests initiated by the third party advert automatically redirected the user's browser to the fake site hosted in Sweden.

In this case, the rogue advert on http://ask.fm/account/wall was again initially served by ADTECH, but the framed content made its next request to a Yahoo ad server (ads.yahoo.com), which in turn made a request to ad.copa-media.com, which itself made a request for content hosted on an AppNexus server at ams1.ib.adnxs.com.

Finally, a request to another AppNexus server at ib.adnxs.com resulted in the user's browser being redirected to the fake Adobe Flash update site at download.adoocobo.us. The setup.exe file is served from a domain which is known for propagating malware.

Mobile browsers have also been targeted by similar attacks on Ask.fm. The example below shows an Ask.fm webpage displaying an intrusive and unsolicited alert dialog which originates from a Yahoo ad server. If the user clicks OK, he will be taken to a site which falsely claims that his phone has severe battery issues.

Within a few minutes, another advert on Ask.fm attempted to download an Android app directly from a website in France as soon as the user clicked OK. The makers of the genuine Mobogenie Market app recommend that it should only be downloaded from reliable sources such as Google Play, mobogenie.com and other partner networks (although it does not specify who these are).

Incidentally, despite encouraging its users not to reveal their passwords to anyone, the login form on http://ask.fm transmits a user's password over an unencrypted HTTP connection:

Most high profile websites only ever transmit passwords over encrypted HTTPS connections, and many sites also ensure that the entire duration of a browser session remains encrypted, i.e. not just the login process. Sending plain text passwords over an unencrypted connection makes them vulnerable to eavesdropping, giving a correctly-positioned attacker the opportunity to gain unauthorised access to Ask.fm user accounts.

PayPal redirect exploited in Apple ID phishing attack

Fraudsters have exploited a redirection vulnerability in a PayPal website in an attempt to steal Apple IDs. Phishing emails sent by the fraudster were disguised as receipts from the iTunes Store for expensive items, enticing victims to try to cancel the fake orders.

The emails stated, "If you did not order the above products and suspect your account has been hijacked kindly visit the link below". The link was displayed with a legitimate-looking location (www.order.itunes.com/verify/cancel) but actually took victims to a URL on the PayPal communications website. The phishing email also noted, "You will be asked some specific questions about you and your financial data to prove you actually owned the account."

The page on PayPal's website at https://www.paypal-communication.com/r/4V2JION/PPPU5A/GDY6I8I/20PEVD/7ZS7MP/7M/h?a=http://192.185.##.###/~broo23yo/ immediately redirected victims to the Apple phishing site specified in its GET parameter, http://192.185.##.###/~broo23yo/. Parts of these addresses have been obfuscated, although the target of the redirect has since been suspended by its hosting company, HostGator, and the PayPal URL used in the phishing emails no longer redirects to the URL specified in the a parameter.

Fraudsters use redirection scripts on well-known and well-trusted websites in order to increase the success of their phishing campaigns. Some email clients block access to links that use IP addresses directly and, as such, would scupper the fraudster's efforts. Using a fully-qualified domain name eliminates this particular problem, and some operators of third-party blocking software might also assume that all PayPal domains can be trusted without exception, which may not always be true. Cautious users who hover over links before clicking on them will see that the disguised links in the phishing email actually go to a trusted PayPal website, which would not seem untoward.

PayPal's site at www.paypal-communication.com uses an extended validation (EV) SSL certificate, which demonstrates that an enhanced set of guidelines has been followed in order to verify the identity of the website's owner. Some browsers emphasise this additional level of verification by adding green cues to the address bar, so a visitor can be sure with reasonable certainty that this site does indeed belong to PayPal, Inc. In this case, however, the redirect was near-instantaneous, so potential victims would not have seen the additional EV browser cues.

Notably, the secondary purpose of extended validation certificates is to address problems relating to phishing, but this is not effective when a phishing attack exploits flaws on a legitimate website using an EV certificate. A somewhat-similar scenario has previously affected PayPal: a third-party website which used an EV certificate was compromised and used to host a PayPal phishing site in 2011.

Incidentally, encrypted traffic destined for www.paypal-communication.com could also be vulnerable to eavesdropping. This Apache-powered website offered the TLS heartbeat extension prior to the disclosure of the Heartbleed bug, so the private key for its SSL certificate could have been compromised. PayPal promptly reacted to this by switching to a new SSL certificate (issued on 14 April 2014), but crucially, the potentially-compromised certificate has not been revoked. PayPal's main site, www.paypal.com, is affected by the same problem: its pre-Heartbleed certificate has also not been revoked.

Failing to revoke the previous certificate means that if it has been compromised, correctly-positioned attackers could use it to impersonate the secure PayPal communication website until the certificate expires in April 2015. As the site used an EV certificate, revocation is all the more important and is often more effective than the checks made for standard certificates. Most major browsers will make OCSP requests for EV certificates and will not display the EV browser cues if the certificate has been revoked or if there is no positive verification of its current status, e.g. if the OCSP request was blocked by a man-in-the-middle attacker. Revoked EV certificates are also more likely to appear in Chrome's CRLSets, which are arguably the most effective form of revocation checking currently available.

is.gd goes down, takes a billion shortened URLs with it

The popular is.gd URL shortening service has been offline for more than two days, taking with it more than a billion shortened URLs. Shortly before the site disappeared on Sunday, the homepage reported that its links have been accessed nearly 50 billion times.

The shortened links generated are usually not more than 18 characters long, including the protocol http://. These links are commonly used in tweets, emails, and text messages where long URLs are impractical. Despite the fact the shortened links do not work, many previously-created is.gd shortened URLs are still appearing on Twitter.

is.gd is owned by and supported by UK hosting provider Memset, who planned to support it as a free service indefinitely. Notably, its sister site, v.gd, is still up and running. Other free services provided by Memset include TweetDownload, TweetDelete and the statistics calculator Tweetails.

For security reasons, both is.gd and v.gd disallow the shortening of URLs which use the data: and javascript: protocols. Nevertheless, the service is still abused by fraudsters who use the shortened URLs to direct victims to phishing sites. Some fraudsters have appended a query string to the shortened URL in an attempt to make it look similar to those used by the phishing target. For example, the following is.gd URL was used to redirect victims to a Taobao phishing site:

http://is.gd/Tb###U?2.taobao.com/item.htm?spm=2007.1000337

Throughout April, is.gd was the fifth phishiest URL shortening service. By far the phishiest was tinyurl.com, which pointed to 17 times as many phishing sites, making it account for 60% of all phishing activity amongst the top five URL shortening services. Privately-held bit.ly, Google's goo.gl and GoDaddy's x.co also pointed to more phishing sites than is.gd.

Three years ago, the is.gd service suffered a shorter outage of a few hours. This was caused by the failure of some of the virtual machines in its frontend cloud, which were responsible for accepting HTTP requests from a load balancer.

Update 21/05/2014: is.gd is now back online. An explanation for the outage can be found at http://is.gd/news.php