December 2012 Web Server Survey

In the December 2012 survey we received responses from 633,706,564 sites - an increase of over 8 million since November.

Microsoft IIS experienced the largest gain this month, with the movement of an advertising network of 4.7M Apache hostnames to IIS 7.5 contributing to an overall 8.2M increase - their largest in over a year. As a result of the switch, Apache saw an equivalent loss, reducing their market share by 1.53 percentage points. Despite Apache's continuing downward trend over the last few months, they still hold on to more than half of the market (55.70%). Strong growth was also experienced by nginx this month, with a gain of 2M hostnames resulting in another increase to its market share.

nginx also further increased its market share within the million busiest sites, which now stands at 12.44%, as did Microsoft, which remains slightly ahead with a 13.22% share. While overall the survey sees IIS/6.0 as the most popular version of Microsoft's web server software, with a 41 percentage point lead over other versions, within the million busiest sites IIS/7.5 looks set to soon overtake it. IIS/7.5 is now used to serve 40% of IIS websites within the top million, just 4.8k and 4 percentage points behind IIS/6.0.


Linux Rootkit Found Infecting Webservers with iFrame Injection

A new rootkit, which can infect web servers running on 64-bit GNU/Linux, has been discovered which attacks web surfers with drive-by-downloads. The malware works by injecting an iFrame directly into the outgoing TCP packets of the infected machine, allowing it to infect all web traffic from the server. It was first discovered on a server running nginx, however it does not appear to be targeting nginx specifically.


ICANN Early Warnings Filed

More than half of the sites found by Netcraft's survey use the .com top-level domain, but ICANN is in the process of creating additional TLDs. On 20 November 2012, the Governmental Advisory Committee of ICANN filed 242 Early Warnings on individual applications for new top-level domains. These warnings are notices rather than formal objections, and do not directly lead to a process that can result in an application being rejected; however, they are indicative of likely formal objections later on in the application process. Most of the warnings that have been issued consist of "requests for information, or requests for clarity on certain aspects of an application".

Prominent among the list of Early Warnings is Amazon EU, which applied for .app, .book, .cloud, .game, .mail, .map, .mobile, .movie, .music, .news, .search, .shop, .show, .song, .store, .tunes, .video, plus several other unicode TLDs in other scripts and languages. Many of these TLDs have been described as generic terms that relate to broad market sectors, which could have a negative impact on competition if Amazon is to exclude other entities from using them.

India, Australia and the United States have each objected to .airforce, .army and .navy being applied for by United TLD Holdco Ltd. The United States simply claims that these strings are confusingly similar to the names of specific government agencies, while both India and Australia note that words associated with the armed forces are protected in national legislation, and the applied for TLDs could mislead users into thinking that a registrant is associated with these national armed forces.

India goes further to state that these applications have the potential to cause irreparable harm to the security and stability of the nation and suggests that the applicant should withdraw their application. The final rationale behind India's warning makes its position clear: "Allowing sovereign functions in the exclusive hands of foreign corporations whose motivations are unknown, and whose jurisdictions are not accessible for national government should NOT be allowed to happen by ICANN."

Applicants who wish to continue with their applications are advised by the Early Warning document to notify the Governmental Advisory Committee of their intended actions and when these actions will be completed. However, ICANN will still continue to process applications which do not receive a response. Conversely, if an applicant decides to withdraw their application, the applicant can receive a refund of up to 80% of the evaluation fee ($148,000).





DeveloperNovember 2012PercentDecember 2012PercentChange
Apache357,865,21557.23%352,951,51155.70%-1.53
Microsoft103,333,17016.52%111,570,01017.61%1.08
nginx74,437,76411.90%76,460,75612.07%0.16
Google21,090,4103.37%21,870,6143.45%0.08
Continue reading

Most Reliable Hosting Company Sites in November 2012

Rank Company site OS Outage hh:mm:ss Failed Req% DNS Connect First byte Total
1 Datapipe FreeBSD 0:00:00 0.007 0.094 0.018 0.037 0.056
2 Server Intellect Windows Server 2008 0:00:00 0.010 0.012 0.064 0.142 0.337
3 Pair Networks FreeBSD 0:00:00 0.014 0.254 0.083 0.169 0.507
4 XILO Communications Ltd. Linux 0:00:00 0.017 0.419 0.067 0.552 0.697
5 ServInt Linux 0:00:00 0.021 0.041 0.053 0.092 0.169
6 Kattare Internet Services Linux 0:00:00 0.021 0.157 0.119 0.242 0.498
7 ServerStack Linux 0:00:00 0.024 0.017 0.031 0.063 0.063
8 GoDaddy.com Inc Windows Server 2008 0:00:00 0.028 0.447 0.119 0.888 1.461
9 INetU Windows Server 2008 0:00:00 0.031 0.122 0.077 0.238 0.463
10 www.hostway.ro Linux 0:00:00 0.031 0.306 0.140 0.917 1.560
See full table

Unaffected by the aftermath of Hurricane Sandy's landfall on the East Coast of the United States, Datapipe had the most reliable hosting company site in November. Webair, Logicworks, Serverstack, and INetU also had no outages in November despite their locations in the path of Sandy. In contrast to the problems with fuel supply at the beginning of the month, Logicworks CEO, Kenneth Ziegler, said "[Logicworks] were able to consistently deliver good news thanks to our partners receiving a predictable resupply of fuel for their generators": this story is repeated across many other successful hosting companies in the area.

Pair Networks' site — located in Pittsburgh, at the periphery of the area affected by Sandy — came third in November. In common with Datapipe, Pair's site is hosted on FreeBSD which is renowned for its reliability and is regularly seen in the top 10 table. ServInt was also on the edge of the area affected by the storm in Reston, Virginia and placed 5th in November, responding to all but six requests.

As one coast of the United States starts to pick up the pieces after a hurricane, another is battered by torrential rain and widespread flooding. The 'Pineapple Express' storm — so named because of its origins in the Pacific Ocean above Hawaii — has not been as damaging as Sandy: few hosting companies appear to have been affected by it at the time of writing this piece. Kattare, 6th in the table, is based in Corvallis, Oregon which is predicted to have minor flooding.

Server Intellect, hosted in Dallas away from the dramatic events on each coast, was placed 2nd, gaining 5 places from last month's rank of 7. Server Intellect specializes in providing Windows-based hosting with data centers located in 3 cities spread across the United States: Washington D.C., Seattle, and Dallas.

Netcraft measures and makes available the response times of around forty leading hosting providers' sites. The performance measurements are made at fifteen minute intervals from separate points around the internet, and averages are calculated over the immediately preceding 24 hour period.

From a customer's point of view, the percentage of failed requests is more pertinent than outages on hosting companies' own sites, as this gives a pointer to reliability of routing, and this is why we choose to rank our table by fewest failed requests, rather than shortest periods of outage. In the event the number of failed requests are equal then sites are ranked by average connection times.

Information on the measurement process and current measurements is available.

Chrome version of Netcraft Anti-Phishing Extension Available

A version of the Netcraft Anti-Phishing Extension for the Google Chrome™ web browser is now available. The Netcraft Anti-Phishing Extension is a tool allowing easy lookup of information relating to the sites you visit and providing protection from Phishing.

Google Chrome Anti-Phishing Extension

The Extension runs on any operating system supported by Google Chrome and displays the hosting location, country, longevity, popularity, and an abstracted risk rating for each site visited. In particular its key features are:

  • Detailed site reports — simply click the Netcraft logo to access a wealth of information about the sites you visit, helping you to make informed choices about their integrity.
  • Risk Ratings — we evaluate the characteristics of the site compared against those depicted by fraudulent sites. The result is a simple visual summary displayed on the site report.
  • Protection against phishing sites — The Netcraft anti-phishing community is effectively a giant neighbourhood watch scheme, empowering the most alert and most expert members to defend everyone within the community. As soon as the first recipients of a phishing mail report it, we can block it for all users of the extension providing an additional level of protection from Phishing.
  • Protection against cross site scripting (XSS) — The extension optionally traps XSS and other suspicious URLs which contain characters with no purpose other than to deceive.
  • Conveniently report suspected phishing & fraudulent sites — At the click of the button you can report suspected web forgeries to Netcraft, helping to protect the community. Netcraft operates an incentive scheme for Phishing site submissions, including iPads, backpacks, mugs, and more... Over five and a half million phishing sites have been detected and blocked by Netcraft since the anti-phishing service was launched.

The Extension is available for download from the Google Chrome Store, and requires no special administrator privileges to install. You can also find the Firefox version from our download page.

Customized versions with corporate branding and navigation are also available.

Download Now!

Phishing attacks using HTML attachments

Netcraft has recently seen an increase in the number of phishing attacks using attached HTML forms to steal victims' credentials. This type of attack is not new - we have received reports of them from our phishing community since 2005 - but have become more popular amongst fraudsters during this year.

The attack works in a conventional way with the distinction that instead of linking to a form hosted on a web server, the form is attached to the mail.

Example drop site mail

A drop site phishing mail against Barclays customers asking the recipient to complete the attached form.

On opening the attachment, the form asks the victim to fill in their credentials. However, because the form is stored locally, it is less likely to be blocked by anti-phishing mechanisms. Some attachments also make use of obfuscated JavaScript to try and prevent anti-phishing software detecting the fraudulent content.

Form attachment screenshot

The form is hosted locally on the user's own computer.

Nevertheless these phishing attacks still have to send the sensitive data to the fraudster. This communication is usually done by sending a POST request to a remote web server, which then processes the information. This POST request can be detected and blocked, thus the user can still be protected. For example, a web browser, or a piece of security software or spam filter can use Netcraft's Phishing Site Feed to detect the phishing attack and block it.

Code snippet of the HTML form element

The form posts the details to a remote web-server.

These phishing attacks are sometimes referred to as "drop site" phishing attacks. This is because the only publicly accessible URL is a page into which the victim's details are "dropped". Drop sites can be difficult to recognise without the accompanying phishing mail. Usually, the "drop" page just processes the victim's details and provides no indication as to its true nature. Some drop sites redirect to the target's real website. This merits suspicion for anti-phishing groups, but may not provide enough evidence for them to block the URL without the accompanying mail.

HTTP Headers for an example drop site

Without the accompanying mail, the drop site URL appears to just be a page that redirects.

Netcraft has recently made improvements to its detection and handling of drop sites, which should be reported to Netcraft by forwarding the original phishing mail, including the HTML attachment(s), to scam@netcraft.com.

As of 1st November 2012, the Netcraft Toolbar community has blocked over 5.5 million phishing attacks. To provide an incentive for the community to continue sending Netcraft reports of phishing sites, Netcraft currently sends reporters the following:

Prize When
Netcraft Branded Mug after 100 validated phishing reports
Netcraft Polo Shirt after 400
Targus Laptop Backpack after 1,000
iPad after 5,000

As a further incentive, reporters become eligible for a separate competition when they reach 5,000 validated reports. To track the progress, we have a leaderboard displaying the people with the largest number of accepted reports so far this month.

CloudFlare accelerates 235,000 websites

Just over two years since its launch, the CloudFlare content distribution network is being actively used to accelerate traffic to more than 235,000 websites in Netcraft's Web Server Survey. In total, we found 785,000 sites currently configured to use CloudFlare's DNS servers. Once a domain has been configured to use these servers, any of its subdomains can be routed through the CloudFlare system at the click of a button. Paying customers can also route their traffic through CloudFlare by setting up a CNAME within their own DNS.

CloudFlare's network is globally spread across 23 datacenters, half of which are entirely remotely operated. Nine of these datacenters were opened during a month-long expansion effort which ended in August and resulted in a 70% increase in network capacity. CloudFlare's content distribution network spreads website content around these datacenters, allowing visitors to request pages from geographically closer locations. This typically reduces the number of network hops, resulting in an average request taking less than 30ms.

In addition to moving static files closer to visitors, CloudFlare also offers an automatic web optimisation feature called Rocket Loader. This combines multiple JavaScript files into a single request, which saves both time and bandwidth. Pro, Business and Enterprise users can also enable beta support for SPDY requests, which achieve better latency than HTTP through the use of compression, multiplexing and prioritisation.

In October, CloudFlare introduced support for OCSP stapling, which it claims has increased the speed of SSL requests by 30%. The Online Certificate Status Protocol allows browsers to ask a certificate authority (CA) whether an SSL certificate it has issued has been revoked. Handling these requests in realtime can be challenging, particularly if the CA has issued a large number of certificates, or has issued certificates to extremely busy websites. OCSP stapling solves this problem by delivering the OCSP response directly from CloudFlare's network, removing the need for the browser to perform an additional DNS lookup and send a request to the CA's own OCSP server. OCSP performance is often overlooked when considering which CA to buy a certificate from, but can have a crucial impact on the overall performance of a customer's website.

With its insight into the kind of requests being sent to many different websites, CloudFlare is well-positioned to identify malicious traffic and provide protection to all of its customers. Depending on which level of security is enabled, CloudFlare can deny requests which are attempting SQL injection attacks, comment spam, excessive crawling, email harvesting, or exploiting cross-site scripting vulnerabilities. Business and Enterprise users can also benefit from CloudFlare's advanced DDoS (distributed denial of service) protection.

CloudFlare's growth accelerated significantly in the summer of last year. This is when many people first became aware of the service, after it was used to handle traffic for the Lulz Security website. High profile attacks against Sony, Fox, PBS and the X Factor helped LulzSec garner 350,000 followers on Twitter, where it extolled the virtues of using CloudFlare to mitigate DDoS attacks.

Some notable high-traffic users of CloudFlare include The Hacker News, Uber Humor, Android firmware site Cyanogen Mod, and the content management system Moodle.

New York Internet rides out the storm

New York Internet looks set to make it through the aftermath of Hurricane Sandy with barely a scratch on its uptime or performance.

NYI's New York City datacenter is just a third of a mile away from the 75 Broad Street datacenter (in evacuation Zone A), which flooded during Hurricane Sandy. NYI is crucially further uphill, in Zone C, and was out of reach of the storm surge. Despite widespread power failures within the area, NYI has so far managed to continue operating without suffering from any outages:

Demand for fuel is very high in the areas affected by the storm. Phillip Koblence, founder of NYI, said that poor availability of gasoline for staff to get to and from datacenters was his worst problem, followed by sleeping and washing facilities. Queues to get gasoline were reportedly two miles long, and vehicles containing fewer than 3 people were not being allowed into Manhattan between 6am and midnight.

Koblence told Netcraft that he had no power or water at home, and difficulties getting food anywhere near work were also adding to the problems. NYI expected commercial power to be shut off as the storm hit, and its New Jersey and New York City datacenters have been running on diesel generators since Monday.


NYI's New York City datacenter is only a few blocks
away from the flooded 75 Broad Street datacenter.

NYI estimates that its 21st floor generator tank in Manhattan can last for 36 hours, while its 8th floor generator can run for more than 3 days. The Bridgewater datacenter in New Jersey is believed to be capable of lasting at least 10 days on generators, far longer than the times between each fuel delivery that the company has been sustaining this week.

Commercial power is expected to be restored to the datacenters late on Saturday night, although NYI plans to remain on generator power until the commercial power supply becomes stable.

Queuing for gasoline in Bridgewater.

Live performance graphs for www.nyi.net can be viewed here.