Site Operator Survey

Netcraft has developed a technique to identify site operators who control large numbers of internet web sites, but host the sites with external companies rather than develop their own datacenters. These site operators may include web site development companies, shared hosting systems, domain registrars, managed service companies, and private label hosting resellers.

Because of their control over large numbers of sites, and their ability to move from one infrastructure provider to another, these site operators often make attractive prospects for hosting service providers, and hosting automation software companies.

Netcraft identifies such companies by comparing the results of a DNS lookup on each site found by the Web Server Survey with the results of a DNS lookup on its ip address. Typically, the name server in control of the hostname belongs to the site operator, and the name server in control of the ip address belongs to the infrastructure provider.

The dataset covers all 22,000 site operators we find with 20 or more active sites in the United States hosted at some 2,250 hosting locations with 200 or more active sites. We preserve the mappings between each, such that there is a clear identification of the relationships between infrastructure providers and the site operators present on their networks. An analysis by operating system platform, is available and similar datasets can be produced on demand for other countries.

Example - Top 10 Site Operators using Verio in the United States, May 2003

Limitations

• Only sites found by the Web Server Survey will be included. The number of hosts found worldwide running internet web sites by the Web Server Survey is large [over 40 million in April 2003], but is not exhaustive.
• Fully attributing a site requires successful DNS lookups on each of the hostname and ip address. If a DNS lookup fails then the site operator and/or hosting location will be unknown to us.
• Name servers that provide DNS for sites hosted on 5 or more Infrastructure providers’ networks are removed from the dataset. We have found that companies that provide a DNS service to sites situated on many providers’ networks are more likely to be specialist DNS providers than web site developers or hosting services resellers.
• Sites where the hosting provider name server is authorative for both the ip address and domain name are tagged so that they can be removed from view - this is the conventional situation where a hosting company runs DNS for its customers, and there is likely to be no third party involved in the relationship.

More Information

The dataset is updated on a monthly basis and is available on a company license basis.

Please contact us (sales@netcraft.com) for further information of costs.