The first Netcraft Web Server Survey was done over the last weekend in July 1995. It had responses from 18,957 hosts. NCSA was then by far the most commonly used server with 57% of the responses, followed by CERN with 20%. When the survey went public, the first people to access the site were the Apache developers, just four months into their project, who were euphoric to find that 658 sites were using their software. These days the number of Apache sites is comfortably into six figures and their most likely source of competition is Microsoft, rather than the other freely available Unix servers.

The growth of the survey

The growth in the number of sites from 18,957 a year ago to todays 342,081 partly reflects the natural growth in the number of sites, and partly that we have got better at finding them. The eighteen fold rise in the number of hosts in the Web Server Survey over the year compares with a doubling in the number of internet domains found by Network Wizards in the six month period between July 1995 & January 1996. Network Wizards are due to report their July 1996 results shortly; presumably the number of domains will be considerably up on the 240,000 reported for January, as from our own research, in mid July there were 420,000 domains in .com alone.

The size of the survey

With over 342,000 hostnames, the Netcraft Web Server Survey is probably the world's largest collection of web sites; the closest published figure that we have seen is the 275,600 hostnames claimed by Digital for its AltaVista database. Until May 1996 we allowed robots full access to the survey, and so by definition, if we had details of a site, then the search engines had the urls too. Eventually, the files containing the urls to sites running the most popular servers became so big that they were only taken by robots, and so we stopped publishing report files that were more than half a megabyte in size. Since then we seem to have outstripped AltaVista in the quest to find more sites, though this could change; as far as we know our methods are completely different from AltaVista's, and there's no 'right' way to do it.

Why do you need so many sites?

Our interest is network exploration rather than market research. The motivation has been simply to find as many sites as we can, and, having found them, report which server software is in use. In some ways this is easier than worrying about what size a sample should be to be representative, and how to select an unbiased sample. The philosophy is simply to read the Network and report what we find.

Major trends during the year

The graphs provide the best illustration of recent trends in survey share. The period has been characterised by a very considerable growth in the number of sites, and so even servers that have lost percentage share in the survey have still experienced a large increase in user base. NCSA's considerable decline in share of the survey from 57% to 18% still reflects a increase in the number of the sites from 10,835 in August 1995, to 62,430 now.

The whole period has been dominated by success of the Apache project, whose server now runs more than one in three internet sites. Throughout 1995, Apache's success was echoed by the Netscape server family; Netscape's share of the survey peaked at 17% in March 1996, and has declined slightly since then. Most recently, the Microsoft Internet-Information-Server has enjoyed some success, curently holding 5.5% of the survey from a standing start in February 1996.

The growth of NT as a WebServer platform has been a subplot throughout the period; the percentage of sites known to be running on NT is 12.5%, and the actual figure, using Netscape's estimate that 40% of their current sales are for the NT platform, is likely to be around 18%.

One thing that has remained fairly constant is the popularity of freely available servers; the difference is that whereas in July 1995 people chose NCSA or CERN, today it is more likely to be Apache or Microsoft.