RSS Traffic Burdens Publisher’s Servers

The popularity of RSS feeds is testing the web infrastructure of at least one publisher, which likens the impact of newsreader traffic to a denial of service (DoS) attack.

Infoworld.com experiences a "massive surge of RSS newsreader activity at the top of every hour," according to Chad Dickerson, the CTO of Infoworld. "If I didn’t know how RSS worked, I would think we were being slammed by a bunch of zombies sitting on compromised home PCs," Dickerson writes. "Our hourly RSS surge has all the characteristics of a distributed DoS attack, and although the requests are legitimate and small, the sheer number of requests in that short time period creates some aggravating scaling issues."

RSS, an XML format known as Really Simple Syndication (or Rich Site Summary, as Netscape's early implementation was known), was popularized by weblogs and has since been adopted by many major news sites as an effective method to "push" headlines to readers. RSS newsreaders regulary refresh the feed to update headlines, generating repeat requests for a small number of files containing the feed information.

Both RSS and the competing Atom format store data in XML files that contain headlines and story summaries, and in some cases the full text of an article. The feeds are viewed through desktop newsreader applications such as FeedDemon or NewsGator or web-based services like Bloglines and My Yahoo. Either offers an alternative to e-mail alerts and newsletters, which are increasingly stymied by anti-spam filtering by ISPs and employers. Many newsreaders are set to update feeds at set intervals, regardless of whether the RSS file has been updated.

InfoWorld, which is hosted by Verio, is committed to RSS. But Dickerson says he's spoken with other large media sites that have delayed implementing RSS feeds, citing potential overhead on IT infrastructure. Some major publishers of RSS feeds are high-traffic sites that already use content distribution and caching to manage server load, such as Yahoo.

While a relatively small number of sites are currently seeing RSS traffic on the scale of InfoWorld, that's likely to change as the technology becomes more popular. "If RSS is going to go from fairly big to absolutely huge, we’re all going to need to do a little more work on the plumbing," Dickerson writes.