I’ve been playing around with ways to take advantage of FeedTree’s prompt feed updates. On my desktop, where I run the FeedTree client proxy, I get new links from the Digg diggall feed and the Reddit new links feed every 6 minutes on average. These feeds are a firehose of links that are about to become hot; between the two, there are about 60 new URLs posted per hour. With FeedTree it’s actually possible to keep on top of this stuff as soon as it’s “discovered”.

It turns out I don’t always have time to read these feeds and hit all those URLs immediately. (OK, I rarely have time.) I typically wait for popular links to bubble up to the frontpages of Digg and Reddit, by which time they’re often unavailable, their webservers slaughtered by hordes of Digg readers.

It occurred to me that a program which listens to FeedTree (for really up-to-the-minute feed updates) and automatically populates the CoralCDN distributed Web cache. That way, Coral can snapshot the pages before they’re swamped, and by the time I get around to seeing these links, there’ll be a Coralized version (thanks, Mike!) all ready to go.

So I wrote Calcium, a Python program that does exactly that. It’s running on my desktop, passing new soon-to-be-hot links through FeedTree and over to Coral. Now everyone will benefit from automatically pre-Coralized Digg and Reddit links!

(If you want to play with it, grab a copy from the Subversion browser or right out of the repository.)

This entry was posted on Sunday, February 12th, 2006 at 8:49 pm and is filed under Progress, Tools. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.