Andrea Ledwell and the Guardian Headlines

Andrea Ledwell mentions the lack of Guardian headlines and the old News section of this site. The news is back and so are the headlines, in a friendlier format. It’s funny you should mention this, as earlier this week I sent the following email to Gary MacDougall at The Guardian:

Back in late 1999, I was an early experimenter with a new technology called RDF, which, in essence, allows website content to be syndicated. The idea is that headlines from many different websites can be amalgamated onto one customizable page, with links to the original source provided. While this is quite common these days (see my.yahoo.com, www.moreover.com, my.netscape.com and others), at that time it was all new and experimental.

In the spirit of this experimentation, on November 3, 1999, shortly after you folks went online with your website, I wrote a little program that would visit your website at 3:00 p.m. each Monday through Saturday and sift through the stories on the various pages, pull out the headlines from each page (this is easy to do, in part because you’re so consistent in your formatting — you always enter headlines the same way), and make these available for syndication. I published a couple of notes about this to my own website, and experimented with adding Guardian headlines to various customizable third-party websites.

And then I forgot about all of this and went on to other projects, leaving the daily headline update in place on my server for all takers.

Unbeknownst to me, a lot of people watched those original experiments, or found links to the Guardian headlines files on my server, and started to use them to provide customizable Guardian headlines, with pointers to your website, on their websites. I only noticed this recently, when I started to analyze the traffic to www.reinvented.net.

Look at this website as an example of this; halfway down the middle column of the page you’ll see a section called “Charlottetown Guardian: Top Stories”. I don’t have any relationship with these folks, in fact I didn’t know they existed until I searched just now — they simply connect each day to 5 little files of headlines on my server which, in turn, point to stories on your server.

These headlines are now showing up in many different places in the Internet. If you go to www.google.com, for example, and search for “Charlottetown Guardian deaths”, you’ll see the first search result takes you to a page at this site, which displays today’s Guardian-listed deaths, with links back to your site for details.

Traffic-wise, in the last 100 days there have been 1,367,606 requests for these headlines files from my server, roughly 14,000 per day. This traffic is now accounting for 90% of the traffic to my server.

In the end this is all a Good Thing for the Guardian because, at least in theory, it should be driving traffic to your website.

When I first started this little experiment, I sent an email to Don Brander letting him know what I was doing; I didn’t hear back from him at the time. This email is simply to let you know that this is all happening, and to seek your comments on whether this is a service I should leave in place or not.