Recalling that the Internet is sometimes called the world wide web makes the concept of spidering make somewhat more sense.
Search engines like Google, Feedster and Altavista — basically any web application that needs to grab a snapshot of the entire Internet so that it can be searched, indexed or otherwise sliced and diced — generally develop this snapshot using automated web browsing computer programs that start at one web page, collect all the links on that page and visit each of them in turn, collect all the links on those pages, and so on and so on and so on.
This process, because it is conceptually like “walking throught the web” is called spidering or crawling.
About This Blog
I am Peter Rukavina and this is my blog. I am a writer, letterpress printer, and a curious person.
To learn more about me, read my /now, look at my bio, listen to audio I’ve posted, read presentations and speeches I’ve written, or get in touch (peter@rukavina.net is the quickest way).
You can subscribe to an RSS feed of posts, an RSS feed of comments, or a podcast RSS feed that just contains audio posts. You can also receive a daily digests of posts by email.
Comments
For still more cognitive
For still more cognitive resonance see e.g. www.webcrawler.com/
I was thinking the other day
I was thinking the other day while visiting a blog and visiting links mentioned on the blog and cutting and pasting bits of articles and links and emailing them to people that the term “mining” was more applicable than “surfing”.
Add new comment