Wednesday, November 29, 2006

How to create a well behaved bot?

From the ever watchful master of bots and scrapers incrediBILL (:-)) comes the following tip, which you might find useful if you yourself plan / need to implement a web crawler as part of your product / service.

No comments:

Post a Comment