200 million crawled pages!

We have reached 200 million crawled pages. Se live status here: http://dcsetup.boitho.com/cgi-bin/dc/topCrawlers.cgi We are crawling ca 120 million pages a month now, so 300 million isn’t far away.

Comments

Calle
May 10, 2005

Wow, congratulations! I must say that this project is really impressing, and I will support you in any way I (as a semi-nerd) can. Good luck with 300 million pages

Leave a Comment