Teleport Pro is the world\'s most widely used webspider

A completely automated, multi-threaded, link-following, file-retrieving webspider