Spider

Spiders (also called robots, bots or crawlers) are programs with whose help search engines comb through the Internet to index the content of websites. The spider finds websites either through the registration of a page or through linking from other pages.