What do you mean by Spider?
Many search engines use programs called spiders to index websites. The spiders are also known as crawlers or robots. They act as automatic date searching tools that visit every site to find new or updated web pages and links. This process is called web crawling. Spiders follow hyperlinks and gather textual and meta information for … Read more