Web spiders, sometimes referred to as Web crawlers, are programs that follow Web links from one site to another, examining content and recording locations.