enable findability by using sitemaps and robots.txt

For search engines (Google, Bing, Yahoo, etc) to be able to properly index websites it helps for the website to publish a sitemap http://www.sitemaps.org/ Sitemaps are a vendor neutral way for websites to list URLs that they would like crawled and optionally when the resources were last updated. Additionally a robots.txt http://www.robotstxt.org/robotstxt.html is used to identify the location of the sitemap, and also... more »


27 votes