Submitted by (@ehs000)

enable findability by using sitemaps and robots.txt

For search engines (Google, Bing, Yahoo, etc) to be able to properly index websites it helps for the website to publish a sitemap Sitemaps are a vendor neutral way for websites to list URLs that they would like crawled and optionally when the resources were last updated. Additionally a robots.txt is used to identify the location of the sitemap, and also ...more »


27 votes