Text, HTML sitemaps, Robots.txt and more

This version includes some new updates that people have been asking for including a new number of sitemap formats.

We recommend that you have a valid robots.txt file, an xml sitemap and an HTML sitemap in your website root folder to optimise your sitemap coverage.

HTML sitemaps

The great thing about an HTML sitemap is that when you publish it any search engine can deal with it whether they officially support sitemaps or not. At the minute we list out all urls in alpahbetical order. If you have any suggestions to improve this please feel free contact us.

Text Sitemaps

We also added text sitemaps to the list of files which is a really simple list of URLs.

Robots.txt

We also produce a robots.txt file for you which you can upload to your website (or use to modify your existing robots file).
The robots.txt file makes it easier for search engines to automatically discover your sitemaps.

Other changes

We’ve also improved the error report and changed it from an XML file to a standard HTML page to make it easier to understand and refined our quick guide to publishing a sitemap.