Question: Why does Sitemap Creator say “Duplicate Content Found” in my Log after crawling?
Answer: This message is to let you know that Sitemap Creator has automatically detected that this unique URL points to the same HTML content as another URL. The Sitemap protocol states that you should only include URLs that point to unique content on your server. To keep your sitemap valid, Sitemap Creator automatically ignores these latter URLs.
Here is an example. On our site, www.inspyder.com, these two URLs point to the exact same content:
- http://www.inspyder.com/
- http://www.inspyder.com/Default.aspx
The second URL listed above includes the default filename (which our server implicitly adds when it is not specified). Including both these URLs in the sitemap would violate the sitemap guidelines, so Sitemap Creator ignores the second URL.
These messages in Sitemap Creator’s log do not indicate there is anything wrong with your website. The information is shown there so that you will know why certain URLs have not been included in your sitemap.