Sitemaps
From Helpful
| For more webdev-related articles, see the webdev category. Among the more interesting are general webdev notes, Javascript related notes, CSS notes, browser peculiarities, jQuery |
| This article/section is a stub — probably a pile of half-sorted notes and assertions some of which may well be wrong, and not verified as a whole. Feel free to add or refine. |
Contents |
[edit]
What
The recent standardization of sitemaps builds on the concept of the older, informal sitemap that consisted of a HTML page that happened to point to various parts of a site, often to get crawlers to crawl everything.
The modern sitemap specification allows web developers to specify details that allow crawlers to crawl their sites more effectively than the standard behaviour does.
Such sitemaps are used by Google, Yahoo, MSN, Ask, and others. They can be said to complement robots.txt in that those can only ask the same crawlers not to harvest something.)
Sitemaps allow specification of
- what parts are available for harvesting
- when a page was last updated
- how often each item will change
- the relative, on the site (see note below)
Sitemaps give farily little added value when all the content is well linked and basic harvesting seems to do quite well.
Sitemaps are useful when
- Things are not well linked yet, from the site itself and/or from elsewhere
- you want to hint to search engines that something (particularly dynamic content) updates quite often, or updates rarely
- You are using Javascript drop-down menus, AJAXed content, in a way that won't be found by crawlers
[edit]
XML or plain text
[edit]
Getting it referenced and used
[edit]
Sitemap indexes
[edit]
