From Helpful
Jump to: navigation, search
Related to web development, hosting, and such: (See also the webdev category)
jQuery: Introduction, some basics, examples · plugin notes · unsorted

Server stuff:

Dynamic server stuff:


This article/section is a stub — probably a pile of half-sorted notes, is not well-checked so may have incorrect bits. (Feel free to ignore, fix, or tell me)

A sitemap is a list of pages on a domain.

This stems from the days where having one page that linked to all pages was the easiest way to be sure all of your public-facing pages on your site got indexed by crawlers.

It still has some value in this regard, though only to make sure things get indexed at all, because after since pagerank became a think it had no effect on search results.

The specification is a formalization of the idea of "yeah just make HTML with lots of links", also allowing you to give more specific information of how to crawl, which can have some minor positive side effects e.g. on your resource use.


Sitemaps allow specification of

  • what parts are available for harvesting
  • when a page was last updated
  • how often each item will change
  • the relative, on the site (see note below)

Sitemaps are useful when

  • Things are not well linked yet, from the site itself and/or from elsewhere
  • you want to hint to search engines that, say, your news page and some dynamic content updates quite often, while some stuff is almost static
  • You are using Javascript drop-down menus, AJAXed content, in a way that means crawlers won't find your links/content.

They have little added value when all your content is well-harvested already.

Sitemaps could be said to complement robots.txt in that those can only ask not to harvest something.

XML or plain text

Getting it referenced and used

Sitemap indexes


These files are generated by Bing Sitemap Plugin for IIS and Apache.

Requests for this will only come from msnbot, so implementing this is useless for other search engines.

You probably want to ignore these requests.

See also

See also