Adding a sitemap to your website allows searchbots to find pages much faster and more efficiently, allowing them to be quickly indexed for search engines.  Sitemaps can be saved as either "/sitemap.txt" or "/sitemap.xml" and should be placed in the root webdirectory ("/"). [10] [11] [12] [13]  Using plaintext (TXT) is much faster and easier than writing extensible markup language (XML).  I recommend keeping the sitemap as plaintext, allowing the SITEMAP protocol to join the ranks of the other plaintext website protocols for ROBOTS, SECURITY, and HUMANS.

[Image.]  A sitemap will help your webpages to get indexed by search engines quickly and efficiently.  Google is the most popular search engine in the world (for 2023), followed in second place by Microsoft Bing. [1] [2] [3]  Photograph depicts the corporate headquarters for Google Search in California (USA) with the Google logo decorated in the rainbow colors of the LGBT (lesbian, gay, bi, trans, et al) flag for the celebration of USA LGBT Pride Month. [4] [5] [6] [7] [8] [9]



As with all webtext files, you should use an advanced text editor such as Notepad-Plus-Plus (not Microsoft Windows Notepad). [14]  Files should be saved with Unix line endings and UTF-8 (Unicode Transformation Format Eight-Bit) character encoding.

canonical links

To create a sitemap, you simply make a plaintext list of each URL (uniform resource locator) for the website with one URL per line and no other content (no comments).  Only URLs for a single domain should be included — do not add URLs for subdomains or alias domains.  You should only list canonical URLs.  This means that if a particular webpage can be accessed from multiple URLs, only one URL should be listed for that webpage in the sitemap.

For example, there are many different ways to access Nicole Sharp's Homepage:

The canonical URL though is

since all of the other URLs redirect to that URL.


To get canonical URLs in Wikimedia MediaWiki, add the following to "LocalSettings.php". [15]

$wgEnableCanonicalServerLink = true;

no subdomains

Here are even more ways to access Nicole Sharp's Homepage:

With the exception of "", none of these other URLs should be included in "".  All of the URLs should have the same protocol (either all HTTPS [Hypertext Transfer Protocol Secure] or all HTTP [Hypertext Transfer Protocol]) and all of the URLs should be on the same subdomain (for example, either all with "www" or all without "www").


The following "/sitemap.txt" example gives a compliant sitemap for "":'s_Website's_Website's_Website

Only canonical URLs are included, all of the URLs have the same protocol ("https://"), and all of the URLs are on the same subdomain ("").  Each new subdomain will need its own sitemap.


Comments are not supported in the SITEMAP protocol for plaintext.  If you are using an XML sitemap, then you can add XML comments.


Once your sitemap is completed, you can add it to the Robots Exclusion Protocol to be indexed by searchbots. [16] [17] [18] [19]  An example "/robots.txt" with a sitemap is given below.

User-agent: *

see also


  4. commons:category:Google
  5. commons:category:Google logos
  6. wikipedia:Googleplex
  7. wikipedia:LGBT flag
  8. wikipedia:USA LGBT Pride Month#Recognition
  15. mw:$wgEnableCanonicalServerLink


bots, CANONICAL, development, hyperlinks, indexing, links, ROBOTS, robots.txt, searchbots, SITEMAP, sitemap.txt, TXT, URLs, web, webcrawlers, webcrawling, webdevelopment, weblinks, WWW