What is a sitemap, and why does your website need one? A sitemap is a machine-readable file, usually an XML file, that lists important website URLs for search-engine discovery. A sitemap can help search engines find new or difficult-to-reach pages, but submission is only a hint: it does not guarantee crawling, indexing, or higher rankings.

Many modern content-management systems generate and maintain a sitemap automatically. Before installing a plugin or paying for a generator, check your site’s standard sitemap addresses, CMS settings, and robots.txt. The practical job is to make sure the sitemap contains the right URLs, stays current, and is submitted to the search engines that matter to your audience.

Guide Helped? Tip Us

Key takeaways

  • An XML sitemap lists a site’s preferred, indexable URLs and may include meaningful modification dates and specialized image, video, news, or language information.
  • A sitemap is a discovery and indexing hint, not a command; Google does not guarantee that it will download the sitemap, crawl every listed URL, index every page, or rank any page.
  • A single sitemap can contain up to 50,000 URLs or be 50 MB uncompressed, whichever limit is reached first.
  • Most WordPress and hosted-CMS users should first check the sitemap generated by the CMS instead of installing another sitemap plugin.
  • A useful sitemap excludes redirects, errors, duplicate variants, blocked URLs, noindex pages, tracking parameters, and pages that are not intended for search.

What is a sitemap?

A sitemap is a file or page that describes important content and URLs on a website. In search-engine optimization, “sitemap” most often means an XML sitemap: a structured document that search-engine crawlers can read. The Google sitemap overview explains that sitemaps can provide information about webpages as well as images, videos, news content, and alternate language versions.

An XML sitemap does not replace navigation, internal links, canonical tags, or crawl controls. An XML sitemap gives search engines a curated list of URLs that the site considers important, while the rest of the site still needs a technically accessible and useful structure.

XML sitemap

An XML sitemap normally contains one record for each URL. The required location element is <loc>; an optional <lastmod> element records when the page was substantially modified. The official Sitemaps protocol defines the standard XML structure.

Sitemap index

A sitemap index is a sitemap that lists multiple child sitemaps. Large websites can divide URLs into files for products, articles, languages, business units, or update frequency, then submit one index instead of managing every file individually.

HTML sitemap

An HTML sitemap is a normal, human-facing webpage containing links to important sections or pages. An HTML sitemap can help visitors and crawlers navigate a site, but an HTML sitemap is not a substitute for an XML sitemap when search-engine sitemap data is needed. A separate Search.gov explanation of XML sitemaps distinguishes the machine-readable format from ordinary site navigation.

RSS and Atom feeds

Google can use RSS or Atom feeds to discover recently published or updated URLs. A feed may be useful for a frequently updated site, but a feed often contains only recent items, so a feed is not a complete replacement for an XML sitemap on every website. Use the format that accurately represents the URLs you want discovered.

Why does a website need a sitemap?

A website needs a sitemap when a direct list of preferred URLs would help search engines discover, revisit, or diagnose the site’s content. A sitemap is especially useful for new websites, large websites, frequently updated websites, JavaScript-heavy websites, sites with weak internal linking, and websites with specialized media or multilingual content.

How does a sitemap help search engines discover URLs?

A sitemap gives search engines a direct set of URLs to consider instead of requiring every page to be found through internal links or external links. The benefit is greatest when a site is new, has few backlinks, contains orphaned pages, uses complex navigation, or generates navigation through JavaScript. A sitemap does not make an inaccessible page crawlable and does not guarantee that a crawler will use every listed URL.

Can a sitemap help search engines notice updates?

A valid lastmod value can communicate when a URL was meaningfully changed. A site should update lastmod only after a substantial change, not on every automated visit or merely to make an old page appear fresh. Inaccurate metadata makes the sitemap less trustworthy and does not force immediate recrawling.

How does a sitemap help diagnose indexing problems?

Submitting a sitemap to Google Search Console creates a useful comparison between URLs the site submitted and URLs Google reports processing or indexing. The submitted and indexed totals should not be expected to match exactly: a submitted URL can still be duplicate, canonicalized elsewhere, blocked, low quality, or otherwise excluded. The Google Search Console Sitemaps report documentation describes the report’s processing and error information.

When are specialized sitemaps useful?

Image, video, news, and alternate-language sitemap data can provide additional context when a site genuinely publishes those content types. A video sitemap can describe information such as duration, rating, or age restrictions; a news sitemap is intended for news content rather than every ordinary blog post. Specialized sitemap data must be maintained consistently with the page markup and, for alternate languages, the site’s hreflang implementation.

Does every website need a sitemap?

No. A sitemap is recommended for many sites and is usually inexpensive to keep, but a sitemap is not a technical prerequisite for every website to appear in search. A small, private, well-linked site may gain little from manually creating one, while a large or frequently changing site can benefit substantially.

Site situation Sitemap value Practical decision
New site with few external links High Use a sitemap to provide a direct discovery route.
Large ecommerce or publishing site High Use automatically generated, segmented sitemaps and monitor them.
Frequently updated news or content site High Keep URLs and meaningful modification dates current.
JavaScript-heavy site High Use a sitemap, but also fix rendering, navigation, and internal-link problems.
Site with orphan pages High, but incomplete Use the sitemap for discovery and add proper internal links.
Small, well-linked brochure site Helpful but usually low urgency Keep the CMS-generated sitemap if one exists; do not overengineer it.
Private or intentionally unsearchable site Usually unnecessary Do not publish a sitemap of content that should remain private.
Important URLs blocked or marked noindex Insufficient by itself Fix crawl and indexing directives first.

The simplest rule is this: if your CMS already creates a valid sitemap, leave it enabled and verify its contents. If your small static site has only a handful of clearly linked pages, manually creating a sitemap is optional. If your site has hundreds or thousands of URLs, frequent changes, filters, products, media, multiple languages, or crawl problems, automated sitemap management deserves priority.

What should a sitemap include?

A sitemap should contain the site’s preferred URLs that return a successful, indexable response and are intended for consideration in search results. Google recommends including the canonical URLs that the site prefers to appear in search; see Google’s sitemap-building guidance for supported fields and submission methods.

Include these URLs

  • Absolute, fully qualified HTTPS URLs for canonical pages.
  • Important home, service, product, category, article, location, and other landing pages.
  • Pages that are accessible to crawlers and intended to appear in search.
  • Accurate lastmod dates when the page has been meaningfully updated.
  • Relevant image, video, news, or alternate-language information when the site supports and maintains it correctly.

Exclude these URLs

  • Redirecting URLs, including URLs that return 301 or 302 responses.
  • 404, 410, soft-404, or server-error URLs.
  • URLs blocked from crawling by robots.txt.
  • Pages carrying a noindex directive.
  • Duplicate URL variants and non-canonical versions.
  • Tracking-parameter URLs such as campaign variations.
  • Private, staging, administrative, thin, or internal search-result pages.
  • Faceted filters, tag archives, author archives, or parameter combinations unless those pages are deliberately maintained as search landing pages.

A sitemap should be a curated list of preferred, indexable URLs, not a dump of every URL that a server can generate. Listing a URL does not override a noindex instruction, a canonical preference, an access restriction, or a search engine’s quality decision.

What are the technical sitemap limits?

According to Google’s sitemap-building documentation, one sitemap file can contain up to 50,000 URLs or be no larger than 50 MB uncompressed, whichever limit is reached first. A larger site must split its URLs across child sitemaps and reference those files from a sitemap index.

Requirement Correct implementation
URL format Absolute, fully qualified URLs such as https://www.example.com/services/.
Encoding UTF-8.
Syntax Valid XML with correctly escaped special characters.
Access Publicly accessible to crawlers without an authentication wall or accidental firewall block.
Scope URLs must comply with the sitemap’s permitted site and directory scope.
Hosting HTTPS is appropriate for an HTTPS site; root placement is recommended when possible.
Large sites Use multiple child files and a sitemap index.

A sitemap does not have to be named sitemap.xml. Conventional names make troubleshooting easier, but another valid filename can work. A sitemap also does not have to be placed at the root in every configuration, although root placement simplifies broad coverage. Sitemap submission and scope depend on how the sitemap is discovered and submitted; consult Google’s large-sitemap documentation when using indexes or multiple directories.

How do you find an existing sitemap?

Start by opening the most common sitemap paths in a browser:

https://example.com/sitemap.xml
https://example.com/sitemap_index.xml
https://example.com/robots.txt

If the first address returns a 404, check the second address and then inspect robots.txt for a line beginning with Sitemap:. A sitemap may also be visible in Google Search Console, Bing Webmaster Tools, CMS settings, or an SEO plugin’s settings.

WordPress installations can have a core-generated sitemap. Yoast SEO commonly uses sitemap_index.xml, while WordPress core or another plugin may use a different sitemap system. The Yoast XML sitemap documentation explains how to identify and control its sitemap feature.

Wix and Squarespace manage sitemaps differently from self-hosted WordPress. Wix’s sitemap documentation says its sitemap index is automatically submitted to Google when the SEO Setup Checklist is completed. Squarespace’s sitemap documentation explains how to view the XML sitemap at a domain-based URL, which can depend on the domain configuration.

How do you create a sitemap?

For most site owners, the best way to create a sitemap is to let the CMS or site platform generate it automatically. Automatic generation keeps the file synchronized when pages are published, changed, or removed.

Option 1: Use the CMS-generated sitemap

Check the CMS documentation and settings before adding a third-party tool. Many WordPress and hosted-CMS installations already generate a sitemap, so creating a second one can add maintenance work without adding value.

Option 2: Use a WordPress SEO plugin

A WordPress SEO plugin can provide extra control over post types, taxonomies, media, and exclusions. In Yoast SEO, the documented general path is:

WordPress Dashboard → Yoast SEO → Settings → Site features
→ Technical SEO → XML sitemaps → On

Yoast commonly produces https://example.com/sitemap_index.xml. Do not install multiple sitemap-generating plugins without checking which system owns the sitemap. WordPress core, Yoast, and a separate sitemap plugin can create overlapping files or conflicting inclusion rules.

Option 3: Generate a file for a small static site

Manual creation or a free generator can be reasonable for a very small, rarely changing static site. Google recommends automatic generation for sites with more than a few dozen URLs because a manually maintained file becomes stale quickly. Ahrefs offers a free XML sitemap generator presented with a 1,000-URL limit; a downloaded file still needs to be checked, uploaded, and regenerated whenever the site changes.

A full SEO platform is not necessary merely to create a sitemap. Ahrefs Site Audit, Semrush, or a specialist crawler becomes more appropriate when the sitemap problem is part of a broader need to diagnose broken links, redirects, canonical conflicts, orphan pages, indexability problems, or crawl discrepancies. Paying for a broad audit platform solely to produce a small sitemap is usually poor value.

Option 4: Generate it from a database or deployment process

Custom websites should generate the sitemap from the same database or content system that controls indexable URLs. A deployment process can create or update segmented sitemap files automatically, reducing stale URLs and preventing unpublished or deleted records from remaining in the file.

What does a basic XML sitemap look like?

A minimal XML sitemap uses a standard namespace, one <url> record per URL, and an absolute <loc> value:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="https://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://www.example.com/</loc>
    <lastmod>2026-08-15</lastmod>
  </url>
  <url>
    <loc>https://www.example.com/services/</loc>
    <lastmod>2026-08-12</lastmod>
  </url>
</urlset>
  • <urlset> identifies the sitemap document.
  • <url> contains one URL record.
  • <loc> is required and must be an absolute URL.
  • <lastmod> is optional and must represent a meaningful modification date.

The dates in the example are illustrative values, not dates that should be copied into a real sitemap. Do not invent lastmod values. URL order does not determine ranking; Google states in its sitemap documentation that it does not care about the order of URLs in a sitemap.

How do you submit a sitemap to Google?

Submit a sitemap through Google Search Console after publishing it at a stable, public URL. The workflow is:

  1. Open the sitemap URL in a browser and confirm that it loads without login, redirects, or a server error.
  2. Verify the correct domain or URL-prefix property in Google Search Console.
  3. Open the Sitemaps report.
  4. Enter the sitemap path or sitemap index URL, such as sitemap.xml or sitemap_index.xml.
  5. Select Submit.
  6. Review the processing status and reported errors after Google processes the file.
  7. Fix the underlying issue and monitor the report again.

Google also supports declaring the sitemap in robots.txt:

Sitemap: https://www.example.com/sitemap.xml

Multiple Sitemap: lines are permitted when a site has several sitemap files or indexes. A stable sitemap URL does not need to be resubmitted after every page change. Search engines can revisit the same URL; the important requirement is that the generator updates the contents correctly. Google describes submission as a hint rather than a guarantee in its build and submit guidance.

How do you submit a sitemap to Bing?

Submit a sitemap through Bing Webmaster Tools after adding and verifying the site. The general workflow is:

  1. Add and verify the website in Bing Webmaster Tools.
  2. Open the Sitemaps tool.
  3. Submit the XML sitemap or sitemap index URL.
  4. Review processing details and errors.
  5. Keep the sitemap reference in robots.txt as an additional discovery route.

The Bing Webmaster Tools sitemap documentation covers submitted and discovered sitemaps. Bing submission does not remove the need to fix invalid URLs, crawl restrictions, or indexing problems.

How do you validate a sitemap?

Sitemap validation must check both XML syntax and the SEO quality of every listed URL. A file can be valid XML yet still be harmful or misleading if it contains redirects, non-canonical pages, blocked URLs, or pages that should not be indexed.

Syntax and access checklist

  • Confirm valid XML and the correct sitemap namespace.
  • Confirm UTF-8 encoding and properly escaped characters.
  • Confirm that the server returns a successful response and an appropriate XML content type.
  • Confirm that the sitemap loads over HTTPS without authentication.
  • Confirm that firewalls, security plugins, or CDNs do not block search-engine crawlers.
  • Confirm that a sitemap index references accessible child sitemaps.

URL-quality checklist

  • Request representative URLs and confirm they return 200 OK where appropriate.
  • Compare each URL with its canonical tag and remove non-canonical variants.
  • Check that URLs are not blocked by robots.txt.
  • Check that URLs do not carry noindex.
  • Remove redirects, error pages, duplicate variants, staging URLs, and tracking parameters.
  • Confirm that every URL belongs to the correct domain, subdomain, protocol, and Search Console property.

Google Search Console can report sitemap processing errors, access problems, and related coverage signals. Search Console is valuable for monitoring, but a crawler or audit platform can provide broader diagnosis when a site also has broken links, orphan pages, redirect chains, or canonical conflicts.

Why are sitemap URLs not indexed?

A submitted sitemap URL may remain unindexed because a sitemap only helps discovery and does not override Google’s indexing decisions. Common causes include low-quality or duplicative content, a canonical tag pointing to another URL, a noindex directive, blocked crawling, server errors, redirects, weak internal linking, or Google choosing not to index the page.

Use this troubleshooting sequence:

  1. Open the exact URL and confirm that the page is publicly accessible.
  2. Check the HTTP response for redirects, errors, or a staging hostname.
  3. Inspect the page’s canonical tag.
  4. Check the page’s robots meta tag or HTTP X-Robots-Tag for noindex.
  5. Check robots.txt for a disallow rule affecting the URL.
  6. Add clear internal links from relevant, indexable pages.
  7. Assess whether the page is original, useful, and sufficiently distinct from other pages.
  8. Use URL Inspection in Search Console for a small number of urgent pages; retain the sitemap for scalable discovery and monitoring.

What do common sitemap errors mean?

Error or condition Likely cause Fix
“Couldn’t fetch” Bad URL, server error, authentication, firewall block, malformed XML, or temporary access failure. Open the file publicly, check the response and XML, then review Search Console processing again.
URLs blocked by robots.txt The sitemap lists URLs that crawlers are instructed not to request. Remove blocked URLs or revise the crawl rule when the URLs should be crawlable. Review the sitemap and robots.txt together.
Sitemap contains noindex pages The sitemap suggests consideration while the page requests exclusion. Remove the URL, or remove noindex if search indexing is intended.
Stale sitemap Deleted URLs remain, or new URLs never enter the file. Repair the CMS, plugin, database query, or deployment process that generates the sitemap.
Wrong domain or protocol Mixed HTTP/HTTPS, www/non-www, staging, or subdomain URLs. Use the canonical production host consistently and submit the correct property.
Duplicate sitemap generators WordPress core, an SEO plugin, and another plugin all generate files. Identify the owning system, remove unnecessary overlap, and submit the authoritative sitemap.
Overloaded sitemap Every parameter, filter, archive, or duplicate URL is included. Curate the file around preferred, indexable search URLs.

A sitemap containing a URL blocked by robots.txt is particularly contradictory: the sitemap asks a crawler to consider the URL while the crawl rule prevents the crawler from fetching it. Google discusses this class of problem in its XML sitemap and RSS/Atom best-practices guidance.

What is the difference between a sitemap and robots.txt?

A sitemap tells search engines which URLs the site considers important, while robots.txt gives crawlers instructions about which paths they may request. Neither file replaces the other, and neither is a complete indexing strategy.

noindex
Control Main purpose What it does not do
Sitemap Lists preferred URLs and optional content metadata for discovery. Does not guarantee crawling, indexing, or rankings.
robots.txt Provides crawler instructions about requesting paths. Is not the ideal tool for removing a URL from search results.
noindex Requests that a search engine not include a page in search results. Does not help a blocked crawler see the directive.
Canonical tag Indicates the preferred version among duplicate or similar URLs. Does not make duplicate pages disappear instantly.
Internal links Help users and crawlers discover and understand site structure. Do not replace a sitemap for large or complex sites.

Blocking a URL in robots.txt can prevent a crawler from seeing a page’s removal instructions. Use crawl restrictions for crawl management, and use appropriate page-level or response-level indexing controls when the goal is exclusion from search.

How should a large website organize its sitemaps?

A large website should divide sitemaps by a meaningful operational boundary, such as content type, language, business unit, or update frequency, and publish a sitemap index that references the child files.

  • Keep every child sitemap within the 50,000-URL and 50 MB uncompressed limits.
  • Submit the sitemap index rather than manually submitting every child file.
  • Use separate files when independent monitoring will reveal problems more quickly.
  • Ensure that the index references only accessible files in permitted locations.
  • Generate the files from the live content database or deployment process.
  • Exclude discontinued products, redirected URLs, filtered duplicates, and non-indexable records.

For multilingual ecommerce, publishing, or JavaScript-heavy sites, sitemap errors may indicate deeper architecture problems. A developer or technical SEO consultant can be more useful than a generator when the site needs database integration, international URL governance, rendering fixes, or large-scale canonicalization.

Final sitemap checklist

  • ☐ A sitemap exists at a stable public URL.
  • ☐ The sitemap loads over HTTPS without authentication or a firewall block.
  • ☐ The XML uses UTF-8 and absolute URLs.
  • ☐ Every listed URL is canonical, indexable, and intended for search.
  • ☐ Redirects, errors, blocked URLs, duplicate variants, tracking URLs, and noindex pages are excluded.
  • ☐ Meaningful lastmod dates are accurate where supplied.
  • ☐ Large sites use child sitemaps and a sitemap index within the documented limits.
  • ☐ The sitemap or index is declared in robots.txt.
  • ☐ The sitemap is submitted in Google Search Console.
  • ☐ The sitemap is submitted in Bing Webmaster Tools.
  • ☐ Processing errors are monitored after submission.
  • ☐ Sitemap generation is automated wherever the site changes regularly.

Frequently Asked Questions

Does a sitemap improve Google rankings?

A sitemap does not directly guarantee higher Google rankings. A sitemap can help Google discover preferred URLs and understand updates, but ranking depends on many other signals, including relevance, quality, links, crawlability, and canonicalization.

Do I need to resubmit my sitemap whenever a page changes?

You generally do not need to resubmit a stable sitemap URL after every page change. Search engines can revisit the same sitemap URL; the important requirement is that the CMS, plugin, or deployment process updates its contents accurately.

Can a sitemap contain pages marked noindex?

A sitemap should not contain pages marked noindex. A sitemap listing suggests that a URL is important for search consideration, while noindex requests exclusion, so remove the URL or remove noindex when indexing is intended.

How many URLs can one sitemap contain?

One sitemap can contain up to 50,000 URLs or be 50 MB uncompressed, whichever limit is reached first. Larger sites should split URLs into child sitemaps and reference those files from a sitemap index.

Does WordPress create a sitemap automatically?

Many WordPress installations provide a core-generated XML sitemap, and plugins such as Yoast SEO can generate a sitemap index with additional controls. Check the existing sitemap before installing another generator because multiple sitemap systems can conflict.

The Bottom Line

A sitemap is worth keeping for most public websites, but a sitemap is not a shortcut to rankings or guaranteed indexing. Find the sitemap your CMS already generates, make sure it contains only canonical and indexable URLs, declare it in robots.txt, submit it to Google Search Console and Bing Webmaster Tools, and fix the site problems that the sitemap report exposes.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi