On this page
Crawlers find pages in two ways: by following links, and by reading a list you give them. That list is your XML sitemap, and sitemaps for AI crawlers work the same way they do for search engines. It matters most for pages that are new, deep in your site or poorly linked, which are exactly the pages a crawler following links might never reach.
Sitemap & Discovery is worth 5 of the 100 points in an AI Crawl Radar audit. It is one of the easiest categories to score in full.
How the check works
The audit requests https://yourdomain.com/sitemap.xml (following redirects) and reads your robots.txt. Then:
| Signal | Share of the category |
|---|---|
A sitemap is served at /sitemap.xml with a 2xx status | 60% |
robots.txt declares a sitemap with a Sitemap: line | 25% |
The sitemap includes <lastmod> dates | 15% |
A sitemap index (a sitemap that lists other sitemaps) counts as a sitemap.

"Publish an XML sitemap"
This recommendation appears when nothing usable is served at /sitemap.xml. A minimal sitemap looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://yourdomain.com/</loc>
<lastmod>2026-10-01</lastmod>
</url>
<url>
<loc>https://yourdomain.com/pricing</loc>
<lastmod>2026-09-14</lastmod>
</url>
</urlset>
You rarely need to write it by hand:
- WordPress: WordPress has a built-in sitemap at
/wp-sitemap.xml, and SEO plugins such as Yoast and Rank Math generate their own, usually at/sitemap_index.xml. - Next.js: add an
app/sitemap.tsfile that returns your URLs; Next serves it at/sitemap.xml. - Shopify, Wix, Squarespace: a sitemap is generated automatically at
/sitemap.xml. - Other stacks: most frameworks and static site generators have a sitemap plugin.
When your sitemap lives somewhere else
The audit looks at /sitemap.xml specifically. If your sitemap is only at /sitemap_index.xml or /wp-sitemap.xml, add a 301 redirect from /sitemap.xml to it. The audit follows the redirect, /sitemap.xml is the conventional location crawlers check, and it costs nothing. Many SEO plugins already add this redirect.
"Declare your sitemap in robots.txt"
Add one line anywhere in your robots.txt, using the full URL:
Sitemap: https://yourdomain.com/sitemap.xml
This is how crawlers that have never heard of your site find your sitemap: they read robots.txt first. The line is not part of any user-agent group, so it applies to every crawler. See the robots.txt guide for the rest of the file.
Missing lastmod dates
<lastmod> tells crawlers when a page last changed, so they can re-fetch what is new instead of everything. Two rules make it useful:
- Make it true. Use the date the content actually changed. A sitemap generator that stamps every URL with today's date on every request teaches crawlers to ignore your dates.
- Use W3C format:
2026-10-01or2026-10-01T09:30:00+00:00.
Do AI crawlers use sitemaps?
AI providers rarely document it, but crawlers commonly read the Sitemap: line in robots.txt, and a sitemap also feeds the search indexes many AI answers draw on. It costs nothing to publish.
What belongs in a sitemap
- Include the canonical, indexable version of every page you want found.
- Exclude pages with
noindex(see technical SEO for AI visibility), pages blocked in robots.txt, redirects, 404s, duplicate URLs with tracking parameters, and private areas such as account and checkout pages. - Keep each file under 50,000 URLs and 50 MB uncompressed; use a sitemap index to split larger sites.
Submit it, too. Add your sitemap in Google Search Console and Bing Webmaster Tools. For Bing, Yandex and others you can also notify them of new and changed URLs as soon as you publish, with IndexNow.
Sitemaps and llms.txt
A sitemap and an llms.txt do different jobs. The sitemap is a complete list for crawlers; llms.txt is a short, described shortlist for AI tools. Have both: the sitemap so nothing is missed, llms.txt so the important pages are understood.
Verify
- Open
https://yourdomain.com/sitemap.xmland confirm it loads as XML and lists your key pages. - Check robots.txt for the
Sitemap:line. - Re-run the AI visibility audit; Sitemap & Discovery should reach 100%.
Related guides
Frequently asked questions
6 questions people ask about this topic
Do lastmod dates need to be accurate?
Where is my WordPress sitemap?
Do I need both a sitemap and llms.txt?
Should I submit my sitemap to Bing?
What should not be in a sitemap?
How big can a sitemap be?
See how AI crawlers read your site
Run a free AI visibility audit: a 0–100 score and a fix for every lost point.
Analyze my website


