Sitemap could not be fetched in Search Console
Diagnose a sitemap fetch failure by checking the exact submitted URL, redirects, response bytes, and Google’s last attempt before changing the generator.
By Crawlito · 5 min read
In short
- A sitemap fetch failure concerns the sitemap URL before its listed pages can be processed.
- Save the response body because a successful status can accompany an HTML error page.
- A repaired live response and Google's recorded processing status are separate observations.
On this page
- Copy the submitted URL before testing another address
- Save the status and the complete response
- Separate retrieval from XML processing
- Reproduce the failure at the layer that serves the file
- Confirm the repair in production
- Check the path after a filename rewrite
- Frequently asked questions
- Check a page from the repaired sitemap
If Search Console says your sitemap could not be fetched, start with the exact address in that report and download it as a new visitor. Check the final status, the response body, and the time Google last tried it. A login page, a transient server failure, or the wrong sitemap address can explain the error even when your articles open normally. Google’s Sitemaps report help
Copy the submitted URL before testing another address
Open the failed row and record the complete URL, the property selected, and the reported last attempt. A site might have a sitemap at /sitemap.xml, /sitemap_index.xml, or /wp-sitemap.xml. A plugin change can leave the old address in Search Console while the new file works. Testing the new address without comparing it to the submitted one leaves the reported failure unexplained.
Follow the link from the live robots.txt as a second route. Compare its hostname, protocol, and path with the submitted address. A www hostname that still points to an earlier deployment can serve a different response from the bare domain. A reverse proxy might also forward the page route correctly while keeping a special rule for XML files. Save the two addresses as distinct test cases whenever they differ.
Check who generates each sitemap. A static file uploaded years ago and a current dynamic route can both exist. Determine which one describes the publication today before removing or replacing anything. The XML sitemap guide explains how the sitemap fits into discovery. Here the immediate question is which response Google was asked to fetch.
Save the status and the complete response
Use a GET request that follows redirects and saves both headers and content. A HEAD request can exercise different server logic and omits the bytes that reveal an HTML challenge. Work with a public sitemap, and keep any local diagnostic files outside the site's published assets.
curl -sSL --max-time 30 -D sitemap-headers.txt -o sitemap-response.xml https://example.com/sitemap.xmlRead the header chain and inspect the beginning of the file. XML might start with a declaration or directly with its root element. An HTML login form, application homepage, or CDN message identifies the wrong representation even if the response says 200. If the file is compressed, determine whether compression is part of the HTTP transfer or the sitemap object itself before choosing a decoder. See content type and encoding checks.
| What the request returns | Next investigation |
|---|---|
| Redirect to sign-in | Access protection on the sitemap route |
| HTML with a success status | Application fallback, error handler, or edge challenge |
| 404 on an old sitemap path | Current generator and submitted address |
| Intermittent 5xx or timeout | Generator workload and hosting logs |
| XML that stops halfway through a URL | Truncated response or incomplete file publication |
Repeat a failed request once at an ordinary interval and retain the time of each attempt. Avoid rapid loops that add load to a generator already timing out. A single successful response establishes availability at that moment; it does not explain a five-minute outage earlier in the day.
Separate retrieval from XML processing
Once you have the intended bytes, parse them with an XML parser. A browser's styled sitemap view can be helpful, but it may conceal the underlying structure. A missing namespace, unescaped ampersand, or incomplete closing tag can prevent processing. Google distinguishes problems fetching the file from problems reading a file it retrieved, and the detailed report helps choose the next step.
For a sitemap index, fetch one of its child files as well. The index can return correctly while a child requires authentication or redirects into another application. Check children across the different generators or content types represented in the index. A blog sitemap and product sitemap can be served by separate services under the same hostname.
python3 -c 'import xml.etree.ElementTree as E; r=E.parse("sitemap-response.xml").getroot(); print(r.tag, len(r))'This check reports the root and immediate entry count. It cannot establish that every listed address is canonical, public, or worth indexing. Use it to answer the narrow question of whether the downloaded response is complete, parseable XML. Keep the original response if you format a copy for reading.
Reproduce the failure at the layer that serves the file
Compare the failed request time with application and edge logs that you are authorized to access. If a dynamic sitemap queries every product on each request, a large catalog can make the route slower than ordinary pages. Generate the file in a controlled publishing step or use the platform's established caching mechanism when the evidence points to that bottleneck. Confirm that newly published pages still enter the next generated file.
If the response is a challenge, find the rule that matched that specific request and verify the crawler identity through the operator's documented method. A user-agent string alone cannot identify Googlebot. The crawler verification guide covers this check. Apply a rule correction to the intended public endpoint and test an unrelated protected route as a control.
A sitemap can also fail during release handover. A generator might publish the index before uploading its children, or replace a file while it is still being written. Publish complete artifacts and make the public switch after they are available. Record the sequence of deployment events so a successful request after the release does not erase the evidence of the earlier gap.
Confirm the repair in production
Retest the exact reported URL with a fresh GET. Parse the result and fetch a representative child when the root is an index. Verify a recently published canonical page is present, then open that page independently. A sitemap repair that accidentally restores a stale file can look healthy while omitting the current catalog.
Google's help recommends correcting a failed sitemap and resubmitting it when repeated fetch or parsing failures have stopped processing. Use the report's current controls and keep the resubmission time with your evidence. Its processing state can lag behind a live successful request. Read the next recorded attempt rather than repeatedly changing a file that now serves correctly.
Keep a small acceptance note containing the submitted URL, previous failure, corrected layer, live response status, parsed entry count, and a sample new page. Include one older working sitemap as a comparison. That record gives the next person something reproducible when a platform upgrade or domain change reintroduces the same symptom.
Check the path after a filename rewrite
A server rule that routes all unknown files into the application can intercept a mistyped sitemap address. Compare the requested filename with the actual generator route and inspect the first body element. If the result is the homepage, correct the declaration or route and repeat the exact request. Keep a deliberately nonexistent XML path as a control so future fallback changes remain visible.
Frequently asked questions
Does a sitemap failure mean all pages were removed from Search?
The report describes processing of the sitemap. Inspect important page URLs individually to understand their indexed state and public availability.
Should I submit a new sitemap URL every time?
Use the maintained public location. Fix an obsolete submission when needed, and otherwise investigate the actual failure at the existing address.
Why does the file open for me but fail for Google?
Your browser may have a session, cached response, or different network path. Compare an anonymous request with the reported attempt and relevant server logs.
Check a page from the repaired sitemap
Run the free page check on a representative listed URL. Pair that result with the sitemap retrieval test to verify both discovery information and the page it describes.
Filed under Access and robots.txt. Topics: sitemaps, search-console, troubleshooting