Crawled, currently not indexed? Diagnose the actual page
Investigate important pages marked crawled but not indexed by comparing the fetched content, canonical signals, useful differences, and crawl dates.
By Crawlito · 6 min read
In short
- The status says Google fetched the URL and has not indexed it at the reported point.
- Compare Google's dated observation with the page your server returns today.
- Technical accessibility, useful content, and Google's indexing decision need their own evidence.
On this page
- Choose important examples and keep their dates
- Compare the fetched document with the expected content
- Investigate competing versions of the answer
- Test the answer against a reader's task
- Make a targeted change and follow the result
- Verify that the useful answer has an incoming link
- Frequently asked questions
- Inspect the excluded page
For a page marked “Crawled - currently not indexed,” inspect what Google fetched and compare it with the current public document. Check for a thin application shell, duplicate content, conflicting canonical signals, and a page that does not answer its intended question. Google describes the status as a common outcome and does not index every crawled page. Search Console’s Page indexing definitions
Choose important examples and keep their dates
Start with pages whose absence would matter to a real visitor. An installation guide, a product specification, or an article answering a recurring customer question is a useful sample. A tracking variant or nearly empty category may be intentionally expendable. Select a few important examples and an indexed page with the same template. That comparison gives you a way to separate an isolated content problem from a shared technical fault.
Record the exact URL, publication date, last substantial update, last crawl reported, and current status. Search Console aggregates observations that may have been made before your latest release. A developer who sees a complete article today and an old report describing it as excluded can easily conclude that the system ignored the fix. Date the evidence before making that claim.
Inspect each example individually. A report group is a starting set for investigation, and the current URL details can be more specific. Keep the page's intended purpose beside the technical fields. “Explain how to renew an expired integration token” is a testable purpose; “rank for integrations” is too broad to guide an editor through the missing information.
Compare the fetched document with the expected content
Open the URL in a new session, save its initial HTML, and inspect the rendered version. Find a distinctive sentence from the main answer in both. A polished screen can be assembled from client-side requests even when a basic fetch contains only a loading panel. Conversely, complete initial HTML might be replaced by an error when a client request fails after hydration.
Check the response status, final destination, visible title, primary heading, and the actual instructions or product facts. A 200 response with “Something went wrong” is a content delivery failure even if the site's navigation is present. A cached sign-in page can also have normal-looking metadata generated by a shared layout. The raw versus rendered HTML guide helps capture the difference.
curl -sSL --max-time 30 -D page-headers.txt -o page-source.html https://example.com/guides/renew-tokenCompare with the indexed control page using the same method. If both use the same layout but only the excluded guide lacks its main text, inspect the content record and data dependency for that guide. If every page in the new section returns a shell while the old section is server-rendered, the route template becomes a stronger lead.
Investigate competing versions of the answer
Find pages on your own site that answer the same question. Compare the body, title, selected product version, and canonical address. A print page, language fallback, or imported duplicate can describe the same content under multiple URLs. Decide which address is supposed to represent that answer and make the site's own links, sitemap, and canonical preference consistent.
A self-referencing canonical does not make identical copies meaningfully different. If two articles were created for synonymous queries and repeat the same steps, examine whether they should be one maintained guide. Preserve the material readers need and map retired URLs to a relevant destination when consolidating. The canonical troubleshooting guide covers how to test the resulting signals.
For pages that should remain distinct, describe the real difference in the visible answer. Instructions for two incompatible API versions should clearly identify their version and show the commands for each. Simply changing headings while copying the same examples gives readers little basis for choosing. Link the versions where the choice matters and confirm that a visitor can recognize which one applies before following the procedure.
Test the answer against a reader's task
Ask a colleague to complete the task using only the page and its linked prerequisites. For the token example, can they identify the affected integration, determine whether they have permission to renew it, find the correct settings, and verify that the connection works afterward? Note where they must guess or search elsewhere. Those missing steps are more concrete editorial work than a request to add a few hundred words.
Use examples drawn from your product or clearly labeled fictional fixtures. A screenshot without version context can be less useful than a short code sample showing the current field name. Define terms that matter to the procedure, include failure conditions someone will encounter, and explain how to confirm success. Keep claims tied to evidence the publisher can maintain. Google's guidance on useful content offers questions for reviewing whether a page serves people well. Google’s people-first content guidance
Review the control page too. Its indexed state does not make every editorial choice on it a rule. It is useful as a comparison for template behavior, route access, and the type of answer the publication already supports. Avoid copying filler or adding unrelated sections merely because the control happens to be longer.
Make a targeted change and follow the result
Write a concrete diagnosis for each page. For example, the public response omitted the renewal steps because its data lookup required an editor session. Repair that dependency and test the route anonymously. Or the page duplicated a current guide while linking to an expired product version; merge the useful details and update the navigation.
After the change, save the new response, run the relevant page tests, and use Search Console's available inspection or recrawl workflow when appropriate. Record the requested action and time. Google's eventual indexing decision remains an observation to check later. A successful live fetch establishes that the corrected page is available now.
Follow a small fixed set of URLs over subsequent reporting updates. Track whether their content changed, whether a new crawl appears, and whether the indexed state changes. Keep the unchanged control in the set. If you simultaneously change all URLs, titles, templates, and internal navigation, it becomes much harder to learn which observed defect you actually repaired.
Verify that the useful answer has an incoming link
Take the excluded guide and find an existing public page where a reader would naturally need its next step. Follow the link from that page and inspect its destination. A guide can have complete text while being absent from the section's maintained navigation. The orphan page guide helps identify that situation.
Add a descriptive link in a relevant location and test the journey as a new visitor. Record why that source page is useful context for the guide. This gives the content a real place in the publication and a concrete navigation improvement you can verify independently of a later indexing decision.
Frequently asked questions
Does this status prove the article is low quality?
It reports a crawl without indexing. Diagnose the fetched page, duplicate signals, and usefulness before assigning a particular cause.
Should I request indexing every day?
Use the inspection workflow after a meaningful correction when appropriate. Repeated requests provide no substitute for evidence that the page improved or a technical fault was fixed.
Can a page move out of this status later?
Its reported state can change as Google revisits and evaluates it. Keep crawl dates and revision dates so later observations can be interpreted accurately.
Inspect the excluded page
Use the free page check on one important URL from the report. Start with the content and technical signals a public request can verify, then investigate the page's distinct purpose.
Filed under Strategy and audits. Topics: search-console, indexing, content-quality