.png&w=2560&q=75&dpl=dpl_HzA238NApwd1uQehZhBej9TfGf61)
Pages usually fail to show up in Google for one of five reasons: Google has not found them, it has been told not to crawl or index them, it considers them a duplicate of another URL, it cannot see their content when rendering, or it has crawled them and decided they are not worth indexing yet. The fastest way to find out which applies is to paste the URL into the URL Inspection tool in Google Search Console, which tells you whether the page is indexed and, if not, why.
Start with the right diagnostic question
Work through these in order.
- Is the page actually missing? Check with URL Inspection. Sometimes a page is indexed but simply ranks poorly — a different problem.
- Has Google discovered the URL? Is it in your XML sitemap and linked from at least one other page on your site?
- Is Google allowed to crawl it? Is it blocked by robots.txt or does it require a login?
- Is Google allowed to index it? Does it carry a noindex directive?
- Is it the canonical version? Or has Google chosen a different URL to represent it?
- Can Google see the content? Does the rendered HTML include the main text and links?
- Is the page worth indexing? Is it substantial, original and useful compared with other pages?
What the Page indexing report is telling you
Search Console's Page indexing report groups non-indexed URLs by reason. These are the statuses you will see most often:
| Status | What it means | Typical fix |
|---|---|---|
| Discovered – currently not indexed | Google knows the URL but has not crawled it yet | Strengthen internal links; reduce crawl waste |
| Crawled – currently not indexed | Google crawled the page and chose not to index it for now | Improve content quality and uniqueness; consolidate thin pages |
| Excluded by 'noindex' tag | The page tells Google not to index it | Remove the noindex if the page should rank |
| Blocked by robots.txt | Google is not allowed to crawl the URL | Adjust robots.txt if the page should be crawled |
| Duplicate without user-selected canonical | Google sees it as a duplicate and picked another URL | Set a clear canonical, or make the page distinct |
| Duplicate, Google chose different canonical than user | Google disagreed with your canonical tag | Check that the content, internal links and sitemap all point to the same preferred URL |
| Alternate page with proper canonical tag | Working as intended — this page points to another canonical | Usually no action |
| Page with redirect | The URL redirects elsewhere | Usually no action; remove it from your sitemap |
| Not found (404) | The URL does not exist | Redirect if it has a relevant replacement; otherwise fine |
| Soft 404 | The page returns 200 but looks empty or like an error | Add real content, or return a proper 404 |
| Server error (5xx) | Your server failed when Google requested it | Investigate hosting, firewall or rate-limiting |
Not every non-indexed URL is a problem — redirects, deliberate noindex pages and alternates belong there. Focus on URLs you want in Google.
robots.txt and noindex do different jobs
This is the most common source of confusion.
- robots.txt controls crawling. It tells Googlebot which paths it may request. It does not reliably keep a page out of the index — if other sites link to a blocked URL, Google can still index the URL without its content.
- noindex controls indexing. A
<meta name="robots" content="noindex">tag or anX-Robots-Tag: noindexHTTP header tells Google not to show the page in results.
The crucial interaction: Google must be able to crawl a page to see its noindex. If you block a page in robots.txt and add noindex, Google cannot read the noindex. To remove a page from results, allow crawling and use noindex; once it has dropped out, you can decide whether to block crawling as well.
Canonical problems
A canonical tag tells Google which URL you prefer among duplicates, but it is a strong hint, not a command. Google may choose differently when signals conflict — say, a canonical pointing to URL A while internal links and the sitemap use URL B. Make canonical tags, internal links, sitemap entries and redirects all point to the same version.
Rendering problems
If your site builds content with JavaScript, Google has to render the page to see it. Use URL Inspection's "Test live URL" to check the rendered HTML. Missing text or links usually come from content loaded only after user interaction, blocked scripts or APIs, or rendering errors. Delivering main content in the initial HTML — server-side rendering or static generation — avoids most of these issues. See our guide to Next.js business websites for how modern frameworks handle this.
Quality and "Crawled – currently not indexed"
When Google crawls a page but does not index it, there is often nothing technically wrong: the page may be too thin or too similar to others. Merge near-duplicate pages, add genuine substance, and link to the page from relevant, stronger pages. Content that demonstrates first-hand knowledge, as discussed in our guide to E-E-A-T, tends to fare better.
Frequently asked questions
Does "Request indexing" guarantee a page will be indexed?
No. It only asks Google to recrawl the URL sooner.
How long does indexing take?
Anything from hours to weeks. Strong internal linking helps.
Key takeaways
- Use URL Inspection first; it tells you whether and why a page is not indexed.
- Read Page indexing statuses carefully — many exclusions are intentional.
- robots.txt controls crawling, noindex controls indexing, and Google must crawl a page to see its noindex.
- Align canonicals, links and sitemaps, and make sure content is visible after rendering.
If important pages are missing from Google and you cannot see why, Wot Digital's technical SEO team can diagnose and fix the cause. For a broader review of everything technical, see what a technical SEO audit covers.
