Crawling and Indexing for Local SEO

The most useful starting point is the exact symptom. "Not ranking" is too broad. A URL can be unknown to Google, discovered but not crawled, crawled but not indexed, treated as a duplicate, canonicalised to another URL, indexed but rarely shown, or indexed and visible only for a narrow set of searches. Those are different problems and need different checks.

Start with the exact symptom

Before changing the page, write down the full URL and place it into one of these branches:

Search Console can provide useful first-party evidence, but it does not expose every internal indexing decision. A status label is a diagnostic clue, not a complete explanation of Google's systems.

indexed version of a URL from a live test. Google also states that a successful live test does not guarantee that the URL will be indexed.

Discovery, crawling and indexing are different stages

A local-business page needs to pass through several stages before it can compete in search results.

Discovery means Google has a route to the URL. That route might come from a crawlable internal link, a sitemap, an external link, a redirect or another source.

Crawling means Googlebot requests the URL and receives a response. A discovered page can remain uncrawled for a period of time. Crawling can also fail because of server errors, authentication, blocked access, redirect problems or other technical issues.

Indexing is the separate decision to process and retain the page for potential use in Search. A crawlable page can still be excluded from the index. Google may decide that another URL is the canonical representative, or that the page does not add enough value as a separate indexed URL.

Ranking comes after indexing. Once a page is indexed, the question changes from "Can Google include this page?" to "For which queries, users and contexts is this page a useful result?" If your page is already indexed but weak, use the Why Isn't My Business Ranking on Google? diagnostic guide instead of continuing to make indexing changes.

Check the live URL before you open another SEO tool

The live page itself can often reveal the problem. Open the exact canonical URL you expect Google to index and check the basics.

1. Does the URL return the intended page with HTTP 200?

An important indexable page should normally return a successful HTTP response and the content you expect. Check for accidental redirects, server errors, soft-404 behaviour, expired pages or a URL that loads a generic fallback instead of the intended service or location content.

A browser can hide some response details, so a developer tool, crawler or header checker may be useful.

The question is not merely "does it look fine to me?" but "what response does the server actually provide?"

2. Is the intended content present when Google renders the page?

If important text or links depend on JavaScript, compare the rendered version with what users see. Search Console's live test can show the rendered page and fetched resources. For most small local-business sites, the goal should be simple: the main content, navigation and internal links should be available without fragile rendering dependencies.

3. Is there an accidental noindex directive?

Check the HTML robots meta tag and, where relevant, the HTTP X-Robots-Tag header. A noindex directive tells supporting search engines not to include the page in the index once the crawler can see that directive.

A common launch error is leaving a staging or development noindex rule in place after the page goes live.

4. Is the page blocked by authentication or staging controls?

Password protection, firewall rules, temporary maintenance settings, IP restrictions and development environments can all prevent the page being fetched normally. A page that works only for a logged-in administrator is not publicly crawlable.

5. What does the canonical tag say?

An indexable page that is intended to stand alone will commonly self-reference its final canonical URL. If the page points its canonical to a different service page, another town page or an old URL, that signal needs to be understood before you chase other fixes.

A canonical tag is a signal, not an absolute command. Google can choose a different canonical if other signals indicate that another URL is a better representative of the content.

Use Search Console URL Inspection properly

For a specific missing page, URL Inspection is usually the most useful first-party diagnostic interface.

Inspect the exact URL and distinguish between two views:

Google Index data tells you what Google's indexed systems currently know about the URL. This can include the indexing status, last crawl information, crawl allowance and canonical information where available.

Live Test data fetches the current URL and checks whether Google-InspectionTool can access and parse it. This is useful after you make a fix, but it does not reproduce every indexing decision.

What to check in URL Inspection

Look for answers to these questions:

• Is the URL in Google's index?
• Was crawling allowed?
• Was the page fetch successful?
• Is indexing allowed?
• What canonical did you declare?
• What canonical did Google select, if that information is available?
• Does the rendered page contain the content and internal links you expect?
• Is the status consistent with the live version you are viewing today?

Do not treat one green live-test result as proof that indexing will happen. Google's own documentation states that the live test does not check every possible indexing condition, including some duplicate and canonical states, and does not guarantee inclusion in the index.

Request Indexing is not a repair button

After fixing a real problem, using Request indexing can be reasonable. It is not a substitute for resolving the underlying issue.

If the page is a near-duplicate, points to another canonical, has weak discovery, contradicts the site's architecture or offers little distinct value, repeatedly requesting indexing does not change those facts.

Google also states that an indexing request does not guarantee that the page will enter the index.

Use the request as a final nudge after the page is technically and editorially ready, not as the first step in diagnosis.

For the wider measurement workflow, see Google Search Console for Local SEO.

Check how Google is expected to discover the page

A sitemap is useful, but internal discovery still matters.

Ask:

• Is the page linked from its parent topic or service hub?
• Can a user reach it through normal navigation or contextual links?
• Is it effectively orphaned except for the sitemap?
• Do relevant service and Location Pages link to it naturally?
• Does the sitemap include the final canonical URL rather than an old or redirected version?
• Are there multiple URLs for the same content competing for discovery?

Google's sitemap documentation explicitly says that a sitemap can help search engines discover URLs but does not guarantee that every submitted URL will be crawled or indexed.

For a small local-business website, crawlable internal links are especially useful because they communicate both discovery and context. A location page that can be reached only from an XML file is harder to justify as an important part of the user-facing site architecture.

robots.txt and noindex do different jobs

This distinction matters because the two controls are often confused.

robots.txt primarily controls crawling. A Disallow rule tells the relevant crawler not to fetch the matching path. Google states that a URL blocked by robots.txt can still appear in search results without a snippet if Google learns about the URL elsewhere.

noindex controls indexing, but Google must be able to crawl the page and see the directive. If you block the URL in robots.txt, Google may not be able to read the noindex instruction.

So do not use "block it in robots.txt" as a guaranteed de-indexing technique.

For an important page that you want indexed, check both sides:

• Is crawling allowed?
• Is indexing allowed?

Search Console's live inspection explicitly separates those concepts.

Check redirects, server responses and blocked resources

A page can look normal to a user and still present a confusing technical chain to crawlers.

Check whether:

• the canonical URL redirects unexpectedly;
• HTTP redirects to HTTPS cleanly;
• www and non-www variants consolidate correctly;
• a trailing-slash variant creates duplicate URLs or redirect loops;
• temporary redirects are being used where a permanent move was intended;
• CSS or JavaScript required for understanding the page is blocked;
• server errors occur intermittently;
• the page is returning a status that does not match its visible content.

Do not overcomplicate this section into a full migration guide. The immediate question is simply whether Google can request the intended URL and receive a stable, useful, indexable response.

Check canonical and duplicate signals

Local-business websites can accidentally create several versions of what is effectively the same page.

Examples include:

• HTTP and HTTPS versions;
• www and non-www hosts;
• parameters or tracking URLs;
• old and new slugs;
• printer or filtered versions;
• near-identical service pages;
• near-identical town pages where only the place name changes.

Google describes canonicalisation as choosing a representative URL from a group of duplicate or highly similar pages. You can signal your preferred canonical with redirects, rel="canonical" and sitemap consistency, but Google may select another URL.

For a page that is unexpectedly canonicalised elsewhere, compare the two URLs. Is the "duplicate" genuinely a different page that deserves independent indexing, or are you trying to force two substantially similar URLs into the index?

Do not solve an editorial duplication problem with canonical tags alone. Canonicals are useful for consolidation; they do not create distinct user value.

Ask whether this URL deserves separate indexing

This is the point many indexing checklists miss.

A technically perfect URL can still be a poor candidate for separate indexing if it adds little beyond another

page. Common local examples include:

• a town page that changes only the town name;
• dozens of templated service-area pages with no local evidence;
• a page created solely because a keyword tool showed another phrase;
• a page with so little useful content that it resembles a soft 404;
• multiple pages answering the same intent with slightly different wording.

There is no reliable minimum word count or "percentage unique" rule that makes a page indexable. Do not invent one.

Instead ask whether a user would understand why this URL exists separately. Does it answer a distinct local need, contain service-relevant detail, connect to real coverage, and offer information that would be awkward or less useful if merged into another page?

For the site's quality boundary around geographic pages, see Location Pages vs Doorway Pages.

Re-test after you fix the likely cause

Once you have made a specific change:

1. Confirm the live URL still returns the intended content.
2. Run a new live inspection if the issue is testable there.
3. Re-check robots and indexing directives.
4. Re-check the declared canonical.
5. Confirm internal links and sitemap inclusion.
6. Request indexing if appropriate.
7. Monitor the page's index status rather than assuming the change worked immediately.

Do not promise a fixed recrawl or indexing timetable. Crawl and indexing timing varies, and Google's own documentation warns that requests do not guarantee inclusion.

Keep a short change record: date, URL, symptom, evidence, change and follow-up result. That makes later diagnosis far easier than relying on memory.

Real Project: McKnight's Flat Pack Assembly

McKnight's Flat Pack Assembly is the live service-area-business case study used across LetsBuzzMedia.

The site at flatpackassembly.net was reviewed publicly on 2 September 2026. Its live pages expose normal crawlable navigation between the homepage, service pages, area pages, projects, about, booking, pricing and FAQ content.

failure, because no current Search Console export or URL Inspection screenshot showing such a failure was supplied for this page.

That means the useful case-study role here is preventive monitoring rather than a manufactured incident.

For an important Flat Pack service or area page we would record:

• final HTTPS URL;
• HTTP status;
• internal links from relevant parent pages;
• sitemap presence;
• robots and noindex status;
• declared canonical;
• Google-selected canonical where available in URL Inspection;
• current indexing status and date checked.

If a future real indexing problem occurs, it can be added to the McKnight's Flat Pack Assembly case study with the original date, evidence and outcome.

When indexing is not the problem

If URL Inspection shows that the intended canonical page is indexed, stop treating indexing as the main problem.

Weak visibility may instead involve search intent, relevance, proximity, competitive prominence, internal architecture, the strength of the page, or simply measuring from the wrong location. That belongs in the not-ranking troubleshooting guide, not here.

The practical sequence is:

identify the symptom → collect evidence → fix the likely indexing constraint → re-test → only then diagnose ranking.

Key takeaway

Do not start with a generic list of "indexing fixes". Start with the page's actual state. Discovery, crawling, indexing, canonical selection and ranking are separate stages. A sitemap can help discovery but cannot guarantee indexing. robots.txt controls crawling, not guaranteed de-indexing. A successful live inspection is useful evidence, not a promise. And if the page is indexed, move on to the correct ranking diagnosis instead of continuing to press Request Indexing.

Next step

Choose one important URL that you expected to see in Google. Inspect it in Search Console, write down which stage is failing, and make only the change that addresses that stage. If the URL is already indexed, continue with the Local SEO not-ranking diagnosis.