iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
If a page is not appearing in Google, use URL Inspection to check what Google knows about that URL and whether it can access the current page. Then use Crawl Stats to look for site-wide crawl or server problems. These reports help separate a page-specific issue from a broader pattern, but neither guarantees that Google will index a page or do so on a fixed schedule.
What each Search Console report can tell you
| Report or view | Scope | Best question to ask | Important limit |
|---|---|---|---|
| URL Inspection: Google Index | One URL, based on Google’s stored index information | What does Google currently know about this URL, and when was it last crawled? | It reflects the version represented by Google’s index data. Changes made after the last crawl may not appear. |
| URL Inspection: live test | One URL as currently served | Can Google fetch and process the current page, and does it appear eligible for indexing? | A successful test does not prove that Google has indexed the page; it is not as comprehensive as the indexed view. |
| Crawl Stats | Site-level crawl history for root-level properties | Are crawl volume, response times, host availability, or response patterns changing? | It is an advanced report, its example URLs are not exhaustive, and it offers no crawl-rate control. |
| Page Indexing report | Site-wide summary of known URLs | Are indexed and non-indexed totals or issue categories changing across the site? | Reporting can lag several days, and URLs Google does not know about are not included. |
Google describes URL Inspection as a way to see information from its index and to test a page live; the views answer different questions. See Google’s guide to inspecting and troubleshooting a single page.
Check the URL’s indexed status first
- In Google Search Console, select the property that matches the URL’s scheme, host, subdomain, and path.
- Paste the full URL into the URL Inspection field.
- Read the Google Index results before running a live test. Note whether the URL is indexed, its “Last crawl” date, whether Google could fetch it, whether crawling and indexing are allowed, and the Google-selected canonical.
The index view is based on Google’s stored information. “Last crawl” tells you when the page was last crawled for the data being shown; if you changed the page afterward, the report may not reflect that update yet. A URL that is known to Google but excluded from the index is a different case from a URL Google has not discovered. Google’s URL Inspection guidance explains these report details.
Pay particular attention to crawl permission, indexing permission, fetch status, and canonical selection. A URL blocked from crawling or indexing needs a different fix from one whose canonical points to another URL. Even when Search Console says a URL is on Google, that does not guarantee it will appear for a particular search.
#1 Best Overall
Test the page as it is served now
From URL Inspection, choose Test live URL to check the current page. This can reveal a present-day fetch, robots.txt, or noindex problem that the stored index view does not yet reflect. It is a technical check of the live page, not proof that Google has stored or indexed this version, and it does not cover every indexing condition.
- If the test reports a fetch or availability problem, address that before requesting another indexing attempt.
- If crawling is blocked, review the relevant robots.txt rule and any access changes.
- If the page sends a noindex directive, remove or correct it only if the page is intended to be indexed.
- After making a fix, rerun the live test to confirm the current page is accessible and eligible as far as the test can establish.
For the distinction between index data and the live test, refer to Google’s single-page inspection guide.
Use Crawl Stats to look for site-wide patterns
In Search Console, open Settings > Crawl stats. The report is available for root-level Domain properties and root URL-prefix properties. It is intended for advanced users; Google says sites with fewer than 1,000 pages generally should not need this level of crawl detail. See Google’s Crawl Stats report documentation.
Review changes over time rather than treating one data point as a verdict. The report includes crawl request volume, average response time, host status, response categories, file types, crawl purpose, and Googlebot type. Where useful, segment the data by Googlebot type, file type, or whether a crawl was for discovery or refresh. An increase in discovery crawling may follow a substantial content addition or sitemap submission, but the report does not show a complete request log: its example URLs are representative, so a URL’s absence from examples does not establish that Google never requested it.
Interpret a crawl decline without jumping to conclusions
A drop in crawl volume is a clue to investigate, not a diagnosis on its own. Compare the timing with technical or site changes, then check for issues that could affect Googlebot across multiple pages:
- A newly broad robots.txt rule or another change that restricts crawling.
- CSS or JavaScript files needed to render pages that Googlebot can no longer access.
- Slower server responses, server errors, or host availability problems.
- Changes in how often content is updated or other factors that may affect crawl frequency.
Google says it can reduce requests to avoid overloading a slow or erroring server. Crawl Stats does not provide a setting to raise the crawl rate. Google also notes that less frequently changing content or content it does not consider high quality may be crawled less often; a site-wide crawl pattern alone cannot establish why a particular URL is being crawled less. See Google’s guidance on managing crawl budget and the Crawl Stats documentation.
Rank #4
Choose the right next step and wait an appropriate interval
For one important page
Fix any access or indexing issue first. Once the page is corrected and the live test is successful, request indexing from URL Inspection. Google treats this as a request, not a guarantee. Its guidance says processing may take up to a week or two, so submitting the URL again does not ensure a faster result. See Google’s URL Inspection guide.
Recommended Free Tools
For many pages
Submit or update a sitemap rather than requesting indexing URL by URL. Keep last-modified dates accurate where applicable; they should reflect real changes. A sitemap helps Google discover URLs but does not guarantee indexing. See Google’s sitemap guidance.
For a page that still seems missing
Google’s missing-page guidance recommends allowing at least a week after submitting a sitemap or indexing request before assuming there is a problem. Its broader estimate for discovering and indexing a new page ranges from a day or two to a few weeks, depending on many factors; this is a possible range, not a promise. Check the Page Indexing report for site-wide patterns, keeping in mind that its data can lag several days and that URLs Google does not know are not counted. See Google’s missing-page guidance and the Page Indexing report documentation.
Quick Recap
A practical diagnostic sequence
- Verify the property. Make sure the inspected URL belongs to the selected Search Console property.
- Read Google Index data. Check indexed status, last crawl, fetch outcome, crawl and indexing permissions, and Google-selected canonical.
- Test the live URL. Resolve any current fetch, access, or noindex issue, then run the test again.
- Check site-wide crawl context. If multiple pages are affected, review Crawl Stats for changes in host status, response time, errors, and crawl volume.
- Submit appropriately. Request indexing for a single important corrected URL; use a sitemap for many URLs.
- Monitor with patience. Allow time for processing, then compare URL Inspection and Page Indexing data rather than repeatedly resubmitting.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
