What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
To keep an accessible HTML page out of Google Search, add <meta name="robots" content="noindex"> to its <head>. Googlebot must be able to crawl the page to read the directive. This prevents the page from appearing in search results; it does not make the page private or stop bots from visiting it.
Use the robots meta tag for an HTML page
Add this element inside the page’s <head>:
<meta name="robots" content="noindex">Google documents noindex as the directive that prevents a page from appearing in Google Search after Googlebot reads it. The conventional place is the <head>. Google says its crawler also respects robots meta tags in the body, but the head is the clearest, standard placement. The name and content values are case-insensitive for Google. See Google’s robots meta tag and X-Robots-Tag documentation.
noindex is about search-result visibility, not crawler access. Google Search Central explains: “Keep in mind that these settings can be read and followed only if crawlers are allowed to access the pages that include these settings.”
Allow Googlebot to read the directive
Do not block the page in robots.txt if you want Google to process its noindex. A robots.txt disallow can keep Googlebot from fetching the page and seeing the meta tag. The URL may still appear in results if Google learns about it from links or other information, even though it cannot crawl the page’s contents. Google explains this distinction in its robots.txt introduction and guide to blocking indexing.
#1 Best Overall
If the page is already in search results, Google needs to recrawl it before it can act on the directive, so removal is not necessarily immediate. Use Search Console’s URL Inspection and Page Indexing reporting to check what Googlebot received and whether the page is still indexed.
Choose the right method for the resource
| Goal | Use | What it does |
|---|---|---|
| Exclude an HTML page from Google Search | <meta name="robots" content="noindex"> |
Signals that Google should not show the page in results after crawling it. |
| Exclude a PDF, image, video, or other non-HTML resource | X-Robots-Tag: noindex in the HTTP response header |
Provides the noindex directive without requiring HTML markup. See Google’s documentation. |
| Limit crawler access or manage crawl load | robots.txt |
Controls crawling, but is not a reliable way to keep a URL out of search results. |
| Keep confidential material private | Password protection or other access control | Restricts access; a robots directive does not protect content. |
Know which directive to use
noindex excludes a page from results
Use noindex when the page may be crawled but should not be shown in Google Search.
Rank #2
nofollow concerns links
nofollow concerns whether a crawler should follow links on the page; it is not a substitute for noindex. If you need both behaviors, Google supports <meta name="robots" content="noindex, nofollow">. Google defines none as equivalent to noindex, nofollow.
Recommended Free Tools
Target a Google crawler specifically when needed
For Google-specific targeting, Google documents <meta name="googlebot" content="noindex">; it also documents googlebot-news for Google News results. A general robots meta tag is intended for search crawlers that support the directive, not necessarily every kind of bot. If you need to address a non-search crawler, target that crawler according to its own documentation. Other search engines may interpret supported directives differently, so do not assume Google’s behavior applies universally.
Rank #3
Do not confuse noindex with snippet controls
Rules such as nosnippet, preview-length limits, and noimageindex control snippets or image indexing; they do not replace noindex when the goal is to keep the page itself out of search results. Google also supports indexifembedded in combination with noindex for a specific embedded-content use case. See Google’s directive reference before combining rules.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
