Meta Robots Tag (Noindex / Nofollow)
In one line
Learn what a meta robots tag is, how to properly implement noindex and nofollow directives, and view exact HTML code examples to protect your website.
Definition & overview
Meta robots tag (noindex / nofollow) is an HTML element placed in a webpage's <head> section that instructs search engines on whether to index the page and crawl its links. It matters because it prevents duplicate content and protects search visibility.
Teams across the industry often grapple with accidental deindexing of high-value pages or draining crawl budgets on low-value URLs. Managing technical optimization requires precise robots meta directives to control organic search visibility. Search engines default to an "index, follow" state if the tag is missing. But adding a robots meta tag allows marketers to override that default behavior to protect a website from duplicate content penalties and preserve search rankings.
Keep in mind that indexing and crawling serve completely different functions. These robots meta tags (or robot meta tags) control indexing, so they dictate if a page appears as a search snippet on the SERP. A robots.txt file controls crawling, so it tells a specific user-agent like Googlebot which directories it can access.
| Directive | Indexing Behavior | Link Crawling Behavior |
|---|---|---|
| index, follow | Shows in search results | Passes value to outbound links |
| noindex, follow | Hidden from search results | Passes value to outbound links |
| index, nofollow | Shows in search results | Ignores outbound links |
| noindex, nofollow | Hidden from search results | Ignores outbound links |
How to implement meta robots tag (noindex / nofollow)
Adding page-level instructions requires a few straightforward steps depending on your website architecture.
- 1Identify the specific page you want to remove from search results or restrict from passing link equity.
- 2Access the page's HTML <head> section directly or open your CMS SEO plugin settings for platforms like WordPress or Shopify to configure your meta robots settings.
- 3Insert the tag with the correct comma-separated directives.
- 4Save your changes and verify the implementation using a site audit tool or browser inspection.
Example
Search engines look for this syntax specifically within the hidden section of a document. Here is a clean, copy-pasteable HTML code snippet showing the exact parameters needed to hide a page and ignore its links.
<head> <title>Internal Staging Page</title> <meta name="robots" content="noindex, nofollow"> </head>
Common mistakes
Technical SEO audits frequently reveal the same critical errors. A common challenge for practitioners is accidentally deindexing pages by mixing conflicting directives. Avoid these practical missteps to maintain control over your crawler behavior.
- Blocking a noindexed page in robots.txt: Never block a URL in your robots.txt file while also using a noindex meta tag. A search engine must be able to crawl the page to see the noindex instruction, so blocking the crawl prevents the deindexation from taking effect.
- Placing the tag in the wrong section: The HTML code must live in the <head> of the document. Dropping it into the <body> renders it invisible to crawlers.
- Applying site-wide noindex accidentally: Developers sometimes push staging sites to production with a site-wide noindex tag still active, wiping out search visibility overnight.
- Ignoring server-level rules: Sometimes a page remains hidden even when the HTML tag is correct. This happens when an X-Robots-Tag is deployed via an HTTP header, sending a conflicting server response that overrides your on-page crawl restrictions.
| Error Type | Consequence |
|---|---|
| Meta Tag Error | Page is accidentally removed from search results or remains indexed against your wishes. |
| Robots.txt Error | A search engine can't crawl the page to read your directives or content updates. |
Frequently asked questions
How to check meta robots tag?
To check this HTML element before consulting your search console, right-click the webpage and select "View Page Source." Next, use Ctrl+F to search for <meta name="robots". This highlights the exact tag and directives currently active on the page.
Where do I find a meta tag?
Meta tags are always located within the hidden <head> section of an HTML document. Web crawlers look for these instructions at the very top of the code, so they must appear before the visible <body> content begins.
What is the difference between noindex and nofollow?
The noindex directive tells a search engine to hide the page from search results, altering indexing behavior. The nofollow directive tells the engine to ignore all outbound links on that page, preventing the transfer of link equity.
| Directive | Definition |
|---|---|
| noindex | Prevents the page from appearing in search engine results. |
| nofollow | Stops crawlers from following and passing value to outbound links. |
Read next · related terms
Want this handled for you?
See how your site performs across Google, AI Overviews, ChatGPT, and Gemini.
Get your free visibility report