Some of the pages of your site serve a purpose, but that purpose isn’t ranking in search engines or even getting traffic to your site. These pages need to be there as glue for other pages, or simply because whatever regulations require them to be accessible on your website. As a regular visitor to our website, you know what noindex or nofollow can do to these pages. If you are new to these terms, please read on and let me explain what they are and what pages they might apply to!
What is noindex or nofollow?
Both are settings you can add to your robots meta tag. We did quite an extensive ultimate guide on the robots meta tag that you should read.
In short:
- It looks like this in most cases:
<meta name="robots" content="[VALUE1,VALUE2]"> - VALUE1 and VALUE2 are set to
index, followby default, meaning the page at hand can be indexed and all links on that page can be followed by search engine spiders to index the pages they link to. - VALUE1 and VALUE2 can be set to
noindex, nofollowas well.noindexmeans that the page shouldn’t be indexed by search engines, but doesn’t mean the search engines shouldn’t follow the links on the page.nofollowmeans that it also shouldn’t follow the links.
Pages that you might want to noindex
Author archives on a one-author blog
If you are the only one writing for your blog, your author pages are probably 90% the same as your blog homepage. That’s of no use to Google and can be considered duplicate content. To keep these out of the search results, you can noindex them.
Certain (custom) post types
Sometimes a plugin or a web developer adds a custom post type that you don’t want to be indexed. At Yoast, we use custom pages for our products, as we are not a regular online shop that sells, for instance, kitchen appliances. We don’t need a product image, filters like dimensions and technical specifications on a tab next to the description. Therefore, we noindex the regular product pages WooCommerce outputs and are using our own pages. Indeed, we noindex the product post type.
By the way, I have seen shop solutions that added things like dimensions and weight as a custom post type as well. These pages are considered to be low-quality content. You will understand that these pages have no use for a visitor or Google, so need to be kept out of the search result pages.
Thank you pages
That page serves no other purpose than to thank your customer/newsletter subscriber. Usually thin content, or upsell and social share options, but no added value content-wise.
Admin and login pages
Of course, your login pages are not in Google. But these are. Keep them out of the index by adding that noindex. Exceptions are the login pages that serve a community, like Dropbox or similar services. Just ask yourself if you would google for one of your login pages if you were not in your company. If not, it’s probably safe to say that Google doesn’t need to index these pages.
Internal search results
Internal search results are like the last pages Google wants to point its visitors to. If you want to ruin a search experience, you link to other search pages. But the links on that search result page are still very valuable for Google, so all links should be followed. The robots meta setting should be:<meta name="robots" content="noindex, follow">
The same setting goes for all the examples mentioned above, there is no need to nofollow the links on these pages. Now, when should you add a nofollow to your robots meta tag?
Pages that you might want to nofollow
Google roughly indicates that there are three reasons to nofollow links:
- Untrusted content
- Paid links
- Crawl prioritization
For instance, we add a nofollow tag to links in comments. We don’t know what all of you are dropping in there, right. It could be anything from the #1 and #2 above. With regards to number 3, this could, for instance, apply to login links, that we sometimes find on WordPress websites, see image on the right. It’s no use having a Googlebot go over these links, as for search engines, they add no value. These are nofollowed.
All of the above is very much on a link level. But if you have for instance a page that shows SEO books, with a surplus of Amazon affiliate links, these might add value to your site for your users. But I’d nofollow that entire page if there’s nothing else that matters on the page. You might have it indexed, though. Just make sure you cloak your links the right way.
To be honest, on a regular website, I don’t think there are a lot of pages I’d set to nofollow. Check for yourself if you have any content that mainly contains links like the ones Google indicated, and decide if Google should follow them or not.
Changing SEO insights
At Yoast, we always try to keep you on top of your SEO game, without per se bugging you about it. One of the settings in Yoast SEO that we have had for years, the “Noindex subpages of archives” checkbox is one of those. It made all the sense in the world to noindex, follow these, and have Google index just the main page, the first page of your (f.i.) category archive.
We were always aware that Google was getting better and better at understanding rel="next" and rel="prev" on these subpages of archives. Yoast SEO adds these tags as well. At this point, we know that rel="next" and rel="prev" cover the way archives should be indexed and noindex-ing subpages isn’t necessary anymore, so we’ve removed that setting from our plugin altogether to make sure it’s done right on your site!
Read more: ‘Prevent your site from being indexed, the right way’ »