Commentary on the August 2020 Google Webmaster Central office-hours hangout
In the latest Google Webmaster Central office-hours hangout, John Mueller answered some questions on indexation of paginated content. Some of his comments have created a buzz of discussion, even argument, in the SEO community.
So, I’d like to shed some light on the topic and help decipher what he means and also help you in your own SEO decisions.
Firstly, below you’ll find a transcript of the YouTube video. If you read through it, you not only get context around the topic but also probably will pick up on a few points that can easily be misinterpreted.
The questions asked in the Hangout were:
- What’s the importance of pagination?
- Are there issues with pagination?
- What about infinite scroll?
- Should paginated pages be indexed?
- What is the best practice of pagination?
The Transcript
“… So, essentially when we talk about pagination, we mean you have one thing that is really large, and it doesn’t fit on one page, so you split it across multiple pages. And that could be maybe one long article where you say, well, it’s worthwhile splitting this. It could be a list of individual products, maybe in a category, where you say, well, I have 5,000 products, I can’t put them all on one page, I will put them on five pages, or 1,000 pages, or whatever.
“So that’s kind of where pagination comes in. And from our point of view, it’s important that we can recognize this kind of pagination and essentially index those individual pages so that we can pick up all the content or all the links to individual items that you have on the paginated pages.
“Usually with pagination, one of the questions that comes up is, well, ‘this creates a lot of new URLs’ because you have to go through all of these pages to get all of the content. And yes, it does generate a lot of URLs, and we have to index a lot of different pages.
“But if we want that content in our index, if we want to understand those internal links to your other pieces of content, we kind of have to do that.
“Infinite scroll is a way of doing pagination without the user having to click ‘Next’. The important part there is that you do infinite scroll in a way that makes that works for Search, with regards to crawling and rendering in particular. And we have some guidelines on how to do that.
“So in particular, one recommendation is to have separate URLs for each page so that you can still go to the individual pages, to have links to those individual pages. And for the user if they scroll down to the bottom then it’s fine to load the next page kind of thing.
“Should all paginated pages be indexed? Yes. Kind of like I mentioned before, if there is something on there that you want to have known by Google – which could be a link to a different product, which could be a part of a longer piece of content then it has to be indexed. And the best practices kind of following up on the other steps there.
“We have to know about these paginated pages, so you have to link to them, usually with a next and previous link. And with normal HTML links, we can pick that up fairly easily. You don’t have to do anything special for this kind of pagination.
“So just link from one page to the next, and link through to the previous page as well. Then we can crawl through all of those paginated pages.
“I kind of ran through this, and I know there is a lot more, depending on what kind of setup you might have. So this is something where we will probably have a bit more information, in particular for e-commerce sites, to make it a little bit easier to understand what exactly should be indexed with pagination.
“In general, we find that most sites implement pagination in a way that just works, so we’ve stepped back from saying we need to define exactly what people need to do for pagination, but rather, if you understand these pages need to be indexable, then you understand they need to be linked. You can test them with a local crawler and usually that just works out.”
Here is the Hangout recording
So, what’s the problem?
Many SEOs thought he said paginated pages have to be indexed. This is because when he answered this question “Should all paginated pages be indexed? with “Yes”. The key is to keep listening/reading though…
Because, he says shortly after, “if there is something on there that you want to have known by Google [ ] then it has to be indexed.” (in bold above)
It says ‘want’. So, now, just like in the past, it’s always up to you what gets indexed and what doesn’t (although, some of you might have noticed, like I have, that even noindexed pages can show up in the index, but that’s beyond the scope of this post).
I hope that helps to clarify any misunderstanding. Write to me in the comments below if you have any questions about pagination or any other SEO topics.
Now, what should you do?
Well, my question to you is: do you want your category pages from 2 to 5000, whatever your situation is, to be indexed by Google? Are they important for your website and business?
If not, there are ways to have all your products and articles crawled, indexed and evaluated by Google according to your website architecture setup. Your products and articles will be found even when your paginated pages are noindexed. Here’s how:
- Submit your product or article pages to Google with an XML Sitemap
- Link to them from non-category pages. For example, link to other pages from product pages with similar products, or from product pages with “customers who buy this also buy the following…” product links.
Category information can also be important information for Google to access, so this is where your website architecture comes into play – your category name should be in the URL, like ‘blue jeans’ in this example: https://yourwebsite.com/blue-jeans/levis501.html. With URLs like this, Google understands that this specific product belongs to this specific category.
To further indicate category to Google, set up breadcrumbs using Schema markup telling Google the exact same ‘story’ as your URL – that this specific product belongs to this specific category.
And, a final tip…
For your most important products – those you want to sell more of and have ranked well – be sure to list those products on the top of the list on your category page. For example:
In short, nothing’s changed
Google doesn’t pay attention to rel=”next” or rel=”prev”. They’ve long stated that paginated category pages on blogs or online shops usually do not have valuable enough content to be indexed. They did state that noindexed pages might be rated as “Soft 404” which could mean that links found on these pages are not given any value.
In summary, what John said is not new and not surprising.