searchmetrics email facebook github gplus instagram linkedin phone rss twitter whatsapp youtube arrow-right chevron-up chevron-down chevron-left chevron-right clock close menu search
1283212832

Google’s move to “mobile first crawl” and the impact on SEO performance

iTunes       Overcast      Spotify      Stitcher

Episode Overview

In our next conversation with Cindy Krum, CEO of MobileMoxie, we discuss the impact of Google’s shift to a mobile first crawl. All over the world the mindset of SEOs is changing dramatically in response to Google’s prioritization of mobile content over the desktop. In this podcast we cover:

  •  Google’s rationale and the impacts on SEO performance
  • Why SEOs are misunderstanding the change and Google’s mixed messages
  • How mobile first index is organizing information around the knowledge graph versus topic clusters of URLs
  • The different types of featured snippets

GUESTS & RESOURCES:

Episode Transcript

Ben:                             Welcome back to mobile optimization week on the Voices of Search podcast. I’m your host Benjamin Shapiro and this week we’re gonna publish an episode every day covering what you need to know about the technical optimizations that will improve your performance and visibility on your mobile sites. Okay. Joining us for mobile optimization week is Cindy Krum, who is the founder and CEO of MobileMoxie, which is a mobile centric set of tools and APIs that help SEOs gain better insights into their mobile site experience. Outside of leading the charge at MobileMoxie, Cindy is the author of Mobile Marketing: Finding your customers no matter where they are. And today we’re gonna continue our conversation with Cindy and discuss the impact of Google shifting to a mobile first crawl. But before we get started, I wanna remind you that this podcast is brought to you by the marketing team at Searchmetrics.

Ben:                             We are an SEO and content marketing platform that helps enterprise scale businesses monitor their online presence and make data driven decisions. To support you, our loyal podcast listeners, we’re offering a complimentary digital diagnostic. A member of our digital strategies group will provide you with a consultation that reviews how your website content and SEO strategies can all be optimized. To schedule your free digital diagnostic, go to searchmetrics.com/diagnostic. Okay. Here’s the second installment of mobile optimization week with Cindy Krum, the founder and CEO of MobileMoxie. Cindy, welcome back to the Voices of Search podcast.

Cindy Krum:                 Thank you.

Ben:                             It’s great to have you back here for mobile optimization week. Covered a lot of ground in our last episode talking about the landscape of mobile optimization. It’s a big topic. We’re in the early stages of it, but I wanna talk about something that is kind of a change in mindset for Google and SEOs all over the world in terms of how Google is prioritizing mobile versus desktop in terms of website content. Talk to me about the rationale behind Google moving towards a mobile first crawl and what impact has that had on SEO performance.

Cindy Krum:                 So, the change to mobile first crawling is a really big deal, but I think a lot of SEOs are misunderstanding it. Or at least are understanding it differently from what I understand it to be. And I don’t think that’s their fault. I think that Google has messaged it one way, but it’s having an impact a slightly different way. So, Google talked about mobile first indexing as just switching to a mobile crawler, end of the call was gonna be … Or the indexing was based on only what was found in the mobile crawl, except that they would also keep anything that they found in a desktop crawl. That’s very confusing messaging.

Ben:                             I’m lost already.

Cindy Krum:                 Right? And it was because they kind of pivoted a couple of times in how they were messaging it, ’cause at first they were talking about a separate mobile first index. When you get moved into the mobile first index, which would imply … That’s language that implies that it’s something separate. And then they were like no, no, no. It’s one thing. It’s definitely just one thing. So, that’s all very confusing, but what I understand it as is yes they, they’ve switched to mobile first crawling and indexing, but for a long time they said that their default color was the smartphone crawlers. So, this is not a fundamental shift. Maybe they were still crawling as the smartphone, but indexing as the desktop or something else very confusing. I think that’s all subterfuge. Like it’s all just there to be confusing.

Cindy Krum:                 What’s really happening in my mind is that Google’s mobile first index is organizing information around the knowledge graph rather than around, oh I don’t know, topic clusters of URLs, which seems like how it was done before. So, organizing things around the knowledge graph is what we call entity first indexing. So, at MobileMoxie, we think of mobile first indexing as entity first indexing and that means fitting in content from the web around the knowledge graph. And that’s important, because it allows Google to do their machine learning faster. And it also allows Google to be better in languages where they have less machine learning, because entities are language agnostic. So, does that make sense to you or do you want me to explain it?

Ben:                             I guess I’m a little fuzzy on what you mean by entity first and the difference between what you’re calling an entity and how that relates to the knowledge graph and then the difference between that and how it affects mobile. So, yes, I’m a little confused.

Cindy Krum:                 Sure. Okay. So, when we talk about entity first indexing, it’s basically fitting things into existing or new entities in the knowledge graph. So, if you think about the knowledge graph as a graph, as a series of things that are related to other things in kind of a fabric, right? This … And the relationships can be lateral, right? Taco is related to hamburger, but then taco is also related to lettuce and tomato or it’s related to food, right? These are multidimensional relationships and those are things that Google knows and that Google can graph, right? They can meet these relationships and those relationships stay static regardless of what language or keyword you’re searching in. So, taco is always … Has the same relationship to lettuce and to food in every language. Does that make sense?

Ben:                             Yes.

Cindy Krum:                 And so those relationships can be put into the knowledge graph or it already exist in the knowledge graph, potentially.

Ben:                             I think I might understand tacos better than I understand mobile marketing.

Cindy Krum:                 Yeah.

Ben:                             Sorry, bad pun. Go on.

Cindy Krum:                 So, when you put things in the knowledge graph, in this entity relationship, then you take out the reliance on language as the primary modifier and the entity becomes the primary point of indexing, right? So, let’s get away from taco, ’cause that might be confusing. Let’s do mother. Mother has the same relationship to daughter, son, father, grandfather, grandmother in every language, right? Even though you use different words to describe it. So, if we start taking in machine learning data on the entity idea of mother rather than the keyword mother, then we build our machine learning and our relationships much faster, because we’re not waiting for low volume languages to catch up with high volume languages. So, Google’s always been really great in English, ’cause it was built in English and because there’s a lot of machine learning feedback and a lot of queries in English. So, they can hit statistical significance really fast, but in smaller languages they’ve struggled, which has made fighting spam harder and stuff like that. And it just means that they have a worse experience in the smaller languages, but when you back it up and you say, okay. Stop doing this based on one language at a time, but do it based on entities and an entity is language agnostic, the relationship is true in every language, then they can learn and build their algorithm faster in all languages. Does that make sense?

Ben:                             I understand. I guess I always assumed that the mobile crawl first was Google getting to the point where they’re saying, hey, there are more queries that are happening from mobile devices, so we need to prioritize mobile optimized search first before we get to the desktop crawls. It sounds like I am way off.

Cindy Krum:                 Well, I mean that’s the way Google’s been messaging it, but that doesn’t tell us anything about why did they need new indexing on that? That’s mobile crawling maybe, but did they need … Why did it take them two years to figure that out? I think there’s just more to the story.

Ben:                             Okay. So, there’s … It sounds like there’s a story behind mobile indexing, which is Google is crawling mobile first, because the way that they’re indexing has changed. So, the information that they’re gathering is more relationship based and less driven by a specific language.

Cindy Krum:                 Right, right. And that’s because, which we alluded to in the last episode, that’s because the relationships are critical when you don’t have a screen to help sort out or to help make it easy for someone to pick what the right answer is. Google has to get the answer right on the first try and they have to be able to drill down in a relational way when there’s voice only, right? You don’t get 10 options. In voice, they have to get just the one.

Ben:                             Right. Okay. So, part of this is not only driven through the mobile device becoming the primary mode of content consumption. It also has to do with Google trying to get better at language processing with an eye on the future in terms of voice search.

Cindy Krum:                 Right. I think that they want to use the same index for voice and mobile phone, right? They wanna be able to leverage what they’ve learned to provide a great voice only result as well.

Ben:                             Okay. So, talk to me about the impact this is having, the mobile first crawl and Google’s change in how they’re indexing or how they’re approaching indexing, in terms of business results. Are there specific people that are taking advantage of mobile crawls being prioritized and what’s the results that you’ve seen driven?

Cindy Krum:                 Yeah. So, what’s happening right now is for a long time we had the increase of knowledge graph showing up and pushing everyone down. And then we had the increase of featured snippets and now we have double featured snippets and hybrid featured snippets and all of these things. And what happens there is that if Google hosts it, they trust it, so they’ll likely rank it better. So that … We talked about that last episode with, but also featured snippets they host or cash the answer and the images that go with it, and now they’re serving those in the people also ask dropdown, right? They’re turning Google more into an answer engine than a search engine. People aren’t searching for websites, they’re searching for answers. The answers just used to always be on websites. Or that was the only thing that Google could get. But now they’re basically lifting and caching all of the answers that they can find, whether it be text, video, audio.

Cindy Krum:                 So, having a business that does more than just text, we call it being a multifaceted business. So, having video, having audio, doing podcasts, that’s all great, because that’s harder to gain in terms of signals of trust to Google. For a long time it’s been about links, but people that like crazy. They’ve said they wanna get away from giving links as much weight as they always have and one way to do that would be to value high engagements, high quality multimedia content, like video and audio. And so that’s where I think the businesses need to be focusing to really do a great job. And then also GMB, Google My Business, because that’s a direct feed and Google hosts it. Anything that Google hosts, they’re gonna rank.

Ben:                             Yeah. I guess the question that I have is what does that … I understand that the rich media content is impactful. How is that related to the shift towards mobile first crawling? Like why is … Sure. Rich snippets and featured snippets and all of the answer type content that Google can extract no matter what medium it is, video, voice, text, all that is being prioritized, but Google shifted to mobile first crawling. Why are the two related?

Cindy Krum:                 I think it’s … They’re almost not related. I mean Google has said mobile first crawling or mobile first indexing and mobile first crawling are the same thing. In my mind, whether they’re crawling with a desktop user agent or a mobile user agent, it doesn’t matter as much as what they’re prioritizing in a search range, right? Like we only care about what user agent they’re crawling with, because we want stuff to rank. And the stuff that’s ranking since mobile first indexing launched is answer based content and the number of featured snippets, like according to STAT, number of featured snippets has gone to 6X up, six times as many featured snippets. To rank in Google, it’s now about getting the knowledge graph featured snippets being found on the web. It’s about being in a hosted inclusion, because those are getting images and those are where the engagement is in a mobile search result. Blue links are not getting the engagement.

Ben:                             So, Cindy I wanna double back a little, ’cause I guess there’s a little confusion in my head where we’re talking about the format of content and I totally hear and agree with you that building content that fits these knowledge graph featured snippet placements are important. That seems to be something that matters for desktop and for mobile. When we think about the mobile crawl, what is the impact of mobile crawl and how is it related to that featured snippet sort of change in prioritization of content?

Cindy Krum:                 Sure. So, what Google’s documentation focuses on for the mobile crawl or mobile first indexing is that they say you need to make sure that your content matches between the desktop and the mobile rendering. And that includes links, internal and external, or it includes just the content, it includes Schema, it includes. So, that’s the main thing. And if your mobile rendering of your site doesn’t include that stuff, then Google says you might not rank as well. But they also still maintain that they have desktop only rendering. Again, I think their messaging there is a bit confusing, but you should make sure that you know exactly what is being sent by the mobile version of your site. So, what … And especially Google didn’t focus on this, but make sure all your canonicals are right and if you’re doing separate mobile pages in a mobile sub domain or mobile subdirectory, then you have a bigger concern.

Cindy Krum:                 But Google says if you’re doing responsive design, you really don’t have to worry about the mobile first crawl, but there are … The only other impacts or thing to think about here is deferred JavaScript rendering, which was part of mobile first indexing where they said, Google’s not gonna try and execute your JavaScript on the first crawl anymore. They’re gonna cue it up in a separate process and then it could take up to a week for your JavaScript content to be crawled. And that changes some of their previous recommendations, because they used to, for instance, recommend inlining some of your critical JavaScript in the head tag. Now, Google said no, no, no, don’t do that anymore, ’cause it means your whole head tag might be delayed in deferred JavaScript rendering. So, anything that’s really critical still, you need to keep it out of JavaScript and some of the SEOs that focused on JavaScript, like and have done more studies and shown that deferred JavaScript rendering still is not perfect.

Cindy Krum:                 Still JavaScript can hurt you. Yeah. So, that’s the main thing. The other thing that I think was made possible for indexing that’s … It’s not mobile first crawling, but it might be considered mobile first indexing, is Google quietly launched an indexing API, where you can just send them a list of pages that you want in the index or send them a list of pages that you want removed from the index. That’s a big freaking deal. It kind of reminds me of 1995 when we were submitting URLs to search engines directly, but this is what I was talking about where like I think that we’re just on the brink of a whole new way that searches are going to look and that SEO is gonna be done.

Ben:                             I think I get it now where we’re talking about Google’s documentation where they are essentially saying make sure that your desktop and your mobile content matches up the links, the content, the form and really we’re gonna crawl mobile first and we’re gonna look at that experience and we’re gonna make sure that it matches against desktop and if there isn’t a great match then there is potentially a penalty in terms of rankings. Then you get in all this really confusing stuff about like, well, how do we handle JavaScript and there’s all these other ways that the mobile and the desktop experiences aren’t alike, but they’re trying to get it to be like, we’re gonna look at mobile first and I am simplifying this as much as I can to the length of my understanding. We’re gonna look at mobile first. We want mobile to be the same thing as desktop, so we can rank your sites accordingly based on the experience, but we don’t really wanna have to do a deep crawl of both environments.

Cindy Krum:                 Yeah.

Ben:                             Okay.

Cindy Krum:                 Yeah. But, I’m gonna warn you, the Internet and especially Google hates the word penalty, so it’s more like it’s a missed opportunity to rank, because you’re not sending all the right signals. That’s what we’re supposed to say instead of penalty.

Ben:                             Right. Not a penalty. It’s just a way that they’re gonna punish you. I mean, something different. Totally different. Anyway.

Cindy Krum:                 Right.

Ben:                             All right. Well, lots to get into. I think that my takeaway here with mobile first indexing is your mobile is supposed to match your desktop in terms of content and some of the external signals and it doesn’t necessarily have a huge business impact. Really the bigger thing that’s gonna affect your business is the content that you have in both experiences and trying to get knowledge graph featured snippet type content, ’cause that’s the direction that Google’s going.

Cindy Krum:                 Yeah, and let me just kind of close the loop on the entity first indexing here, because I think it’s important to look back on, for instance, John Mueller’s AMA, that was right around the time that mobile first indexing really launched. If you look at the questions that he chose to answer and the feedback that he gave, so much of it was about HRefLang. Like, he avoided a lot of the mobile first indexing questions or topics or just really doubled down on HRefLang, and this is one of the many signals that made us look at why is he focusing or why is the messaging focused so much on language. And this is where we kind of settled on this entity first indexing idea and we’ve actually proved it out in some studies where knowledge graph is treated differently from a language perspective than a regular search result.

Cindy Krum:                 So, if you do a search in Greek, but your search settings or your phone settings are set to Japanese, the blue links and the regular old school organic results will be in Greek, but since your phone settings or your search settings are set to Japanese, knowledge graph comes back in Japanese.

Ben:                             Fascinating.

Cindy Krum:                 So, what we’re saying is that knowledge graph is trying harder to suit your specified preferences and that is important for voice search, but the more you can clearly express things about language, things about classification like Schema and where you would fit in the knowledge path, the easier time Google is gonna have in mobile first crawling and indexing, getting your stuff in the right spot in the knowledge graph.

Ben:                             Interesting. Google’s taking a different approach to how they’re indexing and also what they’re indexing. So, I’m sure this will be an iterative process over the future. And I think we’re gonna dive into a couple more ways that Google is not only looking at your content, but how they’re getting it to the consumers faster in our next episode. So, that wraps up this episode of the Voices of Search podcast. Thanks for listening to my conversation with Cindy Krum, the founder and CEO of MobileMoxie. We’d love to continue this conversation with you, so if you’re interested in contacting Cindy, you can find a link to her LinkedIn profile in our show notes. You can contact her on Twitter where her handle is Suzzicks. Or you can visit her company’s website, which is MobileMoxie.com, And if you’d like a free month of their service, you can use the promo code search metrics in all caps.

Ben:                             If you have any general marketing questions or if you wanna talk about this podcast, you can find my contact information in our show notes or you can send me a tweet at BenJShap. If you’re interested in learning more about how to use search data to boost your organic traffic, online visibility or to gain competitive insights, head over to searchmetrics.com/diagnostic for your complimentary advisory session with our digital strategies team. And if you liked this podcast and you want a regular stream of SEO and content marketing insights in your podcast feed, hit the subscribe button in your podcast app and we’ll be back into your feed to discuss Cindy’s tips for optimizing your mobile sites to use AMP. Lastly, if you’ve enjoyed this podcast and you’re feeling generous, we’d love for you to leave a review in the apple iTunes store or wherever you listen to your podcasts. Okay. That’s it for today, but remember, until next time, the answers are always in the data.