Episode Overview: JavaScript is vital for many companies and ecommerce sites as they utilize it to generate links to products and content. Yet many JavaScript websites perform poorly in Google as they’re not optimized for how Google indexes. Join host Ben as he speaks with Searchmetrics’ Director of Services Björn Darko about how important JavaScript SEO is to digital strategies and how to optimize for it correctly.
Summary
- Google uses second wave indexing for JavaScript content.
- The second index wave is when it renders your site in full, and can be harmful as it results in lost traffic.
- One way to bypass the issue is to apply static rendering to a pre-rendered version of your URL and store it in the cache to aid Google’s indexing.
GUESTS & RESOURCES
- Björn Darko: Website // LinkedIn
- The Voices of Search Podcast: Email // LinkedIn // Twitter
- Benjamin Shapiro: Website // LinkedIn // Twitter
Ben: Welcome to the Voices of Search podcast. I’m your host, Benjamin Shapiro. And today we’ll be looking under the hood to provide you with the technical tips you need to make the most out of your SEO strategies. Joining us today is Björn Darko. Björn leads the European arm of Searchmetrics’ digital strategies group, which offers SEO consulting, content marketing services, and strategic consulting to software companies all around Europe, the Middle East and Africa. He’s also the creator of the popular IGTV show, SEO presso, and today Björn and I are going to talk about why JavaScript SEO is so important.
Ben: All right on with the show. Here’s my conversation with Björn Darko, director of the digital strategies group at Searchmetrics. Björn welcome to the Voices of Search podcast.
Björn: Thanks for having me again.
Ben: Always excited to talk to you, always excited to talk a little technical SEO. Let’s be honest. You’re teaching me here. Today we’re going to talk a little bit about JavaScript and why it’s important. JavaScript SEO. First off, let’s talk about what JavaScript is. This is coffee writing on the walls. Write JavaScript. No?
Björn: Actually, actually it’s a developers’ language, which is created after HTML. It’s actually the code language, which developers use to code stuff and code functions and code websites.
Ben: So my very layman’s understanding of JavaScript. You said, “Hey, this is a code. HTML is easily readable and crawl able for SEO. Google loves HTML.” They eat that stuff up, but JavaScript is a little bit of a black box where Google can crawl some things and can’t crawl the others. Why would anybody bother to use JavaScript in the first place?
Björn: JavaScript is probably, I would say one of the hot topics in the SEO industry and you’re right. It’s different than for Google. It’s different than just crawling pure HTML. It’s more complex, I would say. Right? So, I mean, we know JavaScript as important functions. There are a lot of important functions which are based on JavaScript. A video player or it’s a navigation or something which pops up and makes shiny things on a website. It’s mostly based on JavaScript. But since I’d say, I don’t know, five, six years, there are also complete websites built on JavaScript based web applications or JavaScript frameworks. And most of them Angularis or React JS. So React JS. So library from Facebook and Angularis from Google, and what they do is actually they create a so-called single page application, and this is different from normal the websites where you have different HTML fives and they are linked to each other, so different HTML documents and they are linked to each other and already serve full source code to a browser or to a client like Google on a single page application. You just have one single HTML document and the content gets dynamically served to a client or to a browser or to Google.
Ben: The way that I think about the use of JavaScript, it’s all the bells and whistles that you want in your website. All of the, the motion, the triggers, a lot of the tracking is also done in JavaScript. It’s anything that’s not really just the flat base website. I’m sure that there are some shades of gray there where some HTML can be a little bit more dynamic, but really what people are using JavaScript for the most part is all the bells and whistles, all the dynamic content, all the popups and the flashes and the widgets and all that stuff. Am I right?
Björn: Yeah, exactly.
Ben: Alright. So talk to me about what Google can and can’t crawl when it comes to JavaScript. Why is it controversial in the SEO space?
Björn: Well, to explain that we have to actually dig into how Google works to fully understand why it’s so complicated, right? So how Google actually works when it comes to crawling and indexing, it’s a process for Google. And this process includes three steps and it’s crawling, it’s indexing and the Kerry engine would spit out two results in itself. So what is a crawler? A crawler crawls the web, discovers URLs, passes safe, GMR source code extract, all the links and CSS and sends those to the indexer. What does the indexer? The index, and then tries to make sense out of the link and analyzes the content and relevancy and use these four called web rendering services where they actually render the full page in order to check how it actually looks like how the appearance of the website is. I don’t actually execute JavaScript as well, similar to what you may know from fetch and Render in the Google search console.
Björn: And this is like how Google actually works. And when it comes to JavaScript crawling, and we can actually say, that’s the same process, it crawls the web, it downloads the JavaScript, but here’s the problem. So it sends then everything to the index and the index I need. So renders the page and the index that needs to render the page and all that to get the links. So the links with JavaScript, they just get the links after where the rendering there’s different form HTML, where they get the links beforehand. So here you have to actually make sure that the relationship between crawling and indexing is very good, because if it’s not, then it’s too complicated for Google to do the whole process. And marching split is a famous Google employee who teaches a lot about JavaScript. SEO just announced in the last Google developers conference that actually Google uses so-called second wave indexing for JavaScript content, because it’s so complex, Google actually needs to be a bit more smarter here, and the way how they do it is so called second wave indexing.
Ben: So, what is second way of indexing?
Björn: So second indexing actually means that Google is crawling all those JavaScript content and you almost book fire and just throws it into the index without rendering it in the first instance. What they do is they render it in the second, when they visit the site the second time and between the first time, and the second time there might be weeks in between. And this is the dangerous thing here, because if you rely on search engine rankings, and it takes a lot of time for Google actually to render your page and then fully understand your website’s content about this and therefore index it correctly and ranks it correctly.
Ben: So it seems like that’s really one of the biggest problems with JavaScript is that the results that you’re getting for rankings are going to be delayed because Google actually has to go through this second rendering process. If I’m reading into what you’re saying is that, the use of JavaScript because a lot of the content that’s going to be displayed through JavaScript is dynamic ends up being so complicated for Google that they’re just saying, Hey, we’ll get to this part later. We’re going to unpack this box, figure out what’s in it. It’s going to take us a little time. Let’s just go through the HTML first at first pass.
Björn: Exactly.
Ben: So talk to me about some of the tips for optimizing your JavaScript for SEO.
Björn: So there is actually a must have, and this is called server side rendering. So if you are based on a complete or full JavaScript based web application, then you need to make sure that you have server side rendering in place. So this actually means that you provide an already, fully rendered HTML version of your website to Google.
Ben: Okay? So you have the ability to say, “Hey, Google, I’ve already unpacked this box for you. Here’s what it looks like when it’s unpacked that way you don’t have to double back and do this second crawl experience.”
Björn: Exactly. And there are different ways you can do the rendering. So there’s one thing called static rendering. And what you do here is like you have a pre-rendered version of your URL and store it in the cache so that the server is sending this already pre-rendered file to Google, whenever Google comes. And there’s another thing which is called dynamic rendering, which actually allows requests based on the navigation. There’s a navigation point coming to this type. Then the output will be HTML. That’s already doing the job for Google on the server and sending another pre-rendered version here to Google.
Ben: It seems like there’d be some risk here for Google where … Let’s use the security through the airport example. If I say, “Hey, I’ve unloaded my suitcase. And I’m just going to show you a picture of everything that’s inside. Okay, great. Here’s everything that’s in my suitcase.” You should let me go through security and down the road, you’re actually going to get a chance to check my suitcase. You don’t know what’s actually in the suitcase when it goes past. So how does that not … Using this metaphor, how does it not create a security risk for Google, where people are saying everything is legit in my JavaScript, go ahead and rank my page appropriately. And then they actually have a different experience when it comes time to load the page.
Björn: Yes. In the earlier days we would say this actually cloaking, right? So because what you do is you are providing Google something which is different from the file character, then a normal user, the browser, because the normal user who comes through the browser gets the hours for the base website. Right. And you’re completely right. There are a lot of pitfalls, which we also have seen with clients, and this is what people actually see.
Björn: So let’s take one example. We have a publisher, they are running on Angular JS, and they’re using Brenda Tron as a rendering machine who does the job Google. But what they do is static rendering, meaning that they make a snapshot of the website site and the cache and whenever Google comes they get the version of the cache. The thing is that they didn’t renew the version whenever there was a new your article published, for example. So what, who the Goggle got was a version of the website, which was maybe a month old. Whenever there was a new article published Google didn’t get this article. This is one thing. So you always need to make sure that the start version of the website gets updated as soon as there’s new content available. Right.
Ben: Understood. Go on.
Björn: Understood. And another thing which we saw is paginated pages. So you have a pagination on the website, 1, 2, 3, 4, 5, 6, right?
Ben: Yep.
Björn: And if you have a pagination on the website, the content on page 1 should be different than on page 2, 3 or 4 or 5 or 6, But because these pre-caching of the website’s content costs so many resources, this publisher, just did one cache version of page one, and then mirrored it up to page 10. So that you had exactly the same content on 10 pages, which is of course causing duplicate content for example, and is not really good also that triggered a lot of duplicate content as well with some parameter URLs, because what you can do an audit to see what it actually indicates, that you can use a parameter, which actually then shows you the cache version. And what they did is they made it fully accessible even with a non-canonical tech and just throw it into the index so that you actually had two different or sometimes two or three different sites’ content on three different URLs.
Ben: So at the day it seems like crawling JavaScript is a huge pain in the butt for Google. Are they punishing companies that are using JavaScript because it’s difficult for them? How does Google think about SEO? How does Google think about crawling JavaScript and what the impact is on SEO?
Björn: Well they say they fully understand JavaScript and they can do it. Although they recommend that you use methods like server side rendering, because they know it’s very complicated for them. And especially we talked about it in a previous episode, especially if your page feed is very, very weak. This also causes Google to pause a bit here and says, well, it takes way too long for me to get the fire and to render it, I just leave it and so you have to make sure that it’s very, as quickly accessible for Google and that Google actually can quickly grab your content and index it. And so, and this doesn’t work with JavaScript so you always have to make sure that you have a pre-rendered HTML version already sought on your server and then provided to Google, which is of course, always up to date as well.
Ben: Last question I have for you, when you think about optimizing your JavaScript for SEO, what are the tips that you have to make sure that you’re taking as much burden off of Google as you can? So they’re starting to prioritize your pages.
Björn: Yeah. So as I said, the first thing I would do is actually think about server side rendering. However, what you’re already should do is like, normally if you have your JavaScript website, it’s just the JavaScript code, right? What you definitely should do is put already all important meta information like meta title, meta descriptions and tags. Everything which is important for Google to see in the first place, put it already on the not rendered version of the website. Same goes with structured data. So everything was just actually very important for Google to understand, and which is very important for Google to evaluate what pages about, put it already in the, not yet rendered version of the website. This is actually something where you can help to take off some burn.
Ben: We talked a little bit about the Google Lighthouse study. The last time we got together and a big part of that conversation was understanding the prioritization for putting your website together. Mostly when it comes to performance and accessibility, making sure that Google has access to the right information and that you’re able to get it quickly. And this seems like another prioritization exercise as well, making sure that you’re getting Google all of the information that they need getting as much out upfront before you’re asking their search engine, their bots to go through and crawl, try to understand and interpret what your JavaScript is actually doing.
Björn: And if you want to audit your site, if you know that you’re JavaScript, you want to audit the site by itself, you can use the crawler with JavaScript crawl function. I know that’s such a metric system in this as well, so you can use such tricks, but you can also use the so called view source plugin. And what it does is it compares the raw source code with rendered source code. And then you can already see if there are any differences. You can also use touch and renders, any bots tools just Google it, which, you know, renders your website and just gives you a snapshot as Google would see it. A quick way to see If JavaScript is a problem on your website is if you disable JavaScript in your browser, there is a plugin for this, and you also need a plugin where you can actually pretend to be a Google bot.
Björn: And what you do is to JavaScript and you come as a Google bot and then you see exactly what Google is actually seeing. And if you fully rely on the JavaScript and you’ll have no server side rendering in place. What you see is actually a blank page or a loading symbol and nothing actually loads,
Ben: Lots of different tools and tips for how you can optimize your JavaScript for SEO. Björn, thanks again for walking us through some of your technical tips.
Björn: You are Welcome. Thank you.
Ben: All right. And that wraps up this episode of the Voices of Search Podcast. Thanks for listening to my conversation with Björn Darko, director of the digital strategies group at Searchmetrics. We’d love to continue the conversation with you. So if you’re interested in contacting Björn, you can find the link to his LinkedIn profile in our show notes, you can contact him on Twitter. His handle is Björn underscore Björn that’s bjorn_darko. You could watch his famous IGTV show it’s SEOpresso or you could visit his company’s website, which is searchmetrics.com.
Ben: Just one more link in our show notes. I’d like to tell you about, if you didn’t have a chance to take notes while you were listening to this podcast, head over to voicesofsearch.com, where we have summaries of all of our episodes, contact information for our guests. You can send us your topic, suggestions, your SEO questions. You can even apply to be a guest speaker on the Voices of Search podcast. Of course, you can always reach out on social media. Our handle is voicesofsearch on Twitter. and my personal handle is BenJShap, B E N J S H A P. And if you haven’t subscribed yet, and you want a daily stream of SEO and content marketing insights in your podcast feed, we’re going to publish episodes every day during the workweek. So hit that subscribe button in your podcast app, and we’ll be back in your feed tomorrow morning. All right. That’s it for today. But until next time, remember the answers are always in the data.