searchmetrics email facebook github gplus instagram linkedin phone rss twitter whatsapp youtube arrow-right chevron-up chevron-down chevron-left chevron-right clock close menu search
1313313133

Google’s rise to the top of search and how they ate their competitors’ lunch

iTunes       Overcast      Spotify      Stitcher

Episode Overview

Kicking off Searchmetrics’ Non-Google Search Month, SEO strategist Jordan Koene and Ben Shapiro launch their series covering the history, current state, and optimization strategies of Google’s biggest competitors. In this first episode, Jordan and Ben delve into the history of search and cover why Google’s biggest competitors have risen and then fallen.

Learn about:

  • Gerard Salton, one of the founders of search, and how he set the stage with his research on indexing
  • How Yahoo developed an early lead and why Google eventually gained market share
  • How Google’s core strategy of answering questions better and faster lead to their success
  • How Google’s acquisition of DoubleClick in 2007 became their turning point in monetizing search

GUESTS & RESOURCES:

Episode Transcript

Ben:                 Welcome to Non-Google Search Month on the Voices of Search podcast. I’m your host, Benjamin Shapiro, and this week, we’re going to turn the spotlight onto how you can optimize your SEO efforts for some of the most important search engines that don’t start with the letter G. And this week, we’re kicking off Non-Google Search Month by publishing an episode every day discussing the history, status, and optimization strategies of Google’s biggest competitors.

Ben:               Joining us for Non-Google Search Week is Jordan Koene who is a world-renowned SEO strategist and the CEO of Searchmetrics. Today, Jordan and I are going to talk about the history of search and how Google’s biggest competitors have risen and have fallen.

Ben:               But before we get started, I want to remind you that this podcast is brought to you by the marketing team at Searchmetrics. We are an SEO and content marketing platform that helps enterprise scale businesses, monitor their online presence, and make data driven decisions.

Ben:              To support you, our loyal podcast listeners, we’re offering a complimentary digital diagnostic where a member of our digital strategies group will provide you with a consultation that reviews how your website, content, and SEO strategies can all be optimized. To schedule your free digital diagnostic, go to Searchmetrics.com/diagnostic.

Ben:               Okay. On with the show, here’s my conversation with Jordan Koene, the lead SEO strategist and CEO of Searchmetrics.

Ben:               Jordan, welcome back to the Voices of Search podcast.

Jordan:               Hey, Ben. Surprisingly, although we’re talking about non-traditional or other forms of search, think the weird part is that we might actually spill the G-word many times during these various podcasts.

Ben:               I think that’s part of the takeaway and one of the reason why we’re excited about putting this together is specifically in today’s episode, we’re going to talk about the history of search and kind of the fall of Google’s competitors and its rise. And obviously no matter what search engine you’re looking at, Google is always going to be the 800 pound gorilla in the room. So let’s talk a little bit about how that happened. Can we go in the way back machine here and talk about the start of search, pre-Google, and what happened that led to their prominence?

Jordan:               Yeah. Absolutely, Ben. So let’s dive in. I think one of the places we need to first start off with one of the prominent figures and really one of the prominent researchers and scientists, really, truly, genuinely a scientist behind search. Without much of his work, we really wouldn’t have the backbone behind the series and the principles of how search works. It’s kind of funny, we actually know a lot about this here at Searchmetrics because many of the concepts that this individual wrote about really stand true to how we look at even our business, how we collect data. So Gerard Salton, he is really one of the core founders of search. He really wrote a book about the series of indexing that set the stage around how we should be thinking about search and basically this documentation collection process that then allows you to then serve results. And he created this first little concept, prototype, which was called the magic automatic retriever of text. And this kind of set the stage behind many of the ideas and principles that we now experience every day when we do a text-based search.

Ben:               So talk to me about some of the work that he did and what were some of the players that came out of Gerard’s science?

Jordan:              Yeah. So one of the interesting things is that much of his work, whether it’s his books or his other experiments, use theories and tools that we now talk about every day in SEO and in search. Things like TFIDF or vector models or other term and keyword relevancy. He also kind of talked a lot about … He didn’t invent all of these things, but he used them and formed them together to help people understand how to create a full search experience. And so a lot of us think, “Oh, man. These things are pretty new.” They’re really not. I mean, a lot of these theories and principles have existed since the ’50s and ’60s. And then they were just adopted for a digital space in the early and late ’90s. That’s kind of where really search starts to have its first true impact in every-day life, which is in the early to mid ’90s, what we see is properties like Excite, Yahoo Directory, which most people know, and probably the most notable, for those folks who have been around a long time, they’re going to remember this one, DMOZ, so the open directory project. Really kind of the backbone of how you started much of your search engine optimization journey because you needed to be in DMOZ in order to really get recognized in Google and Yahoo and other search engines.

Ben:             So I understand that Gerard is essentially the godfather of search, and he created all of this technology that led to some of the early projects that led to things like Yahoo’s database and to web crawling. Talk to me about how that evolution happened, and let’s get into talking about who some of the competitors were, like the rise of Yahoo as a search engine.

Jordan:               Yeah. So, the rise of a lot of these search engines were kind of focused on a few main themes. Crawling was one of them, natural language was the second one, which was largely one of the biggest pioneers. There was Altavista, Ask Jeeves, and then the final one was kind of like the document retrieval. So how fast can you serve and produce documents, and that’s really where Google starts to play a major role. And ultimately, a lot of those names probably sound super familiar to folks, but ultimately what we see is that there’s a formation of kind of a two-prong search, and we’re fast forwarding quite quickly here. A two-prong search winners, which are Yahoo and Google, and that competitive environment really is what kind of builds and thrives in this industry is that there’s two main head-to-head competitors that are jocking for users, are setting the stage for new innovations around experience, new innovations around speed, and ultimately, especially as we get into the early to mid 2000s, how they make money. So how do you create different paid per click experiences.

Jordan:              And oddly enough, in this particular space, Yahoo had an upper hand early on as Google continued to really force this theory around, “Hey, we just need to provide a free listing and a free experience,” and so really Yahoo kind of had the first upper hand there and started to also acquire companies that supported that mission of ads.

Ben:               So Yahoo was essentially the first modern search engine. While there was Altavista and some of the other internet days, the 1.0 search engines, Yahoo is really the one that sort became an aggregator and had an early lead in terms of gaining some market share in search. What happened to Yahoo, and at what point did Google start to basically eat their lunch?

Jordan:               I’d really like to say that there’s a critical theory here around, really around performance and relevancy. So as Google evolved their performance over time and made that a very clear mission of the company, which is, “Hey, we’re going to give you choices faster than anyone else in the industry, and not only that, we’re also going to give you the most relevant, the most useful results.” That’s what set the stage for this kind of differentiation between Yahoo and Google because ultimately the greatest value add for searchers was how they consumed and utilized the various results as opposed to formulations around how well you can monetize to various ads and ad options on your pages. And I think that’s where the differentiation existed between Yahoo and Google in kind of like the mid to later 2000s.

Ben:               I think there’s a couple different components to this. First off, when Yahoo was running their search engine, there was always a component of what was paid to play, and as much as you’re saying relevance is important, Google focused on relevance. The results Google presented just seemed to be more relevant, right? They seemed to answer questions better and faster. But there’s also a distribution component to this where Yahoo was very much focused on distributing things like toolbars and getting their search out into a browser integration and Google was not necessarily just a search company but also was able to figure out a distribution model. So talk to me a little bit about how the combination of the user experience, the distribution model, and what Google is just general strategy was that allowed them to separate and distance themselves from Yahoo, and was it purely just Google presented better results and the better product experience one or is there a distribution component as well?

Jordan:              So let’s break those up, right? So the first one here around kind of the overall experience and performance, absolutely. Google distinguished themselves there largely because Google was not also trying to be a multi-faceted company in the early 2000s. So as Yahoo became more and more and more of a media company and they also started to really collect a lot of assets where they could distribute display and other networks of advertising. Yahoo became much more of an advertising player while Google continued to heavily push on their technology and technology evolution of search. And those performance investments paid off in the long run. Not just speed but also quality and relevancy ended up being the number one distinction. All of you who are listening experienced those things, especially if you’re using search in the early 2000s. You could definitely tell the difference of quality between Google results and Yahoo results, and that just continued to kind of grow and scale.

Jordan:              Now the one interesting thing here to your distribution point then is this is where actually Yahoo had the upper hand because if you look at Yahoo around 2005, maybe even until 2010, Yahoo had a much larger network. They could distribute ads and collect money and revenue through a multitude of channels, and the biggest crusher of Yahoo, what really not only in search but in overall business performance, the distinguishing tipping point was in 2007 when Google acquired Double Click. Because Double Click then allowed Google to play in the same space and compete against Yahoo, and really control not just search but all of these different advertising channels. Because at that point, then it became a foot race to who can make more money through these new advertising channels, these new digital advertising channels.

Ben:                So what I’m hearing you say is essentially in specifically search, the user experience was so much better for Google, right? They figured out relevance that people were flocking to use it as a search engine away from Yahoo, even though Yahoo had these installed user bases and had distribution channels like their toolbar. It was the user experience. It was the Google algorithm that allowed them to develop and become not only competitive but the market leader in search, and then Google was able to acquire Double Click and then got into the advertising game, which allowed them to monetize search as well or better than Yahoo was. And that’s really where the business of Google started to take off.

Jordan:              Right. There’s a million things, right? We can’t pack all of these different things that occurred between Google and Yahoo, and Google’s evolution, and what happened at Google over those years. I mean, we only have like 15 minutes. So we can’t go over all of those things. But I would also add in that one of the distinguishing I think capabilities that set Google apart from Yahoo in these very competitive search wars was how well integrated Google became when they offered new products to the market. And it may not seem like they were super integrated at first, but there was generally speaking, a technology stack that allowed many of the new products that they offered in the space to leverage preexisting technology. Email, maps. Even when they started to venture out into different verticalized searches, like book search, news. All of these different experiences or different product offers, like email and others, literally leveraged the innovations that they created in the early years around performance and around speed and around accessibility and around design that allowed them to grow and scale these new products.

Jordan:              So I think that’s where Yahoo struggled was, “How do we make all these things work with one another to create an amplification to our value?”

Ben:                I’ll be honest, from an end user experience, I think that Google simplified where you would go to Yahoo search back in the day, late 1990s, 2000s, and it was a cluttered experience. And search was not the sole focus. You went to Google.com and it was a logo and a search box, and then when you went through it, it came up with search results. And there wasn’t a lot of advertising. To me, it was such a simplified user experience relative to Yahoo that it allowed Google to gain this foothold in search, right? It allowed them to start collecting the data and collecting all of the information they needed to be able to then port their search technology into, “Hey, we made it really easy for you to search through your inbox.” “Hey, we made it really easy for you to search through your chat history.” So they started taking that core technology, all of the data they had, and they ported it into these other products. Maps is another example.

Ben:               And so now Google has not only simplified user experience with search, collecting a lot of data, they built in this monetization mechanism where they bought Double Click to basically produce an ad server and start creating the Google Display Network. But they’re able to add utility, and instead of just sitting back and saying, “We’re going to be a search company,” they started making all of these auxiliary tools, which now we see are incredibly valuable and allow them to collect more data and then optimize the search experience further.

Ben:              I think where Yahoo fell short was the technological innovation, and they did not simplify the user experience. Really what they did was they just started prioritizing the business and trying to take advantage of their installed base instead of continually iterating and optimizing and innovating. They got slow. They got big, and they got tired. And then they got passed by Google.

Ben:              So, Jordan, I guess my point when we talk about the difference between Yahoo and Google and how, for lack of a better term, Google innovated and boat raced passed Yahoo. At some point here there was the rise of other competitors. Yahoo is no longer the primary competitor to Google, if there is one. There is the Bing’s of the world. There are localized search engines that are sort of geographic or regionally specific, and there are verticalized search engines as well. We’re going to talk about some of those other things. But talk to me a little more about how you view the landscape today, and who is the primary competition now that it is clear that Google passed the original modern search engine in Yahoo and became the dominant player? Who else is out there that we need to consider?

Jordan:              Yeah. So I guess where we ended off here is this two-headed monster, which is Yahoo and Google. That evolved into partnership of the technology that was being served by Microsoft and Bing, and really where we are today, especially right now here in the U.S. market, is we have-for lack of better terminology-a monopoly with Google owning the majority of non-specific experienced search. So what I mean by that is if you’re just going to do any old search, you’re typically going to start with Google and then we have Bing as a secondary player there. And then there’s a lot of other tertiary players that exist. DuckDuckGo, and there’s a variety of other tools out there that also do non-specific verticalized search. So beyond that pool, then we also have the verticalized search experiences. You got eCommerce, video, local search, and those players. And then finally, when you look outside the U.S. market, there’s a variety of different players based on different regulatory restrictions or language restrictions that exist. So we’ve got players in South Korea, China, Eastern Europe, and Russia. So there’s these different use cases in different markets that exist, and that’s kind of where we are today in terms of the overall search landscape.

Jordan:              But which by no means is a simplification of where we are today because it is hyper complex. But that’s kind of a package summary of what we have right now.

Ben:                So as you think about the search landscape, let’s just talk the domestic search here in the United States. What percentage of the search market does Google own, and as search specific, non-verticalized search?

Jordan:              For the U.S. market, I mean, they’re the market leader. There’s a variety of different studies out there that show anywhere between 80% to the mid-90% market share for Google. I think it’s pretty safe to assume that when we look at a searchers behavior, especially in the U.S. market where many of the English speaking markets out there, so for our listeners, I would assume that’s many of you. It’s really Google that dominates and controls the mind share. Even users who are using say Bing or some other secondary search engine on a nationalize and a generalized type search experience, those other search engines are often being supplemented by Google. So what I mean by that is you may start a search in DuckDuckGo, but there’s a pretty good chance that you’re also going to do that same search in Google to validate it. That’s how much market share Google has.

Ben:                So, Jordan, as we think about Google being obviously the dominate player in what we’re calling traditional search, help me think about the greater landscape of digital marketing and the internet. When you think about, and there probably isn’t accurate data for this, but how many searches are traditional searches versus these more verticalized search experiences, like the YouTube, Amazons, eBay’s, Yelps of the world, right? All these different search experiences. What percentage of search do you think Google owns for all searches, not just traditional web searches?

Jordan:              Yeah. So since we’re going over the history, right? If we look at the situation from 2010-2012 to now 2019, so it’s like five or seven year gap here. Really what we see is the proliferation of search in just this like explosive of different search experiences to the point now today where we spend a lot of time and energy talking about IOT search and voice search and these different experiences. And now there’s platforms that are driving those things, like Siri and Alexa and others that are not Google based or Google connected at all. And so the reality is that, like you said, Ben, there’s all this different search that’s now being taken place.

Jordan:              Again, this is an estimation. It’s really hard to get to very specific number here. But I think it’s safe to assume that Google by and large, including the properties that they own, like YouTube, control probably somewhere around 20-30% of all search in the United States. But that has shrunk over the years as new searches have taken place. You have to also recognize that our search behaviors and patterns are ever increasing over time. And so that’s the other realization is that us as human beings are performing more searches than we ever have before. In the 1930s to the 1950s, people were only performing a few searches a week. They would open up their phone book and they would find a phone number. They would open up a physical directory and they would read through the directory and find what hotel they wanted to stay at or whatever it was.

Jordan:             Today, we’re doing sometimes hundreds of searchers per day, and that’s incredible, right? I mean, we’re doing voice searches on our Alexa apps. We’re doing text-based searches on Google. We’re doing video-based searches or local searches on Yelp.

Ben:               So I think the takeaway here, as we think about the greater landscape of search technology, it’s obviously grown a lot since the ’30s and ’40s and the Yellow Pages, and as the technology has increased and Google has become the dominate player in internet text-based searches. Even if they own 90% of the share, there are regionalized search providers. There are still the secondary tier of search providers. There’s also the verticalized search, and we’re going to spend a lot of time this month talking about some of the other search engines that you need to optimize for, like the YouTubes, the Amazons, the local searches, the Yelps of the world. Because in aggregate, those actually represent more searches than Google does, even though Google is the single most dominate player in the search landscape.

Ben:                So that’s what we’re going to be focusing our time on for the rest of the week and also for the rest of the month. And that wraps up this episode of the Voices of Search podcast.

Ben:               Thank you for listening to my conversation with Jordan Koene, the lead SEO strategist and CEO of Searchmetrics. We’d love to continue this conversation with you. So, if you’re interested in contacting Jordan, you can find the link to his LinkedIn profile in our show notes or you can send him a tweet where his handle is @JTKoene. If you have general marketing questions or if you want to talk about this podcast, you can find my contact information in our show notes or you can send me a tweet @BenJShap.

Ben:              If you’re interested in learning how to use search data to boost your organic traffic, online visibility, or to gain competitive insights, head over to searchmetrics.com/diagnostic for your complimentary advisory session with our digital strategies team.

Ben:              And if you like this podcast and you want a regular stream of SEO and content marketing insights in your podcast app, hit the subscribe button, and we’ll be back in your feed tomorrow morning to discuss how you can prioritize traditional web-based search for English speaking, non-Google search engines.

Ben:             Lastly, if you’ve enjoyed this podcast and you’re feeling generous, we’d love for you to leave us a review in the Apple iTunes store or wherever you listen to your podcasts.

Ben:               Okay. That’s it for today, but until next time, remember the answers are always in the data.