The SERP Evolution Strategy

Thomas Jefferson once said, “I like the dreams of the future better than the history of the past.” But if we aspire to rank well for a particular search term, it behooves us to understand how that SERP has changed over time.

The Goal of Analyzing Changes in SERP

The purpose of the SERP Evolution Strategy is to understand what’s causing the change in the SERP for a specific search term. Knowing this can help with both offensive and defensive tactics.

If we’re an up-and-coming site, we’re looking to see how other sites have broken into the top page. If we’re already on the front page, we need to determine what defensive actions we need to take in order to, at a minimum, maintain our position.

• Has user intent changed? Search engines are getting better at determining user intent, and that could be a reason for the changing SERP. It could also be that searchers have changed the way they use that term.
• Has page quality improved? Perhaps the top pages have been busy updating, expanding, and improving on the topic. If so, we need to incorporate consistent article updates into our content plan!
• Is there more supporting content and, if so, how much? Discovering this will significantly impact your content strategy and plan surrounding this search term. One page targeting that term may no longer be sufficient. In that case, it’s important to figure out how much supporting content is required.

What You Can Learn From Analyzing a Changing SERP

There’s a lot you can learn from observing the changes in the SERP over time for a specific search term. Your focus may vary depending on your position in the SERP. Here are a few discoveries you can make:

1. Determine who has been consistently performing
   well and why.
2. Discover the new kid on the block and how they
   got there.
3. Find out who has been sinking in the SERPs and
   how not to end up like them!
4. Figure out what accounts for the difference
   between staying on the top of page one versus never getting above second page.
5. Why some sites appear to bounce all over the place.
6. Why a site can be a steady performer in the
   middle of the pack.

Determining How the SERP Has Changed Over Time

Here are the tools we used in this example:

1. MarketMuse
   Suite (for quality scoring)
2. SEMrush (or
   any tool that provides historical ranking data)
3. Wayback
   Machine (to historical content on specific sites)
4. Moz
   (link velocity and SERP feature analysis)
5. DiffChecker
   (determine changes in text)

Get Ranking Data for a Specific Keyword

The first step is to download SERP data for the time period in which we’re interested. Historical data is presented in a monthly format. Depending on what data you select, you can conduct a monthly, monthly, or yearly comparison. But you will have to download the data month by month.

Open up SEMrush, go to the Keyword Overview page and enter in a specific keyword. Select the historical month from which you want to pull data.

Scroll down to the bottom of the page and hit the export button to export the first 100 entries in the SERP.

Open up that new file. For each entry, you’ll need to add the historic date and its position in the SERP.

Repeat this process for each month you wish to download.

Take all the downloaded files and merge them together into one master file (cut and paste is the easiest way).

Take the master file and create a pivot table. (Here’s a detailed explanation if you’ve never created a pivot table).

Analyzing SERP Changes

With this pivot table in hand, it’s time to do some preliminary analysis. We’ll add more data points later. But for now, we’re just looking at changes in ranking. Depending on your scenario, you may want to identify:

1. What’s gone up in ranking?
2. What gone down?
3. What’s been bouncing around all over the place?
4. Who is the newcomer tearing up the charts?

Take for example businessdictionary.com which managed to hang on to the last spot on the front page during 2017. Since then it has fallen on hard times dropping far down the SERPs. As a content strategist, I don’t want to guess what happened. I want to know what they’ve done wrong so I can avoid the same fate.

With the help of the Wayback Machine, I can find a historical snapshot of this page from 2017. Putting the URL into the Optimize app in MarketMuse Suite I can quickly see this page has a very low content score.

A quick check of their current page confirms a nagging feeling. They haven’t updated that article since at least March 2017. Old and outdated, poor-quality content. A look at the link profile for this page will confirm my suspicions.

Link velocity has remained stagnant all this time. No surprise there! Who links out to crappy content?

The page for Pardot has seemed to suffer a similar fate, dropping from 12 to 41 over the past year! Like the previous example, they too suffer from low quality and word count. It would be worth further investigation to see if the article has been updated and whether it has a healthy link velocity.

What’s happening is we’re looking to establish a pattern. At this point, I suspect that the competitive landscape for this search term has changed dramatically since 2017.

According to MarketMuse analysis of the top pages for this term, we should aim for a target content score of at least 37 and a word count of 1,257. Expecting to rank on page one for this term with just a definition is no longer realistic.

Since SERP placement is a zero-sum game, let’s take a look at who replaced the ones that fell off the first page during the past year. Perhaps that will give us some additional insight as to what is required to rank well for this term, given the current environment.

Zoho, BigCommerce, and Kuno Creative are three examples that have moved onto the first page of the SERP over the past year. BigCommerce had been a second page entry which now, in 2019, is solidly on the first page.

What did it do to get there?

Applying the same process as before, use the WayBack Machine to view the history of that page. Plug that URL into MarketMuse Suite to get the content score at that time. Compare that score to their current one to see how they have improved.

If the content score has changed, use DiffChecker to determine what text they’ve altered or sections they’ve added. Moz Link Explorer can help determine if there’s been any change in link velocity as a result of those content changes.

Examine Changes in SERP Features

Changes in SERP features can also hint at how Google perceives the intent behind the search term. Use Moz to see if there’s a local pack, answer box or knowledge panel or another feature that wasn’t there before. Continuing to layer on intelligence about the evolution of this SERP to understand what Google finds to be important.

Going Beyond the SERP

To compete with the top pages for this SERP, we need a target content score of at least 37 and a minimum word count of 1,257. But is that enough? To find out, we need to take this yet another step further and examine what supporting content the top-ranking sites are creating around the search term.

A Google site search of the keyword phrase sheds some light on what content has been published. Set a custom date range by clicking on “tools” in the menu bar and selecting a date range. Put the URL and publication date in a spreadsheet so you determine the amount of content published, along with its cadence.

Make sure to do this for all the up-and-coming entries on the first page of the SERP. It will give you a better of how much supporting content is getting produced.

The Impact of SERP Changes on Strategic Planning

We want to quantify how much content is needed to support our bid to get onto the first page of Google. If in the last year the competition published 24 pieces of supporting content about lead-management, it impacts both strategy and planning.

That may be what it took a newcomer to get onto the first page. What will it take to stay there?

For this search term HubSpot, Marketo, and Capterra have consistently appeared in the top 10 for the past couple of years. We need to determine how much supporting content each of these sites produced over the last year. That will give an idea of the level of commitment necessary to maintain that position.

And it’s all backed up by data.

What’s it Going to Take to (Keep my) Rank?

We can surmise that a simple definition-type page has little change of ranking in the current environment. High-quality pillar-style content is what will need to be produced.

But that’s not enough. We’ll need to update that page on a regular basis if we want to maintain front-page ranking.

A cursory analysis revealed that the competition has been busy creating supporting content around this key topic. With a little research we can determine exactly how many supporting articles are being produced and adjust our output accordingly.

Even with all the wonderful tools available, this is still largely a manual process that takes time. No doubt you’ll reserve this type of deep analysis for only the most important keywords.

In the end, you’ll have the insight and data to determine whether that objective is worth pursuing. If it is, you’ll have a realistic plan for achieving that objective.