How to Use Data Forecasting for SEO [Scripts Included]

Most Ahrefs users will be familiar with our traffic graphs. They show the estimated search traffic for any website or page over time.

Here’s the graph for ahrefs.com:

Having historical data like this is useful for analyzing the past performance of a website or web page, but what about future performance? Is there a way we could use the data to forecast organic traffic growth or decline?

Let me answer that with another graph:

That blue line is the future traffic prediction for Ahrefs. The blue shaded area is the upper and lower uncertainty level (basically, there’s an 80% probability of our forecasted organic traffic being within that range).

Even better, this graph took all of two minutes to make using Ahrefs data and open-source scripts.

In this guide, I’ll show you how to create graphs like these to predict future performance for your website, pages, and competitors.

SEO forecasting lets you use data to make predictions, such as future traffic levels and the value of that traffic. It assumes that past patterns are likely to continue and can help you allocate resources, prioritize SEO efforts, and make adjustments before it’s too late.

The reasons why you’d want to do this vary depending on your situation.

If you work in enterprise SEO, you’ll usually have to make projections of how much you’ll grow SEO and increase revenue once or twice a year. The company will also set goals for you and your team. If there’s a discrepancy between your estimates and company goals, you can argue for more resources. Say you project an uplift of 15% year over year (YoY) at the current effort and resource levels, but your target is 25%. Your expected impact doesn’t align with expectations. In this case, you can push for a lower target or more resources to meet the company’s goal.

If you’re at an agency, SEO forecasting can help with sales or predict the future impact for a client. Most companies want to know when they will be a market leader or beat a particular competitor. Suppose a forecast shows they lag behind their target at current effort levels. In that situation, it’s easy for a salesperson to justify an increased budget for projects, a shift in strategy, or additional hours that will allow them to close the gap or overtake competitors. Adding a forecasting chart to a sales pitch can help elevate things above other agencies and justify your requested budget.

Most SEO forecasts use first-party data sources like Google Analytics or Google Search Console. If you want an accurate estimate for your website or page, this is absolutely the best data to use. If you need more granularity in your data or to predict seasonality, use one of these sources.

The problem is that SEO isn’t a single-player game. What you need to know is how your site or page is likely to perform relative to the competition. If your competitors are likely to outpace your SEO performance, that will influence your strategy and where you devote resources. And the only way to get data on your competitors is to use third-party sources like Ahrefs.

You may wonder if you can reliably use third-party data for forecasting. At Ahrefs, the way we calculate our data does not see daily granularity or seasonality, but it is useful for finding patterns and forecasting trends. We also re-calculate previous data and update our graphs to remove irregularities that would otherwise be caused by updates to our data or calculations.

It probably comes as no surprise that creating graphs like the one I showed above usually requires some knowledge of coding and modeling libraries. That’s why many companies and agencies don’t do forecasting. They don’t have a data analyst or data scientist on their payroll at their disposal.

With this in mind, I spent some time creating a bunch of plug-and-play scripts that you can run for free in Google Colab. Just choose what you want to forecast below. I’ve included a link to the script and instructions on how to use it under each section.

Predicting future traffic

This script will forecast your future traffic one year from now. When I run this one for ahrefs.com, the trend (dark blue line) shows that one year from now, we will likely be up to 1.3 million monthly organic visits from our current level of 880,000.

Here’s how to predict organic traffic for your site:

1. Download your data in Ahrefs

• Navigate to Site Explorer and enter the domain you want data for
• In Overview, go to the “Organic search” tab
• Find the “Organic traffic” chart and select the period you want (it defaults to “All time”)
• Click Export > CSV and save the file

2. Make a copy of the notebook and open it

• Open the Traffic Forecast Notebook
• Click File > “Save a copy in Drive”
• It should open a new window with the notebook, but if not, then go to your Google Drive, find the file, and open it.

3. Run the notebook and upload your file

• Click Runtime > “Run all”
• Upload your file by clicking “Choose Files,” selecting your downloaded file, and clicking Open.

Predicting future traffic value

If you’re not familiar with Traffic Value, it’s what it would cost to buy your current level of organic traffic from Google Ads. Another way to think of it is how much money the target website or URL has saved by ranking for keywords organically instead of buying paid traffic.

This script will forecast your future traffic value one year from now. When I run this for ahrefs.com, the trend shows that one year from now, we will likely be at a Traffic value of $2.9 million, up from our current level of $1.9 million.

Here’s how to predict organic Traffic Value for your site:

1. Download your data in Ahrefs

• Navigate to Site Explorer and enter the domain you want data for
• In Overview, go to the “Organic search” tab
• Find the “Traffic value” chart and select the period you want (it defaults to “All time”)
• Click Export > CSV and save the file

2. Make a copy of the notebook and open it

• Open the Traffic Value Forecasting Notebook
• Click File > “Save a copy in Drive”
• It should open a new window with the notebook, but if not, then go to your Google Drive, find the file, and open it.

3. Run the notebook and upload your file

• Click Runtime > “Run all”
• Upload your file by clicking “Choose Files0,”, selecting your downloaded file, and clicking Open.

Predicting future competitor traffic

This script will forecast not only your growth but also the growth of your competitors. Notice that in the next year, Ahrefs is projected to overtake Moz at current effort levels. If you’re not at the level you want to be or the position you want to be in versus competitors, then a change is required. Maybe that change is a new strategy, new projects, or additional resources to get more done and have more impact. This can help you justify further investment in SEO by estimating your position vs. competitors in the future.

Here’s how to predict traffic performance for your site vs. competitors:

1. Download your data in Ahrefs

• Navigate to Site Explorer and enter the domain you want data for
• In Overview, go to the “Organic search” tab
• Find the “Organic traffic” chart and select  the period you want which defaults to “All time”
• Click Export > CSV and save the file
• Repeat for each website that you want to compare

2. Make a copy of the notebook and open it

• Choose the number of competitors you want to compare.
• Open the correct Traffic Forecast Notebook below:
• Click File > “Save a copy in Drive”
• It should open a new window with the notebook, but if not, then go to your Google Drive, find the file, and open it.

3. Run the notebook and upload your file

• Click Runtime > “Run all”
• This time there are multiple upload boxes. For each competitor, upload your file by clicking “Choose Files,” selecting your downloaded file, and clicking Open.

Predicting future competitor traffic value

This script will forecast your growth in Traffic Value and the growth in Traffic Value for your competitors. Notice that we expect no significant changes in the order of competitors in the next year at current effort levels. If someone wanted to break out and make a change, they would likely need a change in strategy or increase resources to have an impact. An estimate like this can help justify the need for change or funding. For instance, if I were lower on this chart and stagnating while competitors seem to be pulling ahead, I’d be making a case for more resources while also looking into what competitors are doing that I am not.

Here’s how to predict traffic value performance for your site vs. competitors:

1. Download your data in Ahrefs

• Navigate to Site Explorer and enter the domain you want data for
• In Overview, go to the “Organic search” tab
• Find the “Traffic value” chart and select the period you want (it defaults to “All time”.)
• Click Export > CSV and save the file
• Repeat for each website that you want to compare

2. Make a copy of the notebook and open it

• Choose the number of competitors you want to compare.
• Open the correct Traffic Value Forecast Notebook below:
• Click File > “Save a copy in Drive”
• It should open a new window with the notebook, but if not, then go to your Google Drive, find the file, and open it.

3. Run the notebook and upload your file

• Click Runtime > “Run all”
• This time there are multiple upload boxes. For each competitor, upload your file by clicking “Choose Files,” selecting your downloaded file, and clicking Open.

Predicting future competitor page traffic

With this script, I’ve graphed the top pages’ traffic for searches about “Google search operators.” As Ahrefs, I’m reasonably confident that we don’t need to update this post anytime soon as we seem to have comprehensive coverage and a sizable lead in traffic vs. competitors. However, Spyfu appears to be coming on strong, and it looks like their page will surpass Moz and possibly Google in traffic in the next year. If I were Moz, this might be a sign I should do a re-write or add additional content to my page. The same for Google Guide, it looks like their traffic has stagnated compared to competitors, so it may be time for a content refresh if they care about this page. Do you see where I’m going with this?

SEO forecasting for pages can help you estimate your market position in the future. You can use this information to prioritize your content updates based on which pages are most at risk of falling behind competing pages.

Here’s how to predict traffic performance at the page level for your site vs. competitors:

1. Download your data in Ahrefs

• Navigate to Site Explorer and enter the domain you want data for
• In Overview, go to the “Organic search” tab
• Find the “Organic traffic” chart and select the period you want (it defaults to “All time”)
• Click Export > CSV and save the file
• Repeat for each website that you want to compare

2. Make a copy of the notebook and open it

• Choose the number of competitors you want to compare.
• Open the correct Traffic Forecast Notebook below:
• Click File > “Save a copy in Drive”
• It should open a new window with the notebook, but if not, then go to your Google Drive, find the file, and open it.

3. Run the notebook and upload your file

• Click Runtime > “Run all”
• This time there are multiple upload boxes. For each competitor, upload your file by clicking “Choose Files,” selecting your downloaded file, and clicking Open.

Making performance predictions during a core update

I think everyone wants to know how they will perform during an update, especially websites who’ve had issues in the past. I wanted to show how a traffic forecast can progress as new data is added.

For this, I’m going to use a website that historically has seen ups and downs during core updates, verywellhealth.com. Notice that many of their ups and downs occurred during core updates, including a decline in the last major update on May 4th.

For the animation below, I have a blue line for the actual data until July 30th. The red line has no data beyond the days displayed on the GIF. Even before the May 4th update, the model was predicting a decline. When I added the update on the 4th, it predicted an even steeper decline.

Each additional day of data changes the prediction slightly as it incorporates the new information. All of the models were expecting a steeper drop than what happened, but they also predicted a slight recovery. Each predicted model seems to reasonably accurately match the actual traffic by the end of July, nearly two months in the future.

For fun, below is the current forecast for one year from now to see how it does. You’ll notice that based on the fitted trend, the model thinks the traffic at the time of writing should be a little higher than it is now (where the blue line ends), but the site is likely to see some significant declines in the future. It’s predicted that the website will lose roughly half it’s traffic in the next year alone. I guess we’ll see if that holds true.

By default I’ve made some assumptions in the scripts above including:

• Core updates until the time of writing (July 2020);
• A window of 14 days to measure the data after core updates;
• Predictions for 365 days.

Here are some things you may want to do differently that will impact your forecast.

Add additional updates

While I have core updates in the table that start with the earliest Ahrefs data, you may want to add updates that have happened or add future updates to the scripts. You just need to add additional dates to the array in the section below. The format is YYYY-MM-DD.

Change the holiday period

I set the period for holidays as 14 days to give time for Google updates to roll out and Ahrefs to update the database. While I think this is reasonable, your data may show faster or slower changes. Check after the updates how long it seems to take for your data to stabilize and adjust this time if you need to. You can find the holiday time period in the code as upper_window as shown below.

Forecast longer or shorter periods

By default, I have the scripts set to predict 365 days into the future, or one year. You can easily change this if you need to make predictions for a different period like a few months, or even predict multiple years into the future. For each item where you’re making a prediction, there will be a section that includes periods=##. The number here is the number of days you want to predict, so update to get predictions for the period you want.

For instance, if I want to forecast for three years, I’d change periods=365 to periods=1095 and my forecast for future organic search traffic for ahrefs.com would be what is shown below.

Change the period of your initial data

I generally use the full historical data to make predictions by downloading all the data from the organic search overview chart. If you want to use less data, you can either edit the downloaded .csv files or download data for a different period, such as one year.

You’ll notice that in the forecast below for ahrefs.com, the estimate in one year is higher. That’s because a lot of recent growth pulls the trend-line higher. Different periods will yield different forecasts.

Use only part of a website

The script accepts any .csv file. You can use an entire website or filter to a particular section, such as the blog’s path. You can even filter and use traffic for specific pages.

Change the line colors

If the colors on the graph don’t align with those of your brand, that’s easy to change.

First, find the line that corresponds with the website you want to change and update the color for it in the section shown below. Adjust the colors to represent the different websites or brands.

Next, re-run this cell. Click the play button, and the graph will update.

Final thoughts

These forecasts take a simplistic view of the SEO data. A lot can change on yours and your competitors’ websites, such as deleting or creating new pages that target new keywords. More advanced predictions may take into account things like changes in search volume, SERP features, keywords you know you will target, etc. We’re also taking into account both branded and non-branded traffic in the forecasts when you might want to look at that data separately.

Forecasts are always going to have some amount of uncertainty because changes are continually happening. You or your competitors can make website changes, change overall strategy, or make additional investments. Any of these can change the trajectory of the data, so you may want to update your forecasts periodically to take into account any recent trend changes. Have fun forecasting!

Have questions? Let me know on Twitter.