Testing is essential for anybody running a paid search account, and there are many guides for how to go about executing one. However, the majority of them cover how to test in high volume accounts, accounts where it is arguably much easier to accrue the data necessary.
In low volume accounts, tests typically take 30 to 60 days to reach any sort of significance, and on occasion up to 120 days. If you’re isolating a variable, that’s a long time to keep things the same in a testing environment that by definition, is ever-changing. One thing I have learned while working with low-volume ad accounts is that setting expectations with key stakeholders is a necessity.
These expectation setting conversations typically revolve around three themes.
‘It is going to take time’
This conversation should be had early on, as soon as the line “let’s test it!” is triumphantly declared in a meeting. My immediate reaction is to see how long the test will take and investigate the reality of running it, before agreeing it’s a good idea.
Chances are, in any account with a lower budget, your test is going to take 30 – 60 days and now is the time to bring that up. It is in your best interest to address what you’re looking to find, the hypothesis, the metrics you are going to measure success by, and then use historical data calculate how long it is going to take.
Sending an email about the test you are going to run, how long it will take, and why it will take that long puts the information in writing and ensures that everybody is on the same page. If you receive any pushback later, it can be helpful to refer back to what everybody decided on initially.
‘Please don’t touch that’ (aka: ‘If you change that it will affect results’)
Things change over the course of 30 to 120 days, and it is inevitable that the “please don’t touch that” conversation is going to come up. Due to the length of tests in low volume accounts, I’ve had to train myself and my team to check what is currently being tested before doing optimizations so as not to disturb any tests in progress. It’s natural that people will forget testing is occurring when it can take up to four months.
Any change to the testing environment will affect results, and if the test is multi-campaign, then you have to be incredibly careful about every change you make. However, there are numerous occasions where you may need to make a change, whether out of necessity for the account or simply because a superior/client requests it.
In these situations, the best thing to do is to describe the effect making that change will have on the test and document the fact that the environment has now been disturbed. This helps ensure that everybody is on the same page for when you interpret the results later.
‘These results are going to be random’
It isn’t unusual for somebody to simply declare an end to the test, a ‘winner’ per se, whether or not we have reached statistical significance. One way to avoid this is to be very thorough in the first conversation, by defining what metrics you are going to measure success by.
However, if you have an unavoidable end to a test, it is important to explain what this means. The results of that test, without significance, are random. You cannot be sure that the favoring of one variable over another is scientific, or due to chance. Therefore, any decisions made off that test will also be random.
The danger here is that you continue to test variables by building off of random results, and ultimately missing out on making the better decision because the test didn’t run to significance. However, it happens, and when it does, the best thing you can do is explain why the results are no longer scientific to those involved. In that case, you’ve done your due diligence.
The bottom line
There is no perfect testing environment in PPC, and there never will be, as there are many variables we cannot control. Low volume ad accounts are especially vulnerable, as things like user intent and seasonality will always affect tests that run for multiple months.
Controlling how we communicate testing to those involved in our accounts, and ensuring that we are proactively discussing testing, will allow us to make the best possible decisions when optimizing them going forward.
Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.
About The Author
Amalia Fowler is the director of marketing services at Snaptech Marketing. She manages a team of strategists who develop holistic digital marketing strategies for clients. Passionate about testing, marketing psychology and digital strategy, Amalia speaks frequently at industry conferences and events. Outside of marketing, she’s a coffee, paddleboarding and Vancouver enthusiast. You can follow her on Twitter @amaliaefowler for all things marketing related.