As search and social marketing professionals, we live by the mantra: “Always Be Testing.” However, many search marketing professionals focus on the testing part and ignore the problem definition, goals and results which always need to guide the process. Regardless of what you test, new sets of keywords, landing page layout, even a new coupon code in the check-out process, you must take several elements into consideration before the tests begin. Today we discuss four elements of testing and why you should never enter the test grounds without them. Let the games begin.
The test purpose, and overall effect on the account
Many advertisers and search marketers forget to consider what ultimately will result from a successful test. Asking the simple questions, what do I expect to accomplish with this change; increased CTR, greater conversion rates, increased impressions and clicks, lower bounce rate or some other metric? With a successful test, what overall effect will the changes have on performance? Is this test something that is limited in overall impact, or will the results from this test be able to impact performance across campaigns, accounts or medium?
Your tests keep the overall health and growth of your campaigns, but since time is limited, you must focus your testing on areas of greatest impact and versatility. Finding your new ad text winner impacts your accounts overall performance, but maybe another use of time and testing could result in a greater impact. Implementing a small successful change in the checkout process impacts a much larger portion of your clients and therefore your bottom line. Decide on a series of tests that you want to implement with your client or account. Start by rating these on their potential impact on overall performance. Begin your testing with the elements that pose the greatest impact and then work your way down your list. Once you are done with your list, write a new on or start at the top again. Always be testing, properly.
Defining a valid test
Creating a valid test sounds easy, but it is a tricky business. You must take into consideration a lot of elements when you are doing a test to make sure that only testing variants are being changed. You’ll find a very clean test like a simple A-B is not always possible within the same demographic for various reason.
Often, the tricky situation of testing when the test and control setup can’t run at the same time. You’ll need to look at prior performance compared with the test situation. However, even if you isolate other variables like changes to bids, keyword changes, ad changes, competitor’s actions, etc. you also need to consider if things like product discounts, seasonality or even dramatic weather changes could skew the results. These challenges make your test more difficult to set-up and interpret accomplish shouldn’t discourage you from running the test. However, prepare yourself to throw out the results if something unusual or expected happened during the test. Just because you ran a test, doesn’t mean you gathered valid or actionable results. Toss them out before heading down a road that results in worse performance.
Size of sample and duration of test
When setting up any test, look for your best friend volume to rely on. Turn the volume up to eleven if you want results quickly and decisively. Slow moving accounts make gathering and interpreting results difficult to impossible. With slow moving accounts, get creative and run your test over a larger amount of campaigns, a greater split of your website traffic or across sources. The greater you expand your scope, the greater chance to bring invalid elements to the test such as different audience types lumped together. However, avoiding problems by running a very long test results in seasonality issue. Create the best balance possible by designing understanding the volatility of your account and working around the natural ebb and flow.
The never-ending test steals your attention, your budget and opportunity. Know in advance how much activity you will send through your test and how long you expect the test to run. When the time frame of the test expires, make a decision. Generally, “I need more time” is not the right decision. If you observe little or no difference between the test and control then move on. Do a different test. Perhaps you didn’t allocate enough volume. In that case, stop the current test, expand your scope or pick a different measurement (like CTR instead of conversion rate). Begin again.
Budget for testing and communication
Finally, many accounts run so close to the overall goal, the idea of running tests terrifies the search marketer. Unfortunately, you must test to grow or even keep up. Without any money in the testing budget, accounts decline as competitors hone their efforts and yours remain stagnant. Create well defined tests that show what we plan to accomplish. Plug holes in your budget from long running tests that never bear fruit and set expectations with clients. Work with your clients or management on understanding your use of testing so everyone buys into the costs and goals.
Testing follows the “fail to plan, plan to fail” concept. However, not many aspects of search marketing provide the potential impact of successful test on a large scope concept. Take the time and effort to properly consider your testing practices before you dive in head first. Your results provide less headaches and many more high scores from the judges clients.
