Skip to main content

Test-and-learn steps for agencies

We have seen a rapid expansion of entertainment and news options for consumers over the past few years. This explosion of options has fractured the traditional marketing mix, presenting a myriad of new choices for clients and the agencies who support them.

Leading agencies are always seeking opportunities to drive their clients’ business forward. Understandably, however, agency staff and account managers are leery of failure: Nobody wants to tell a client something they tried didn’t work. So, how do agencies prepare their clients for a robust test-and-learn approach?

In this article, I’ll outline a few steps that can improve the chance of finding the next great idea for clients.

Step 1: Develop a clear hypothesis for client success

Agencies need to have a clear understanding of their client’s problem and present well-informed ideas about potential solutions. The more clearly the agency and client define the challenge together, the easier it will be to design tests and interpret results.

At this stage, it’s essential to be specific. Does your client want consumer consideration? Are they looking to increase consumer loyalty? Clear goals help define a testing journey that will lead to more definitive results.

Step 2: Focus on ideas with the highest potential

We all know there are bad ideas! In fact, many of us have probably been burned by one or two of them. As marketing professionals, we’re generally looking for ideas that will generate an above-average ROI for our clients. No client is looking to spend more for website visits, consumer purchases or in-store traffic. Agencies seeking to find more of these good ideas will have a simple way to identify concepts with a higher-than-average chance of increasing ROI.

One way we use to identify better options is running a quick back-of-the-napkin spend-to-impact response function (SIRF) analysis. SIRFs are the building blocks one can use to drive increased return on investment.

Here are the numbers needed:

  1. Spend: Cost of program (Consider costs to produce separate from distribution).
  2. Impact: Estimated sales lift (Calculate potential loss of effects with increased frequency).

Agencies can do this analysis very quickly. Just compare results to average ROI and see if you have a shot at increasing marketing productivity.

If the ROI doesn’t seem to work with this simple analysis, move on. You want to focus your attention on the ideas that have the most merit.

Step 3: Figure out if the idea has staying power

If the idea seems to have merit, you’ll need to figure out if the approach will scale. Is the idea going to end up being a one-and-done, or can it be repeated? Does it have an impact on a niche of potential customers, or will a more extensive audience respond?

In our simple approach, we did a simple analysis of 1,000 people. We only included the cost of media and left out any potential diminishing returns. But in any test of a marketing effort, we have to remember that the first impression is probably going to give us the most significant impact. Subsequent exposure will have less impact. The ROI will decline as the program reaches more people or reaches the same people multiple times.

Here’s a simple approach to see if an idea can scale.

In this example, the client receives +6 pts from first impression, another +3 from the second impression, and +1 from the third impression. You can plot this on a simple formula with a reach and frequency curve.

Step 4: Identify the benefits of reach versus frequency

There are two factors in every marketing effort that one needs to consider: reach and frequency. Seeing how these will impact the successful idea is a key to identifying winners. Here are two scenarios:

In the first, the client’s message reaches a large number of people, but the majority see the campaign only once. This example gives your client more reach but less frequency.

In the second, fewer people see the client’s message, but they see it more often. This example would give your client less reach but more frequency.

We already discussed that fact that most of the immediate impact and ROI will come from the low-hanging fruit: consumers already inclined to respond. Subsequent efforts (more frequency) will of necessity be reaching harder-to-sell, less responsive consumers. ROI will decline as frequency grows.

This quick sensitivity test can help the agency better identify efforts more likely to drive a client’s ROI and improve business results.

Step 5: Identify meaningful success metrics early on

If you’ve made it this far, you’re probably ready to move forward and present the idea to your client. Now is an excellent time to reassess your hypothesis and make sure you’re collecting the data that will help you assess the performance of the idea.

For example, one of our clients was testing new messaging in digital advertising. When the data was initially analyzed, it appeared that the new ads were a home run: Their online traffic was experiencing huge gains. However, digging a little deeper into the metric that mattered (conversions), it was apparent that most of these new visitors were leaving without taking any meaningful action. They were “empty calories.”

Final thoughts

I am a big believer in test-and-learns. I’ve baked them into my company’s culture, and they’re a part of every one of our client engagements. The process we’ve outlined above can help you unlock more potential for your clients through tests-and-learns. It will also lead to a database of institutional knowledge that becomes a strategic asset for your agency.

We recommend every client dedicate at least a portion of their agency’s time and their budget to challenging existing plans and uncovering the next big idea.

via Marketing Land


Popular posts from this blog

6 types of negative SEO to watch out for

The threat of negative SEO is remote but daunting. How easy is it to for a competitor to ruin your rankings, and how do you protect your site? But before we start, let’s make sure we’re clear on what negative SEO is, and what it definitely isn’t.Negative SEO is a set of activities aimed at lowering a competitor’s rankings in search results. These activities are more often off-page (e.g., building unnatural links to the site or scraping and reposting its content); but in some cases, they may also involve hacking the site and modifying its content.Negative SEO isn’t the most likely explanation for a sudden ranking drop. Before you decide someone may be deliberately hurting your rankings, factor out the more common reasons for ranking drops. You’ll find a comprehensive list here.Negative off-page SEOThis kind of negative SEO targets the site without internally interfering with it. Here are the most common shapes negative off-page SEO can take.Link farmsOne or two spammy links likely won’…

Another SEO tool drops the word “SEO”

This guest post is by Majestic’s Marketing Director, Dixon Jones, who explains the reasons for their recent name change.
Majestic, the link intelligence database that many SEOs have come to use on a daily basis, has dropped the “SEO” from it’s brand and from its domain name, to become Since most people won’t have used Google’s site migration tool before, here’s what it looks like once you press the “go” button:

In actual fact – there’s a minor bug in the tool. The address change is to the https version of (which GWT makes us register as a separate site) but that message incorrectly omits that. Fortunately, elsewhere in GWT its clear the omission is on Google’s side, not a typo from the SEO. It is most likely that the migration tool was developed before the need for Google to have separate verification codes for http and https versions of the site.
The hidden costs of a name change
There were a few “nay sayers” on Twitter upset that Majestic might be deserting it…

Software Review Site TrustRadius Has A New Way to Treat Reviews Obtained Through Vendors

Online user reviews are the most powerful marketing technique for influencing purchase decisions. But do they accurately represent the views of most users?Today, business software review platform TrustRadius is announcing a new way — called trScore — to handle the bias introduced in reviews by users obtained through the vendor of the reviewed software product. The site says more than two million software buyers visit each year to check out its product reviews.To understand trScore, let’s first look at TrustRadius’ approach.The site says it authenticates all users through their LinkedIn profiles. It also requires users to answer eight to ten questions about the product, in order to weed out users having no familiarity. Additionally, a staff person reads every review before it is posted, and the site says about three percent of reviews are rejected for not meeting guidelines.As for the reviews themselves, TrustRadius puts them into two main buckets: independently-sourced reviews and ven…