Blog

At DIGO, we’re often asked how we approach our e-mail and other direct response testing. Do we have a list of variables and guidelines?

The answer is, at DIGO we don’t so much have a list as a testing philosophy. Of course we read Direct Marketing News, belong to the associations, read their websites, attend webinars, get newsletters, judge and attend award shows, and generally do all we can to get into the stream of information that professionals in the industry ought to digest.

Lists of testing options and opinions based on that sort of information can be accessed with a few Google searches, and the advantage of Google is that the world is continuously updating the results.

But at DIGO, our philosophy leads us to use that stream of information differently and, we have good reason to believe, to get better results.

Most e-mail (and most direct) marketers are Incrementalists. We’re Optimizers.

An Incrementalist wants to isolate a variable and produce a measurable lift. Four-word subject lines vs five-word subject lines, for example. Three paragraphs vs. two. Picture vs. no picture. Color vs. B&W. Picture on the right vs. picture on the left. There is nothing to say that this kind of testing can’t create large lifts and return on investment, except that it usually doesn’t. People trained at large companies need only create small percentage lifts in order to generate adequate returns on investment. Smaller companies, and more ambitious companies who want to win over many more people, are better off going for not just incremental improvement, but optimal results.

This is messier, and you will typically have to rely on intuitive understandings of why one cell outperformed another.

Optimal testing is built on insight-based hypotheses. Instead of artificially limiting the expression of the hypothesis in order to isolate variables, the implications of the hypothesis are extended to each touchpoint.

For example.
INSIGHT: Growth oriented business leaders are drawn to solving problems and answering key questions.
CREATIVE IDEA: Are You An Incrementalist? Or an Optimizer?

This insight would affect the subject line, the headline, the imagery, if any, the design, the call to action, perhaps even the offer. A lot of work… but worth it if the insight is as powerful as I suspect it might be.

INSIGHT: E-mail marketing is undergoing a revolution, and growth-oriented marketers are hungry for information.
CREATIVE IDEA: Optimizing Your E-Mail Channel.

INSIGHT: Growth-oriented brand and business leaders are drawn to a challenge.
CREATIVE IDEA: Your email testing strategy isn’t cutting it.

INSIGHT: Brand-driven growth leaders are inherently curious. Many read widely.
CREATIVE IDEA: The Freakonomics of E-Mail Testing.

Beyond the insight-centered hypothesis that drives each test for us, we approach the development of an email as iterative direct response and engagement design. Therefore, the Subject Line must compete with the other subject lines in the Inbox. It’s important to design in context. Look, for example at how information is displayed in the most popular e-mail providers’ applications. The “From” for example, might be a worthwhile variable. The job of the subject line is to get the email opened. The job of the first impression of the opened email is to get the email read and to build excitement toward the call to action. The job of the call to action is to get clicked. It is productive to test ways to increase interest, but also to test ways to reduce friction or resistance. Friction or resistance would involve factors that discourage response, such as fear of commitment, or fear of accidentally losing privacy or autonomy by responding.

Incrementalism in direct response testing can be a narcotic. Your results are better, and you feel you’re learning valuable things. The money you’re leaving on the table is out of sight and out of mind. So, you feel you’re doing your job pretty well. Often you don’t even notice that an Optimizer has eaten your lunch until it’s too late.

“If we keep doing better and better, then how come we’re losing market share?” The right sort of leader will start asking some sharp questions. The wrong sort of boss might begin to lose faith in direct response marketing altogether – decidedly the wrong conclusion. But that’s the way of the world. Optimizers gain power, credibility and budget authority. Incrementalists have orderly, comfortable, and brief tenures punctuated by complete loss of control when someone else must take responsibility and set things right.

Which is not to say that there is never a time or place for incrementalist experiments. We’ll talk about multivariate testing dos and don’ts in another entry.