Silverpop - 3 Things Marketers Should Be Testing (But Probably Aren’t)
It appears you are using an older version of your browser. This site was developed to be progressive and future-compatible. Please take a minute to upgrade your browser for an optimal experience.
Skip to content
  • Subscribe:

3 Things Marketers Should Be Testing (But Probably Aren’t)

blog post thumbnail image
by: Adam Steinberg (@adams472)
15 October 2014

With the rise of big data, the most effective marketers are constantly testing their assumptions and using a data driven-approach to optimize their marketing programs.

For many marketers, being data-driven starts with A/B testing. While A/B testing isn’t as sophisticated as multivariate analytics, it’s a simple way for marketers to drive immediate results. For example, one marketer attributed a $200 million gain to A/B testing.

While there are many sophisticated A/B testing products available for marketers to use, there are also several basic tests that marketers can run to generate results. Here are three quick ways to get started with A/B testing.

1) Subject Line Testing

Your email subject line is like a window to the soul of your content. Without a compelling subject line, you run the risk that even the best email content won’t be opened.

What to Test

Start by running a simple experiment to test two different versions of your email subject line. You’ll want to focus on testing just one specific element of the subject line.

For example, here’s an example of a good subject line test:

  • Option 1: Learn how synthetic clothing helps you run faster
  • Option 2: Learn how synthetic clothing helps you run 10% faster

This is a good test because you’re testing one thing: the value proposition. Does a general proposition (“helps you run faster”) or a specific one (“10% faster”) drive more conversions?

Here’s an example of a less ideal test:

  • Option 1: Fall sale spectacular. 30% off everything!
  • Option 2: See our newest styles. Plus, 20% off everything.

Why is this a less effective test? Because you’re testing two different elements. First, you’re testing the difference between a 20 percent and a 30 percent discount. Then, you’re also testing the difference in the copy: “Fall sale spectacular” versus “See our newest styles.” Because of the differences in your copy and the discount amount, it will be difficult to understand which aspect led to the difference in email performance.

How to Run the Test

To run a subject line test, you’ll want to test your different subject lines on a small percentage of your email list.

  1. Segment your email list so that you can send your subject line test to 10% of your total audience. (You may need to tweak the percentage based on the size of your audience.)
  2. Send each subject line test to 5% of your audience.
  3. Identify the subject line with the best performance. Use that subject line and send the email to the other 90% of your audience.

2) Cart Recovery “Message Timing” Testing

Cart abandonment programs have one of the highest ROIs of any marketing program. For example, S&S Worldwide sees a 25 percent recovery rate with its cart-recovery campaigns.

There are many elements of a cart recovery email that can benefit from testing: number of messages sent (one versus a series); inclusion of images and pricing information; use of countdown timers; offering of discounts (see #3 below), and more. At the top of this test list is the amount of time that should elapse between when a user abandons a cart and when that person receives a reminder email or display ad.

Depending on your customer and the products you’re selling, a real-time abandonment reminder may be appreciated and drive great results – or it may be seen as overbearing and decrease engagement. The only way to find out is to test.

What to Test

In this case, you want to test how quickly a customer will receive a follow-up offer for their abandoned cart. Should the campaign be triggered immediately, or should you wait a few hours, or even a few days? Remember to dig beyond open and click-through rates when you’re determining the winning timing — conversion rates, average purchase amount and unsubscribe/spam complaint rates should also be factored in.

How to Test

Within your abandoned-cart campaign, you’ll want to set up two different times that your campaign will be triggered. For example:

  • Option 1: Trigger abandoned-cart program immediately
  • Option 2: Trigger the campaign four hours after the cart is abandoned

Make sure that 50 percent of your customers are entered into Option 1 and the other half are entered into Option 2.

If your abandoned cart solution doesn’t have A/B testing, you can use different time periods for the A/B test. For example, use Option 1 for Week 1 and then use Option 2 for Week 2. Then, compare your results week to week.

3) Discount Amount Testing

Offering a promotion or discount is an effective way to drive clicks and traffic, but what’s the optimal discount amount? Most marketers just use a default amount for every campaign. For example, your default might be 20 percent.

But is there really a performance difference in offering a 20 percent discount versus 15 percent? What about 25 percent or 10 percent?

What to Test

An extra 5 percent can make a huge difference in your profit margins, so you want to make sure it’s driving an incremental increase in revenue. Try starting with your default discount and testing it against a discount 5 percentage points less. Based on your initial learnings, you can test additional discount offers.

Bonus tip: Don’t just look at conversion rates! You’ll also want to make sure you’re examining total lifetime spend and profitability for the customers you acquire with each discount amount.

How to Test

Without using advanced personalization tools, you can test your discount amount by simply offering different discount levels each day or week. For example, on Monday try offering a 25 percent discount on your website. Next Monday, offer a 20 percent discount. What’s the difference in performance?

Want to get even more sophisticated? Using personalization and marketing automation products, you can segment your campaigns so every customer receives a unique discount offer. For example, in your weekly email newsletter, half of the recipients that open your email could receive a 15 percent discount, while the other half could receive a 10 percent discount. Then, you can track the conversion rate and revenue for each group of customers.

Wrapping Up

Testing is critical for optimizing your marketing campaigns and driving the most productive results. If you’re not doing any testing today, pick one of these ideas and get started. With a little bit of effort, you’ll be amazed by the results.

Related Resources:
1) Blog: “Subject Line Testing: Tips for Getting It Right

2) Ebook: “Ultimate Guide to Assessing Your Digital Marketing Program

3) Blog: “5 Steps to Optimize Your Content in 2014


Sign up Now!

Subscribe to IBM Marketing Cloud's Digital Marketer Newsletter!

Popular Categories

Top 5 Posts


To give you the best experience, this website uses cookies.

Continuing to use this website means that you consent to our using cookies. You can change your cookie settings in your browser at any time.
Find out more here or by clicking the Cookie Policy link at the bottom of this page.