How to Run an A/B Test

When you run an A/B test, you compare one page against one or more variations that contain one major difference in an element of the control page. After a set amount of time, or visits, you compare the results to see how the change affected your results.

A/B testing is actually a nickname for a variety of testing that involves single element changes across multiple variations, so it might actually be A/B/C/D testing, but it is still called A/B testing and if you see A/B/n or Split Test that is just a more accurate expression of one control with many variations.

This article will help you out with:

  • Instigating A/B tests
  • Assigning traffic to your page variants
  • Deciding which elements to test and what to test them for
  • Editing different variants of your landing page

A Quick Start to A/B Testing

From the Page Overview you'll see a section called your A/B Test Centre.

To create a variant of your landing page you can click "Start with a copy of your Existing Page" (easier), or "Create a New Page Variant from Scratch" (more difficult).

You'll see the variant appear in your A/B Test Centre and you can click the edit button to adjust the page how you'd like in the Page Builder.

Each variant has a letter name on the far left that you cannot edit - it goes up sequentially with each variant.  You can edit the variant name by hovering over it and clicking the pencil.

Assign traffic to the variant by clicking the weight percentage and adjusting how you'd like. Once you make a change here you'll need to republish your page to see it take effect.

If you want one of your variants to become the new Champion, just click the associated gear menu and select "Promote to Champion" - you can also duplicate or discard (and then delete) a variant from this menu.

Note: you can't delete your champion variant or assign it 0% traffic.  You'll need to promote another variant to champion and then delete it once it's a challenger variant.

You'll see all the data from your test in the A/B Test Centre - Visitors, Views, Conversions, Conversion Rate, and Confidence.


How Do I Run an A/B Test?

Step 1 - Create a Page Variant

Start_AB.png

 Starting with a copy of your existing page is the easiest way to start a test.

The basics of A/B testing are that you run different versions of your page – called variants – side by side with a series of changes in each variant. The idea is to see which one converts the best.

You can run as many variants in your test as you like (e.g. an A/B/C/D/E/F/G… test) but the more variants you have, the more traffic you need to send to your campaign to be confident that the statistics are not a coincidence.

To create a variant:

  • Open a page from your Pages tab
  • Click one of the two big buttons in the A/B Test Centre to duplicate an existing variant (easiest method) or create a variant from a blank page

That’s it! You now have 2 page variants  that you’ll use as the basis for your test.

Step 2 - Decide WHAT to Test

Now we need to decide what you’re going to change on your new page variant. Here are a few suggestions of page elements that are worth testing:

  • The primary value proposition or campaign message
  • Your hero shot – the main photo or product shot
  • The body text that describes the details of your offer
  • Your bullet points (features & benefits)
  • Change the wording of your primary Call To Action (CTA)
  • Change your lead gen form

You can change just one thing or change many at once to create a completely different page.

Changing many things at once might produce a more dramatic change in conversion, but will provide you with less causal information than just trying one thing at a time. It’s totally up to you.

Step 3 - Decide HOW to Change Your Test Elements

Now that you’ve decided which page elements you want to test, you need to think about *how* to change them to give you a conversion lift.

Looking at the list of elements in Step 2, you could try some of the following things:

  • Make your lead gen form shorter (ask for less information)
  • Give your lead gen form a more prominent position (higher up the page and try it on the right-hand side)
  • Simplify the page with less text, or separate it into bullet points for greater clarity (as an exercise – try removing 50% of your page copy)
  • Make the CTA bigger
  • Add a directional cue (such as an arrow) that points to the CTA to draw attention to it

Successful manipulation of emotional and physical response is your key to conversion success. By utilizing some principles of design you can start refining your pages.

For more design ideas check out the blog post below:

Designing for Conversion - 8 Visual Design Techniques to Focus Attention on Your Landing Pages

Watch the Stats Promote a Champion

Testing_Center.png

Now that you’ve created one or more new page variants you need to start the test. To begin your A/B test follow these steps:

  • Assign each variant some traffic (weight). To do this, click on the 0% beside your variant (in the weight column) in the A/B Test Centre, input the % weight you want to assign that variant, then press enter to confirm the changes. You can assign as little or as much weight as you like. Note: the more traffic you give it the faster you’ll gain confidence that your test results are valid. For a test with one Champion page (the original one) and one variant, you can just try doing a 50/50 split.
  • Publish (or re-publish) your landing page
  • Start driving traffic to your page. Note that you will only ever have one URL for this page – Unbounce randomly selects which page a visitor sees based on your traffic weighting.  

When a visitor first accesses your page, we deposit a cookie on their machine to track the view and the variant they saw. As long as that cookie is present when the visitor returns, they will continue to see the same variant. The same goes for you visiting your own page.

  • Watch the confidence column to see how statistically valid the test is. The more traffic you drive to your page, the faster this number will climb. Once the confidence is above about 90% you can be more confident that the conversion results you’re looking at are valid.
  • At this point if your test variant is outperforming the Champion, (higher conversion rate) you can promote the variant to become the new Champion page! From the Page Overview screen, promote a challenger variant by clicking on the gear next to that variant and then choosing "Promote to Champion" from the drop down menu.

Your old Champion will now be discarded (but not lost) and you will have to begin the testing process over again to see if you can make it even better

champion_variant.png


Glossary

Page Variant: a slightly different version of your landing page. For A/B testing, only one element is different amongst variants.

A/B test: Run variants of your landing page simultaneously to see which one converts the best.

Champion Variant: original version of your landing page or a new landing page that you've promoted to be Champion

Confidence level: shows how statistically valid your A/B test is and that your conversion results didn't occur by chance.

 

Back to: Testing Your Landing Pages >>