If you’ve read my previous posts on analytics, you’re already measuring your data and getting ready to set up a test. You know what assumptions you want to test and are ready to jump into the world of testing. Or, if you just want to don the hat in this video, skip today’s post 😉
The simplest type of test—both to run and to interpret the results of—is the A/B Split test. Essentially, you are testing one variable: one page versus another page, or one button versus another button, or one form versus another form, and so on.
Ultimately, it is a univariate test; that is to say, there’s only one variable. The reason that it’s called a “split” test, though, is that in order to run the test, you need to split your traffic so that a random sample of visitors see your control (group “A”) and the rest of your traffic sees the version you’re testing (group “B”).
You can actually…
…split your visitors into more than two groups and run tests A, B, C, D, and E. As long as you’re testing only one variable (regardless of how many iterations of that same variable you’re testing), it’s still considered an A/B Split test.
If you’re testing one small variable—like the color or size of a button—you’ll have a pretty good idea which specific color caused conversion to go up. If, on the other hand, you’re testing an entire homepage and run two drastically different versions of the homepage, then you won’t necessarily know what made the difference. You will still be able to tell which page had a better conversion rate, but you may not know why.
The magnitude of the change you’re looking at (i.e. a button color versus an entire homepage), may be dictated by how heavily trafficked or revenue-generating your site is. In other words, if you have an e-commerce site, you may want to test out the shopping cart button by itself rather than testing two (or more) homepages that have been laid out differently.
Google Analytics "Content Experiments" (formerly Google Website Optimizer) is the easiest option. Just sign up for a free account, peruse the installation instructions and/or demo, and start rocking and rolling. As an overview, we’ve included an easy step-by-step of the process, below. Google will give you considerably more information, but these are the basics.
- Choose the page or element you want to test. This is your “A” variable.
- Identify which page a user will ultimately get to if he/she completes the desired action. This is known as the “conversion page.” (For example, if you are testing “Buy It Now” buttons, you may be aiming for someone to complete a purchase. Thus, your conversion page would be the post-purchase receipt/thank-you page.)
- Create at least two different versions of the page or element you wish to test. This is your “B” variable (and, possibly, your “C” and “D” and so on).
- Run the iterations through Google Analytics "Content Experiments", which will spit out a code designed to divvy up your traffic and track the results of your test.
- At this point, you’ll have to know how to embed the HTML script tags that Google gives you into your site’s code…or hire someone who does. You will embed both control scripts and tracking scripts to your original “A” page. On the “B” variation page, you will add only tracking scripts. The control script divides your traffic evenly, while the tracking scripts record the user’s action.
- Then, you can run the test for the desired period of time. (Note: You need to have enough visitors to your site in order to run a reliable test. In order to figure out how long you should run your test for, use VisualWebsiteOptimizer.com’s Test Duration Calculator. Obviously, this number isn’t set in stone. Duration can vary based on a number of factors, but this tool offers a useful estimate. It will also tell you whether your test is doable or not.)
Now, you can make an informed decision about what you want to change.
- After your designated duration has ended, you’ll analyze the report to determine whether your hypothesis was correct and interpret the results. (“Adding the ‘Buy It Now’ button increased my single-order sales by 10%.”)
- Implement changes to your site based on the conclusions you have made (i.e. adding the button permanently).
- Now, pick a new thing to test, create a new hypothesis, and run another test (i.e. “I think that a yellow button will increase single-order sales more than a red button.”).
- Keep making continuous improvements!
