Split Testing Software — What You Need To Know.
Split testing works. Use it well, and you will improve your conversion rates, and, thus, your profits. And, if you are advertising, or have significant product costs, raising your conversion rate just a little can raise your profits a lot. But there are many different kinds of split testing software out there.
I can’t discuss every feature of every testing tool out there, but I can give you a major overview of the different kinds of split testing software.
First, there’s a difference between multivariate and single variable testing.
Simple split testing is easy. You’re just “trying stuff” one thing at a time. You want to try a different headline? Split test it. When that test finishes, you want to “try” something else? Split test that. You can test one alternative version or many. As long as it’s a single element of your page you’re testing, that’s simple split testing. All you need is a tool that rotates the different versions of the headline (or whatever) on your page, and keeps track of which one produces the most sales.
Google offers a single variable split testing tool, but you have to have an Adwords account to use it.
But simple split testing can be discouraging. When you test a new idea against your control version, sometimes the control version wins. In fact, it often wins. You can get a string of disappointing tests all in a row. What would it be like if you could test 10 things at once? Well, you’d be bound to find at least one or two ideas that led to improvement. And what if you could test all those ideas simultaneously with the same amount of traffic it takes to test one idea with simple split testing? Well, it sounds like magic, but that’s exactly what multivariate testing can do for you.
Among multiple variable software solutions, there are hosted plans and self-install plans. If you want to install software on your server (which can be faster and cheaper) you will have more control over your data. And you can usually find someone to install it for you (often the vendor provides this service for a nominal fee). If you want to go with a hosted plan expect to pay more.
The Split Test Accelerator is a good choice if you want to install the software on your server. Kaizentrack is another optoin. If you want a hosted solution, you can look at Sitespect and Vertster.
The next distinction probably causes the most confusion among people looking into split testing for the first time. It’s also more important than the last distinction, becuase it determines how fast and accurate your tests can be. The three kinds of testers are 1) random independent rotators, 2) Taguchi testers, and 3) full-factorial testers. Taguchi ranks first for speed and second for accuracy. Basically Taguchi testing involves using a marvel of mathematics known as “orthogonal arrays”. These have special properties that allow you to test every combination in a space while running only a small fraction of the actual combinations. You have to be a little bit careful about interaction effects with Taguchi testing. The Split Test Accelerator and Kaizentrack are Taguchi testing tools.
When you run full factorial tests, as you would with Google’s Website Optimizer, you create a few factors, and a few options for each factor, and the software cycles through each combination. So, if you have 5 factors with 2 options each, you would cycle through 32 different pages. FFT can be used like a Taguchi test for small test designs. Or, if you run a lot more traffic to it, you can deal even more effectively with interaction effects by just letting all the interactions (good and bad) play out. Note that you can’t run as many combinations with full factorial tools as you can with a Taguchi too.
The Random independent rotators are a third option. These treat factors independently. So you can just plug in a new factor with as many options as you want at any time, and ignore the fact that other factors are being tested at the same time. This is the most flexible form of testing, but it is also plagued with the most interaction effects.
There are other distinctions and features worth noting. And each solution has a different mix of pros and cons. But this should get your oriented a bit before you make your decision.
Dominic DeLong has used taguchi testing tools for years to improve landing pages.
No related posts.
Related posts brought to you by Yet Another Related Posts Plugin.
