If you’ve been reading StartupStud.com for long, then you already know that smarter wins the race every time. Our goal with any online business that you’re involved in is to always have the most intelligence possible and then to execute your business with flawless precision based on datanot your gut feeling.

This is where A/B and Multivariate Testing can come in and help you to optimize your site and substantially increase conversions from the traffic that you already have. In the simplest of terms, this testing is basically taking the traffic to your site and then using a recommended testing service to divide it between different versions of a minor (or major) change. It can be changing a headline, color, image, product title, sign up form or anything else you can think of comparing.

You are A/B Testing (aka: Split Testing) when simultaneously comparing one (or more) versions of a single change to your project.

You are Multivariate Testing when simultaneously comparing combinations of multiple changes to your project.

And while I’m the first to admit that all of this sounds boring enough to make a dead person fall asleep, the potential benefits from just taking the time to regularly test and optimize your projects are a little more exciting.

Imagine if you knew that by merely changing the title of an e-commerce product, you would get 10% more sales out of every 1,000 visitors. Or increase newsletter signups 200% by simply changing your mailing service. Or by optimizing your site for mobile users, you would immediately increase profits by $5,000 per week. For obvious reasons, if you like money, then this is worth doing properly.


Be a yardstick of quality. Some people aren’t used to an environment where excellence is expected.

– Steve Jobs

There are dozens of A/B and Multivariate Testing services available, which you are welcome to further research on Google, but I’ve kindly done the legwork for you and narrowed down recommendations to the seven best options that are suited for a small-to-medium online business to use with varying levels of marketing expertise.

In other words, these are the ones that are affordable, powerful and relatively easy to learn to use:

  • Marketizator: Marketizator tops my recommendation list because they offer a powerful and simple to use testing interface that is free up to 10,000 tested views per month and also offers surveys and personalization. Monthly plans range from $55 and beyond.
  • Optimizely: As one of the most popular options available, Optimizely offers a stunningly simple interface that is jam-packed with testing and data analysis features. Free trial available and monthly plans start at $17.
  • Convert: With a two week free trial and available self-service plans starting at $26 per month, Convert‘s advantage is an impressive list of simple integrations with other platforms and services ranging from WordPress, Volusion, Google Analytics to even a few of the other more specific recommendations like Kissmetrics and Unbounce.
  • Kissmetrics: If you are serious about analyzing data, Kissmetrics is one of the best available options for tracing your testing data down to specific users. Their offerings are impressive to say the least. And with packages starting at $150 with a 1-year commitment, it’s a good thing.
  • VWO: Visual Website Optimizer is an excellent blend of powerful features with a still reasonable monthly service fee. At $49 a month, VWO offers A/B, Multivariate, Split URL, Targeting, Usability Tests, Conversion Tracking and Heatmaps, whereas many other services pack those into much more expensive packages.
  • Unbounce.com: Unbounce is your go to if you’re ONLY focusing on split testing landing pages. That’s all they offer and they are pretty damned good at it. A 30-day trial is available with packages afterward ranging from $49 – $199 a month.
  • Google Analytics: I’m a major fan of all things Google and use Google Analytics on a daily basis for every project I’m involved in. But when you get into some of the more advanced features, like their “Content Experiments” (their A/B Testing platform), you’re faced with a steep learning curve and some less than ideal visual testing features. I’d suggest at least playing with the free and trial options on the other services to get your feet wet on testing before delving too deeply into Google’s offerings. Also, watch as many training videos as you can find, Google’s interface is up there with the NASA Control Room in terms of complexity.


Risk comes from not knowing what you’re doing. 

– Warren Buffet

What Is Required To Test Properly?

It’s important to always start with a specific hypothesis that says, “by changing [X], I think [Y] will happen.” Then apply enough traffic for the testing to reliably produce a statistically significant result over a given amount of time.

Your goal is to be 95% sure that “[C]” change will produce “[R]” result, so it’s wise to start with a testing calculator that gives you a ballpark of how many visitors you need to test with to get to that magic 95%. Here is a good one to use. Another great option is to plug in your current traffic and goals to get a minimum amount of days you’ll need to test (as in, at 1,000 visitors per day, i’ll need 15 days worth of testing). Check that out here.

What Should I Test?

You can test anything that you’re wondering about, but I recommend starting with these essentials:

  • Calls-to-Action: This is usually one of the more important (and obvious) features of your site to test as it has one of the most direct effects on your conversions. Just be crystal clear what you are testing. Smart tests to run are: where the CTA is on the page, what the CTA looks like, verbiage of the CTA, shape and style, etc.
  • Copy Length: Look for that sweet spot of copy length that gives the reader everything that they need to know without scaring them off. This is especially important for sales and sales landing pages.
  • Headlines: The reader’s experience generally starts with the headline, so this is an important one to become proficient at knowing what your audience likes.
  • Images: Test a variety of pictures to find what best captures your reader’s eye and inspires their desired action.
  • Mobile: The vast majority of your site users are now viewing your pages on a tiny screen, so testing with the mobile experience in mind is essential.

Can I spy test Results from similar companies?

If there is one inalienable truth about the internet, it is that you can find damn near anything on it – including the results of other websites A/B Testing! While taking all of the obvious differences and variables into consideration, it is worth peaking at other’s results for testing ideas and insight into how their tests faired: WhichTestWon.com.

How Can I Become More Proficient At A/B and Multivariate Testing?

There aren’t yet too many quality online courses out there to learn advanced data analysis strategies (if you find some good ones, let me know), but each of the major recommended testing services at the start of this article offer fairly extensive knowledge bases and blogs on the subject. You can also peruse the internet marketing forums. Online, that’s your best bet.

Offline, however, there are a few great books available on Amazon.com for it: