Slide Down

Thank You

Now find out how we make great projects, check our blog and meet our team.

The Beginner’s Guide to A/B Testing


Ever feel like you hit a brick wall when it comes to designing your perfect website? It seems that even after conducting usability tests, you just can’t quite put your finger on what it is that isn’t getting you enough subscribers or users.

Maybe it isn’t usability or user experience that’s causing the problem, what if it’s the layout, the images you’re using or the pricing on your website?

But how do you test that? A not-so-secret weapon online business pros use is A/B Testing.


What’s A/B Testing?


In short, A/B Testing is making two versions of one element in your design and seeing which one gets you better results.


For example, it’s like testing to see if the ‘SUBSCRIBE’ button written in upper-case letters gets you more subscribers than the ‘subscribe’ button written in lower-case letters. If you find out that the ‘SUBSCRIBE’ button gets you more clicks than the other ‘subscribe’ button, then you’ll simply go for the better option.


The way it works is that you make two versions of one design, one is the design you’re already using, which is design ‘A’, and the new design you want to test, which is design ‘B’. You run both of them at the same time, then you divide your live traffic between the two and see where both versions take you. After a sufficient amount of time, you compare the analyses of the two versions and see which one gets you more clicks or subscribers.


Alright, so now you know the basic idea behind A/B Testing, but what good will this do for your business?


The first obvious benefit of A/B Testing is that it will help you figure out the ideal marketing strategy for your product or company. Also, you’ll have a clearer understanding of your target market and what catches their attention the most. With that said, it’ll become easier for you to make decisions regarding design and you won’t even have to take it out for a test run because you’ll know what gets your users going… or buying in this case.


Now the question remains, how do you go about A/B Testing without messing it up?


So You Want to Run an Effective A/B Test? Ask Yourself These 4 Important Questions…


Q1: What’s Your Plan?

As tempting as it is to just start testing right away, you need to make sure you have a plan worked out. How will you know what ‘good’ results are to you when you don’t even know what you want exactly? Therefore, it’s crucial that you have a clear idea of what kind of results you’re looking for before you start. To do that, you need to figure out what kind of results you’re getting now, what kind of results you want to see and list the objectives of the test.


Another important thing you should look into, is what A/B Testing tool suits your needs the most. There are many testing tools out there for you to check out, some are even for free; there’s Google Optimize, CrazyEgg, Optimizely, etc. Don’t be afraid to explore your options!


Q2: What Do You Want to Test Exactly?

This question should be answered based on the list of objectives you made during ‘planning phase’. You first need to identify whether you’re planning on running an off-site or on-site test. If you want to test an ad design or marketing copy or anything that happens anywhere outside your website, then that’s off-site testing. However, if you’re testing the headlines or any kind of graphic you use to direct your user on your website, then that’s on-site testing.


Once you figure that out, make a list of the variables you want to test. For example, if you want to test the headline on your website, you would probably want to test font color, font style and alignment.


Q3: Are You Ready to Keep a Steady Schedule?

That’s right, you need to make sure you’re keeping a controlled testing schedule for your website. Be consistent and always focus on your main objective. Avoid jumping from one element to another; don’t go running tests on one of your ad campaigns and then randomly start testing one of the headlines on your site, this will only confuse you and derail your progress.


Q4: So How Long is This Going to Take to Do?

This highly depends on the amount of traffic you get, so one test may take a few days while another could take weeks. This is why it’s important to set a time limit for your tests, this can be tricky though, because giving a test insufficient time could lead to skewed results, because you didn’t get enough people for the test to be statistically accurate. Same goes to running tests for too long. Always know when to draw the line, so set a minimum and maximum duration, number of clicks, bounce rate, sales etc.




One thing worth mentioning is that it’s easy to take things at face value and just pick the version that works best for you, but remember, analytics is your friend, study it thoroughly and carefully. Otherwise you’ll end up eliminating certain aspects of the design that were actually working for you. You’ll find yourself on an endless loop, not knowing what your audience wants or where the problem is anymore. For effective A/B Testing, devise a plan and keep your goals in sight.

Join the discussion

No Comments


Let's build your idea and make it the world's next big thing!

We are designers, researchers, information architects and software engineers with a successful record in building successful applications. Let’s talk and see how can we be the best help to build your product!

Ready to start your project?