BT
x Your opinion matters! Please fill in the InfoQ Survey about your reading habits!

A/B Testing for Mobile Applications

by Ralph Winzinger on Nov 29, 2013 |

Best.ly offers support for A/B-testing mobile applications without the need for code changes and resubmitting to an app store. Currently, there is only a native library for the iOS platform but libraries for the Android eco-system are already in development and will be available soon.

A/B-testing is often used in marketing and advertising to optimize the click-through-rate of customers. Therefore, for example a web site is published in two versions (A and B), that differ in one certain aspect. Test users are divided into two groups, each using one version of the site. At the end of the test, the conversion rates for both versions of the site are compared to measure which version was more successful. The winning version will then be published for all users.

After signing up, Best.ly lets developers of mobile apps administrate different applications online. For each application, there can be an arbitrary number of experiments and within an experiment an arbitrary number of variations. An experiment represents the component in the application which needs to be tested, a variation is one of the different versions of the component.

Integration of Best.ly with an iOS application is done via a CocoaPod library. The different variations for the experiment are declared as blocks, the Objective-C construct for a lambda or closure. Which block actually gets executed is chosen by the Best.ly library depending on the administered settings for the experiment.

During the test, Best.ly allows to monitor user behavior in real-time on a dashboard. The weights for the variations can be edited during runtime and eventually, the desired variation of the experiment can be committed. After committing a variation, the experiment has finished and all users will then use the chosen variation. Since Best.ly selects the variations to be executed, the application code does not need to be changed for adjusting variation weights or committing a variation and there is no need for resubmitting the app to the app store.

Pricing depends on the number of monthly users of the tested application. Up to 200 monthly users are free, 2.000 are $19, 20.000 are $79 and 200.000 users are $399 per month. All plans include multi-variant testing and an unlimited number of experiments.

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT