A/B Testing for Mobile Applications

by Ralph Winzinger on Nov 29, 2013 | offers support for A/B-testing mobile applications without the need for code changes and resubmitting to an app store. Currently, there is only a native library for the iOS platform but libraries for the Android eco-system are already in development and will be available soon.

A/B-testing is often used in marketing and advertising to optimize the click-through-rate of customers. Therefore, for example a web site is published in two versions (A and B), that differ in one certain aspect. Test users are divided into two groups, each using one version of the site. At the end of the test, the conversion rates for both versions of the site are compared to measure which version was more successful. The winning version will then be published for all users.

After signing up, lets developers of mobile apps administrate different applications online. For each application, there can be an arbitrary number of experiments and within an experiment an arbitrary number of variations. An experiment represents the component in the application which needs to be tested, a variation is one of the different versions of the component.

Integration of with an iOS application is done via a CocoaPod library. The different variations for the experiment are declared as blocks, the Objective-C construct for a lambda or closure. Which block actually gets executed is chosen by the library depending on the administered settings for the experiment.

During the test, allows to monitor user behavior in real-time on a dashboard. The weights for the variations can be edited during runtime and eventually, the desired variation of the experiment can be committed. After committing a variation, the experiment has finished and all users will then use the chosen variation. Since selects the variations to be executed, the application code does not need to be changed for adjusting variation weights or committing a variation and there is no need for resubmitting the app to the app store.

Pricing depends on the number of monthly users of the tested application. Up to 200 monthly users are free, 2.000 are $19, 20.000 are $79 and 200.000 users are $399 per month. All plans include multi-variant testing and an unlimited number of experiments.

Rate this Article


Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

General Feedback
Marketing and all content copyright © 2006-2016 C4Media Inc. hosted at Contegix, the best ISP we've ever worked with.
Privacy policy

We notice you're using an ad blocker

We understand why you use ad blockers. However to keep InfoQ free we need your support. InfoQ will not provide your data to third parties without individual opt-in consent. We only work with advertisers relevant to our readers. Please consider whitelisting us.