Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Interviews Steve Levin on Challenges Developing for Multiple Mobile Platforms

Steve Levin on Challenges Developing for Multiple Mobile Platforms


1. My name is Ryan Slobojan and I'm here with Steve Levin, the VP of Operations and Mobile at scanR. Steve, can you tell us a little bit about what scanR does?

Sure, scanR, in its most basic sense, allows you to use your camera phone as a digital scanner. You can scan a picture of a document, a business card, a whiteboard, submit it into the service where we process it, take it from something that looks like a picture and will correct the color, the straightness, the angle - essentially do our best job to turn it into something that looks like it was scanned on a flatbed scanner or copier and then return it as a PDF or as a VCF content file, depending on if it's a business card or if it's a document.


2. When scanR first launched all of these services were accessed using an e-mail endpoint. So, I would take a picture and I would e-mail it to and that would do all the processing. Why did you go with that approach?

We went with that approach not as much for the idea that you would take a picture and send it in via e-mail, but that you would take the picture and send it in via MMS, which, in some markets, especially in all the North American markets, you can send an MMS to an e-mail address. Since most phones didn't really support having any kind of client on them, that was the only way we could get the images off of the phone short of having them on a memory card, going back to your PC, putting it into your PC, reading it off and sending it as an e-mail, which is a really long course that most people won't bother.


3. Why did you choose to do this via MMS as opposed to having clients on the assorted handsets at the time?

Initially it was a combination of 2 things. One is that in mobile phones, when we started this, which is now 5 years ago, the phones themselves weren't really powerful enough to run any kind of mobile client. Even just re-rendering a JPG, say to create a thumbnail on the screen, could be pretty torturous, 10 or 15 seconds on a lot of phones. I think the smart phone market, when we started and this doesn't even include smart phones that had good enough cameras in them to really present a usable image to scanR, that number was maybe 10%. At that time having a Treo or Blackberry wasn't that common.

As we've gone on, of course, smart phones have rocketed and I think it's up to 40% of the market and at least half of that is represented by phones that have more than 2 Megapixels and have auto-focus, which in practice is more important than resolution. High resolution can make up for auto-focus if you are doing things exactly right, just because you get enough pixels, but fundamentally we'd always rather have an in focus picture. With good focus we don't need a whole lot of pixels. We also have to do a lot less guessing on the image processing side.

Now, with Nokia, Blackberry and the iPhone as well as the Android coming out to market, you have a lot of handsets with good cameras and with both rich enough processing languages as well as good enough screens, a good enough interface that you can develop mobile clients.


4. Which of the platforms was the first one that you targeted and when did you target it?

The first platform that we went with was Symbian mainly because Nokia has a ton of Symbian devices in the market, tens of millions of devices with cameras that were good enough for us - 2 Megapixels and auto-focus. Not all of them are really smart phones. They may just have a regular T9 keyboard and so forth, but the Symbian OS allowed for a reasonably rich client experience. The downside of that platform became, because it's been around so long, it has all the cruft of an old operating system - it's very difficult, it's very hardware specific and the debugging tools are old, shall we say.

It's not uncommon to simply throw a number of errors all the way up the stack and all you get is a generic panic. If you are running the debugger, you can get a little more than that, but the problem is that no matter what you do as a developer, ultimately we're going to put it into QA, we're going to give it to people to try and those people don't have a debugger. What they get is a generic "Oh! It crashed!", but we actually still have a lot of traction in the Nokia market. Not a high number relative to the devices that Nokia has, but a high number of our users still use the Nokia devices. The cameras are excellent. They always have been and they continue to invest. That is really in some way Nokia's claim to fame, that their camera phones are truly useful as replacement compact cameras for many people.


5. Another platform that you've targeted is the iPhone platform. Can you describe a little bit what some of the good and bad things were that you ran into with that platform?

The iPhone is a great platform and ultimately the advantages of it are that it's the newest in both the execution and the OS. It's by far the most modern and well thought out of the platforms. That means you can do all of the things you would expect fairly simply. You are not building thumbnail generation, you are not building communication libraries, you are not having to really worry about chunks sizes and network stacks, it's all very neatly put together.

You can quickly build up a very rich application and spend more time working on the logic of a complex application rather than the actual fundamentals of "What do I need to do with this bit?" and "How do I move this bit from here to there?". We've spent a lot of time integrating things like JPG handling libraries into other platforms - that's just been a given with the iPhone. The disadvantages, of course, are you have Apple's black box of an approval process and you have Apple's guidelines to interface development and "guidelines" is an old term that really means "This is how you'll do it or else!"

But within those rules it's been a great platform to develop on. Apple really enforces that at a hardware level things just work and that's been a huge advantage in mobile development. We spend our time building features, building the app itself, building our UI and functionality. We don't spend time chasing hardware and capabilities for the most part.


6. The Android platform is another one that, if I recall correctly, you've been working on. Can you tell us a little bit more about that?

Sure, Android looks to be a really great platform. It has - I would say - the best networking stack in any of the phones that I've seen. Again, it's a big platform that's been stripped down to run on a fairly capable processor, when you think about it. All Android phones, much like the iPhone, have the processing horse power in many ways that desktop machines had 10 years ago, so that freedom means that you can do a lot of things right. The downsides of Android that we've seen have really been that it doesn't provide a really rich UI set.

It's a functional UI, it gets things done, but if you want to make a pretty application or you want to step outside of the default controls, you do get back into a lot of investment and custom controls and custom images for the UI. While that's not necessarily technically difficult, it's definitely resource intensive from just the time of building pixel after pixel, whatever it is that you want. What you want this button to be there is not a lot of choices yet. I think as it matures, we will see that you get richer and richer features.

That's one of the reasons that some of the non-Android mobile Linux players in the market have gone different directions as far as toolkits is addressing that specific issue of trying to present more than just basic UI elements. I know that at least 2 of the main handset makers are going with QT for example, as a UI widget set and it does more than that. The main reason is that they wanted a much richer interface without saying "Build it yourself!". I think Android is really exciting. It has a lot of potential.

That said, the devil still remains in the details of "Do the hardware manufacturers, implement everything properly as we go along." Will camera APIs, which obviously at scanR we really care about the ability to invoke the camera and to invoke it automatically at either a high resolution or the maximum resolution and with auto-focus on? While it sounds like "Oh, gee, that shouldn't be a hard thing for the manufacturer to make sure it works, on other platforms we've experienced that being able to do that invocation has ranged from "It mostly works" to the "It almost never works".


7. What are some of the other platforms which you are targeting?

Blackberry is definitely the biggest platforms for us, both as far as installed user base and the demographics to go with it as well as the fact that they remain - depending by whose number it is - either the number one or the number two smart phone manufacturer. When you couple that with the demographics that I mentioned, it's probably 5 times more important than the iPhone is today, because there are lots of people who get iPhones, because they want to check their mail, sure, but because they want a music player and a movie player that is also a phone, so that they are not carrying multiple devices.

There is nothing wrong with that, but those people tend to scan fewer business cards and fax fewer signature pages than your typical Blackberry user who is still by and large a business user. I can't say that I don't know people who haven't just bought Blackberries for personal use, but it's a much smaller number than people who have Blackberry supplied to them by the company.


8. What was your experience targeting the Blackberry platform with the scanR application?

It's generally been good, especially given the age of the Blackberry platform. On the UI side it's not terribly bad, some of the networking stack area is, I'd say arguably, the weakest point of the OS. Many of those compromises come from the fact that Blackberry OS is still an evolution of the original OS and that was very low powered and very focused on getting e-mail in. You would read a lot more e-mail and you would want to get it quickly, receive stuff rather than send much data. Relatively speaking, sending an image is not a hugely intensive bandwidth task, but compared to typical e-mail responses that one might send, one picture probably covers 500 e-mails for the average user.


9. What about the Windows mobile platform?

Windows mobile was the first smart phone platform that we actually developed for, even before we had a working Symbian client, we had a working uploader for Windows mobile. It was originally a pure uploader - that is you took a picture and you keyed-in your e-mail address and then it sent over HTTP the image, which we then processed and sent to that e-mail address rather than doing an MMS. That was released and it was a part of the whole Palm Treo 700, publicity era.

At that time, because of the fixed focus and low resolution of the camera it was strictly marketed at whiteboards, where you don't need the resolution because the density of hand-written data is pretty low. We've continued to build the Windows mobile application. We have gone on and on and we have one today. In fact, it is preloaded on some devices that are sold in Japan. The downside that we've hit with Windows mobile is that the ability to produce a generic application was just beyond us. That goes to the camera API that I mentioned before. It is an area that manufacturers simply have not focused on and is not an area that Microsoft mandates works.

It's an optional implementation for the camera API. Your camera functionality has to work per their guidelines, but programmatic activation of it is not a requirement. What we discovered is that in many cases the APIs are either incomplete or don't work at all. We've been forced, with Windows mobile, to approach it from the standpoint of per phone deals essentially.

When we had manufacturers in Japan that had a specific phone and a specific ROM that we built to, we successfully got a tremendous amount of use of those applications. But the idea of releasing a generic application through the Windows marketplace is impractical, because the only way to effectively use the app is to take a picture in the camera app, then start our application, go into your image chooser and choose an image and then send it. It's more complicated than the average person wants.


10. One of the things which happened in the last year and a half in the mobile development space was Apple's introduction of the App Store concept and the idea of App Store seemed to pretty rapidly be adopted by the platforms. What are your thoughts on that? How did the App Store idea change the game for mobile development?

I think it really changed the game because it's made distribution the most important thing in selecting what mobile platforms you focus on rather than having to try and work within the whims of carriers, for example, who are choosing phones not because they are too concerned about scanR or even any mobile application. There are a lot of other decisions, a lot of other criteria in their decisions for them to choose phones and you have to bounce around and work within those constraints, which you won't even have the input or knowledge of, you just get the end result "These are the phones we're supporting" and you don't even know how long they'll support them for.

They come out, they may have committed to a particular run, it may be large or small, and then they stop selling it, they move on to the next thing because they are purely retail driven and they want the shiniest newest things in the stores all the time. App Stores of any form give you the ability to market directly the phones and give you an ability to get a lot, to get visibility to your application that's really difficult in the current carrier models. For example, many of their App Stores are WAP driven. There is no way to advertise online through any of the mobile advertising vendors and have someone click on a link that, somehow, ultimately opens the ability to buy that app on your phone.

With iTunes, and with a couple of the other stores, you can do that. We can send someone to a landing page of ours that talks about how great the application is, that it allows you to scan, copy and fax with your iPhone and then you can click on a link in your iPhone browser that will open iTunes right there and you can buy it. You're done! It automatically installs, it does all the Apple goodness. Cutting out all these other middle steps and being able to directly market is orders of magnitude more efficient than selling an application.


11. Compared to the Apple App Store process, what is the App Store like for some of the other platforms?

Apple's Store is by far the most fleshed out, as we know, mainly because iTunes existed as a purchasing marketplace even before the iPhone, so, they are years ahead. The other stores are in their infancy. In the case of Blackberry it currently runs a special client rather than being an HTTP-based store, for example. That presents its own barriers to entry. You do have to get people on a Blackberry to download and install the app to run their store before they can access it.

Once they are over that and as they get catalogue in their store, they get more and more users, then it's a very clean interface from introduction to sale, but you do have that initial step which is difficult. Nokia has a store which again requires an application to run. It has the same hurdles of getting the installer to the installer installed. The US carriers, I think, are by far the biggest laggards in marketing directly.

That said, all 3 major carriers have large scale initiatives that are moving towards HTTP based stores. They are moving to make it very simple, so that they become much more of a billing platform and they are also talking about taking on much more content than they ever did before, for the same reason that it's been successful for Apple - you don't just want to sell applications. If that's all you sell, you only get people who are interested in applications.

If you sell applications and music and ring tones and who knows what - movies - you get people more engaged. They see more value in taking the time to sign up or just bother looking. That whole mind share of "Will I even go look?" is important. Android is up and coming. The physical store itself is a little rough around the edges and the fact that there is no PC side browser is still a limitation.

People certainly access it on their phones and there is a solid argument that most applications are bought directly to phones, but what that number doesn't factor in, in my experience, is that people will use a PC to find out about an application and then turn around and use their phone to go search for the specific item and buy it. I think if you looked at the numbers of people who search and buy entirely on the handset, you'd find that that number is not small, but it's nowhere near the 90% that has been considered as purchases directly to phones.


12. When developing the scanR application, I've noticed that you targeted each platform's SDK individually. There are also platforms available which will target multiple handsets at the same time with a single code base. What was the reasoning behind choosing the individual platforms versus going for one of these more general solutions?

It really boils down to 2 areas. One is in application performance and one is in hardware compatibility. When it comes to performance, you are looking at everything from BREW onto various WebKit-based items. Usually what trips them up is that they don't support multithreading or the performance, especially around large mathematical operations, which is essentially image handling, which is what we do, are very poor. The sacrifices, especially around the lack of multithreading, you end up with an unsatisfactory application.

The hardware issue is one of those things that everybody says will be solved or doesn't happen, but our experience has been that even within fairly tight manufacturers running BREW, for example, there was a lot of porting - I would say that less than 50% of the phones that we developed our BREW application in Japan on -- and I picked that mainly because the Japanese market is both much more mature around it and has used BREW even longer than the US carriers -- So that number, the fact that you couldn't cover half of the phones with a single build in the end, speaks volumes to what would happen in the much less mature and more rapidly evolving markets in Europe and the US.

Certainly, as a company we'd love to develop a single code base. If we could find something that would give the performance or even come close on the performance and cover a large number of phones, we'd certainly release it, even if we specifically targeted what we might call the Big 4 platforms with specific releases.

There are still plenty of phones out there that don't fit into those models - whether it be Windows Mobile, or even some of the new mobile Linux platforms coming out. Having a generic thing that just works, if performance was the only limitation, we'd certainly add it as a 5th major platform, but the truth is you always get so close, but in consumer level devices, the bar is much higher than in server-side implementations of anything. It really has to just work at a four-nines level, where we might accept a two-nines level in server-side stuff.


13. The App Store model that Apple has imposes certain restrictions upon your development. What kinds of challenges have you faced creating a business application within those restrictions?

It's a good question. Apple, as well as the carriers, have operated on a pretty narrow set of assumptions for pricing. That is either you bought something one time and that was essentially the Apple model. You bought music, you bought a TV show and the carriers were used to you bought one time a game or you paid monthly for it and those were the 2 options. scanR, and I think an entire new breed of applications, are very high value, but infrequent use. When you need to send a signature page, you need to send it.

In truth, we could charge $15 for you to take a picture and scan and then fax a single signature page, if you were away from home and needed to get it done. A friend of mine always talks about, that he used it -- he was selling a house and they needed a signature that they had forgotten to get and he was 800 miles away and it made it much easier to use it, and he would have paid anything at that point. But, in general, those models break down because people don't want to pay $5 a month and they don't want to pay a lot of money upfront. Apple has just come out with their in-app purchasing, which gives you the ability to buy stuff in the application in small chunks. The idea that you can buy it as you need it.

The other markets - both Blackberry and Android - are moving in that direction and have said that they will have it some time in 2010. I think that's exciting not only for scanR, but just in mobile development in general because it opens up all of those possibilities - anything from games where you can buy stuff like "I'm playing this war game and I need a tank and I'm not willing to play for 6 months to get the tank. I'll pay $5 to have a tank so I can play with my friends" to the scanR model of you need to send a fax and maybe you want to send a 10 page fax. OK, will you pay $3 or $4 for that right now and you can send it? And you haven't committed to the ongoing monthly bills. That's a sensitivity that has gone up during the last couple of years. People are going "I don't want to be committed to this stuff forever! I just want to buy it as I need it." I think that's going to open up a lot of things, not just for scanR but in mobile applications in general - getting away from the monolithic models that we've had for so long.

Apr 18, 2010