This article first appeared in IEEE Software magazine and is brought to you by InfoQ & IEEE Computer Society.
Few advances in computing and software have had such a visible impact on our world as the development of smart mobile devices. Consumers seem to have an inexhaustible appetite for being able to perform all kinds of tasks and services on the go, and the rate at which new apps are being rolled out is impressive. Anyone who’s traveled much recently will surely recognize the scenario of looking around any airport departure lounge and seeing fellow travelers using their mobile devices to not only take care of email but also to do shopping, pay bills, watch movies, and even apply for mortgages. (Rumor has it that some of these devices even make phone calls).
Human-Computer Interaction
How we got to this point has a lot to do with the ease of use that manufacturers and app developers have achieved, which includes aspects such as quick response time, intuitive interfaces, and well-designed functionality. To explore how this came about, I spoke recently with Ben Shneiderman and Ben Bederson, both of whom are former directors of the University of Maryland’s Human-Computer Interaction Lab (HCIL), the oldest center in the US focusing on HCI research. The role of the HCI field in building this world has been significant, although its successes in making interfaces so intuitive and transparent means that we don’t often think about them explicitly. In the words of Shneiderman, "HCI researchers take pride in having helped design mobile interfaces that enable 5 billion people to benefit from communication services that improve their lives, facilitate commerce, promote education, and bring medical care when needed".
The HCIL prides itself on working with a variety of stakeholders, including not only developers and mobile users, but also policymakers and other folks who don’t consider themselves to be technologists. Ben Bederson agreed to answer my questions about some key issues in contemporary mobile app development—starting with giving some background as to why designing good mobile interfaces is a very different endeavor than designing good desktop applications.
In his view, while there are many important ways in which developing for mobile devices really isn’t so far removed from developing for more "traditional" devices, there are also many differences. Mobile apps have to deal with inherently constrained resources— small screen real estate, small memory, lower computational abilities, limited bandwidth, and so on. But there are two important mitigating factors. The first is that, although resources are limited, the tasks for which users employ the devices are also typically commensurately small—for example, the weather or stock prices. These tend not to be complicated or multi-step activities, which wouldn’t work well for such limited devices.
The second mitigating factor is that mobile devices come with a lot of builtin sensors from which data can be easily accessed, meaning that developers can exploit a lot of information to improve the user experience: for example, depending on device capabilities, mobile apps might be able to detect the orientation in which the device is being held, remaining battery life, or the user’s location (via GPS). It’s also not hard to imagine an app capable of making educated guesses about the user’s environment based on images received through the device’s camera (for example, based on the texture of a wall in front of the device).
Requirements
Because user tasks on mobile devices tend to be relatively short and discrete, users typically aren’t willing to invest much time in figuring out how to do them. Bederson pointed out that while users of PC apps have been conditioned to expect a learning curve, the set of tasks that PC users expect to accomplish are typically much more complicated and naturally require an investment of time. For example, if I have just purchased a software product to help me with my taxes, I typically expect that I’m not going to completely master it just out of the box. But when user tasks are meant to take only a minute or two, overcoming a learning curve just doesn’t seem worth it—the user interaction must be very intuitive to make that task worthwhile.
These observations led Bederson to propose four important user interface (UI) principles, which are crucial for creating good user experiences. Mobile apps should be:
- universally usable,
- useful,
- efficient, and
- appealing.
It certainly appears that many developers are taking these to heart. Remembering the mobile phone that I had just five years ago—a clunky device with a clunky, hard-to-use textbased interface—I asked Bederson how UIs had gotten so much better in such a relatively short period of time. He attributed it partly to research into better UI technologies and partly to product testing. But he also felt that this was a good example of how so much of what consumers respond to actually comes from things other than the application interfaces. For example, compare th PalmPilot (an early mobile product that many users loved) and the Apple Newton (which wasn’t a commercial success): both devices had similar apps packaged with them, and actually employed similar interfaces, but the Newton made a lot of small design decisions that taken together actually made for a very different user experience.
The Future
When I pressed Bederson as to what revolutionary UI improvements we can expect in the next five years, he was somewhat cagey: "If I knew the answer to that, that’s where I would be working now!" He pointed out that although many mobile devices and apps have reached an apparently high level of intuitive usability, evolutionary improvements are certainly still possible. For example, the iPhone has a very hard-touse interface when it comes to making corrections within texts or emails that the user is composing. It’s a small issue, but one that users have to deal with multiple times per day (also, not coincidentally, one that’s being worked on and that Bederson expects will be fixed soon). Other areas of ongoing research include multiple devices, privacy and security, and voice.
Multiple Devices
Not only do many of us now have multiple devices that we use daily, often from different vendors, but we want to be able to move from one device to another seamlessly. We should be able to access our calendar from whichever device is handy at the moment, but often, when out of the home or office, we want to be able to pick up a friend’s machine and keep working. Today, users can synchronize their work across devices, but it’s not particularly convenient. Making this more of a seamless, easy-to-do task will be a potentially important area of progress.
Privacy and Security
Despite the fact that many users are becoming more attuned to privacy and security issues, there hasn’t been a huge amount of work done in this area. It’s not yet something that vendors use to distinguish themselves from the competition. But this is likely to be an area where we’ll see a lot of work and progress in the future; new approaches might take the form of mechanisms for safeguarding personal data in use across various apps or the development of features—such as a guest mode for mobile devices—that let users share physical hardware with friends without putting their data at risk.
Voice
Both Google and Apple’s latest devices have well-integrated voice input, and it’s finally starting to be used significantly. But while quality has improved tremendously in recent years, Bederson believes that this technology still has another increment or two to go before it catches on even more widely. When voice recognition works with background noise and speaker accents to the point where its reliability approaches that of typing, it will be much more broadly attractive. Similarly, when question-and-answer systems such as Apple’s Siri get connected to many more data sources and can respond to a much broader set of queries, you can expect to see much wider usage. However, even when these things happen, Bederson says that typing won’t go away. There are a number of reasons for that, not the least of which is that sometimes you want to enter data privately.
Other Issues
And of course, we should expect additional research into understanding and exploiting "design for touching" and better gestures—essentially, even more intuitive ways of interacting with our devices. There are some really innovative interfaces that are coming out and Bederson expects that will continue as well.
Challenges
Given the multitude of devices on the market, I’ve been curious about how much of a headache it is for app developers that dozens, if not hundreds, of different devices might be running their software. How do you test across this diverse space? Bederson described this as more of an engineering problem than a UI problem. Rather than worrying about dozens of devices, developers and testers typically collapse the problem to focus on the three to five major platforms in the marketplace (each of which relies on a different programming language). In the end, there’s no getting away from good engineering practice: like many software engineering problems, dealing with this issue requires solutions (like modularizing code and using a clean design that separates out the platform-specific code from the rest) that are easy to talk about and comprehend but require a lot of time and effort on the part of developers.
Finally, although good UIs are becoming increasingly ubiquitous, I asked Bederson whether his trained eye sees common problems that developers continue to get wrong. He said that one common problem area is applications that do too much or have too much complexity in the interface. A theme throughout this whole discussion has been the need to keep it simple—to not overload the small screen size, to allow users to very quickly get a concrete task done. Complexity in this context is a real turn-off for users.
Another problem Bederson pointed out is inconsistency across (or within) applications. No matter how hard developers try to keep it simple, inevitably some complexity in the resulting apps arises, and users have to manage this on a fairly small screen. Consistency in design and layout is something that can help users quickly master the complexity and intuitively manipulate the mobile device. Although obvious in theory, a countervailing trend is that many developers and vendors have been looking for novel and unexpected interface designs to differentiate themselves in a competitive marketplace.
Bederson closed with an age-old problem: insufficient attention to design principles. Good design is no longer optional. Users have high expectations!
Despite these difficulties, the ubiquity of easy-to-use and intuitive apps on mobile devices is a success story hiding in plain sight—an impressive technical accomplishment that too often we take for granted due to the good design of the apps themselves. I think it’s fitting to end with another statement from Ben Sneiderman to describe this work: "Mobile apps force designers to deeply understand user needs so as to provide just the right functions in a learnable and usable interface. These interfaces work across languages and cultures, for young and old, for experts and novices, and for users with diverse disabilities. It’s a striking accomplishment— a grand success that has transformed the world".
Forrest Shull is a senior scientist at the Fraunhofer Center for Experimental Software Engineering in Maryland, a nonprofit research and tech transfer organization, where he leads the Measurement and Knowledge Management Division. He is an associate adjunct professor at the University of Maryland College Park and editor in chief of IEEE Software. Contact him at fshull@computer.org.
This article first appeared in IEEE Software magazine. IEEE Software's mission is to build the community of leading and future software practitioners. The magazine delivers reliable, useful, leading-edge software development information to keep engineers and managers abreast of rapid technology change.