Performance: Planning Costs Less than Rearchitecting
Developers often say that one of their goals is to be “fast”. Yet when they release their customers still complain that it is slow and unresponsive. According to Microsoft’s internal research, the most common root cause of this is the lack of planning. Making an application “fast” isn’t a real goal because it can’t be measured. So as performance starts to slip, developers often don’t notice it. Or worse, performance is tested on the right kind of hardware or environmental conditions.
The pillars of performance are Fast, Fluid, and Efficient. But rather than looking at these as abstract concepts, you can discuss them in terms that lead to real plans with specific targets such as “load 30 images into the main view in 1 second or less”. This kind of performance planning allows for meaningful testing and early detection of performance problems.
One way to determine whether or not an application is “fast” is to view it in terms of “interaction classes”. An interaction class is used to describe a type of interaction and an acceptable time frame for the interaction to complete. Here are some examples used by Microsoft on their projects:
- Fast: 100 to 200 ms – Press a button, open a menu, bring up an app bar
- Typical: 300 to 500 ms – Resize, Semantic Zoom
- Responsive: 500 ms to 1 sec – Navigate to a different page
- Launch: 1 to 3 sec – Launching from a cold state
- Continuous: 500 ms to 5 sec – Downloading files
- Captive: 500 ms to 10 seconds – Somewhat long running operations. The user may switch to another app while waiting.
Complex interactions may need to be broken down into multiple time points. For example, navigating to a search result page can require three time points:
- First Response (Fast): The user see that something has started so they don’t keep pressing the search button
- Responsive: The textual information for the search results are visible and the UI is usable.
- Visible complete (Continuous): All content is loaded, including images.
Fluidity can be measured in terms of frames per second. For most applications, the recommended goal is known as “Buttery Smooth” or 60 fps. Applications that cannot maintain this level of fluidity should consider simplifying their screen.
Efficiency should be considered from the user’s perspective. They usually don’t care a lot if the CPU or network is running hot while the application is doing something they requested. But when the user is idle, so should the application. So a goal of all applications is zero ambient resource consumption.
Applications that consume more memory are more likely to be swapped out to disk or tombstoned, so another goal should be low memory usage. Since what’s considered to be acceptable varies widely by the type of application, there are no specific recommendations in this area.
Battery life and, for mobile devices, data plan usage are also major concerns for the end user. According to one study Microsoft commissioned, 50% of users cite heavy battery usage as a reason why they uninstall applications. Fortunately modern performance tools can not only measure the amount of I/O traffic, but also approximate the about of energy being consumed by running the application. Again, these
Planning and implanting rich instrumentation is necessary to truly understand the performance behavior of the application. Look into using sophisticated logging frameworks like Semantic Logging can be used to correlate the beginning and ending of specific operations.
A quiet system is key for testing; if the system is doing any work in the background it can throw off the test results. To this effect Will Sergeant of Microsoft recommends that you
- Turn off background applications. If you are using Windows 8, this means turning off your “Lock Screen Apps”.
- For managed code, generate the native code image. This can be done by running the application for 30 seconds or more, then running the Windows 8 System Maintenance tasks.
- Always capture multiple runs.
Always test on a variety of hardware and network topologies. A slow network can have a disastrous effect on the performance of an application’s performance, especially when it tries to download data on the UI thread. Screen sizes can also have significant effect on performance, as large screen needs to display more data at one time.
This information was originally presented in the Build 2013 session by the same name.
Ronny Kohavi Dec 12, 2013
Christian Legnitto Dec 12, 2013