BT

The Current and Future Performance of the Mobile Web

by Zef Hemel on Jul 12, 2013 |

In a substantial, well-researched article, Drew Crawford, owner of an iOS development company, lays out all the reasons why he believes mobile web applications are slow today and why he does not expect this dramatically improve in the near feature. The blog post is a follow-up to a previous post in which he pointed out that JavaScript performance on mobile is an order of magnitude slower than on desktops. This post was heavily criticized, so in response Drew wrote this much more elaborate article. The reasons for the poor performance and lack of improvement thereof fall into three categories:

  1. Processor speed of mobile ARM processors versus desktop x86 processors
  2. Performance trends of JavaScript engines
  3. Memory consumption in particular related to garbage collection

The two bottlenecks that Drew describes are CPU and memory. CPU-boundedness has two aspects: the power of your CPU and efficiency of execution. Drew points out that current generation x86 processors are ten times faster than current generation ARM processors which power today's mobile devices such as the iPhone and high-end Android devices.

I am not a hardware engineer, but I once worked for a major semiconductor company, and the people there tell me that these days performance is mostly a function of your process (e.g., the thing they measure in “nanometers”).   The iPhone 5′s impressive performance is due in no small part to a process shrink from 45nm to 32nm — a reduction of about a third.  But to do it again, Apple would have to shrink to a 22nm process.

Knowledge and investments in reducing the size of transistors is mostly in the hands of Intel. Drew claims it's not likely that ARM will catch in the foreseeable future. In fact, it's far more likely that Intel will produce a x86 processor that competes power-consumption wise with ARM than that ARM will close the performance gap.

The second aspect of performance is efficiency. How efficiently are the CPU cycles used, i.e. how efficient can the machine instructions get that are generated from JavaScript code?

Here is where a lot of competent software engineers stumble. The thought process goes like this–JavaScript has gotten faster! It will continue to get faster!

The first part is true. JavaScript has gotten a lot faster. But we’re now at Peak JavaScript. It doesn’t get much faster from here.

Here’s Chrome v8 on my Mac (the earliest one that still ran, Dec 2010.) Now here’s v26.

Can’t spot the difference? That’s because there isn’t one. Nothing terribly important has happened to CPU-bound JavaScript lately.

If the web feels faster to you than it did in 2010, that is probably because you’re running a faster computer, but it has nothing to do with improvements to Chrome.

The reason why JavaScript performance hasn't improved significantly is clear, according to Drew:

The thing is, JITing JavaScript was a 60-year old idea with 60 years of research, and literally thousands of implementations for every conceivable programming language demonstrating that it was a good idea.  But now that we’ve done it, we’ve run out of 60-year-old ideas.  That’s all, folks.  Show’s over.  Maybe we can grow another good idea in the next 60 years.

The second limitation of the mobile web is memory. Once again, there are two aspects to memory usage: the amount available and efficiency of usage.

While modern mobile devices have a fair amount of memory (usually 512MB or 1GB of memory), the operating system does not allow the application to use that much. A lot of memory is used by the operating system itself as well as other running applications (multi tasking):

[...] essentially on the iPhone 4S, you start getting warned around 40MB and you get killed around 213MB.  On the iPad 3, you get warned around 400MB and you get killed around 550MB.

Drew notes that at the iPhone 4S' resolution, a single picture taken with the camera takes up 30MB of bitmap data. That means there is space available for 7 photos in RAM before the operating system kills the application because it runs out of memory. Therefore, especially if an application handles media like pictures and video, it has to be extremely careful in what it keeps in memory and for how long, as memory is extremely limited.

The second aspect of memory is efficiency. JavaScript is a garbage collected language. The developer doesn't have to do manual memory management, which is designed to make life easier for the developer. However, garbage collection comes at a cost. A cost that increases exponentially in memory-constrained environments:

What this chart says is “As long as you have about 6 times as much memory as you really need, you’re fine.  But woe betide you if you have less than 4x the required memory.”

The ground truth is that in a memory constrained environment garbage collection performance degrades exponentially.  If you write Python or Ruby or JS that runs on desktop computers, it’s possible that your entire experience is in the right hand of the chart, and you can go your whole life without ever experiencing a slow garbage collector.  Spend some time on the left side of the chart and see what the rest of us deal with.

This behavior could be the reason why Apple has never supported Objective-C's garbage collector on iOS, and is replacing it with ARC (both on iOS and the Mac), which is not a garbage collector.

While Drew makes interesting points in his article, as pointed out by Brendan Eich in a tweet, not all applications are CPU or memory bound. It is only a certain category of applications that hit these problems, for instance games and multi-media applications. Nevertheless, Drew's article (10,000 words long) is worth a read for anybody interested in performance on the mobile web.

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Good Objective View by Faisal Waris

I skimmed through the original article and found it interesting and objectively written.


I wrote a simple 3D flying simulator type game using XNA and F# on WP7 (wp.me/p1buGO-3R). The performance is actually quite good even on my 2 year single core WP7. Many other XNA 3D games run very well on the same phone.

One thing to note is that the 'garbage generation signature' of functional langauges is different because immutability is the default. Some of this difference is seen in the Google paper on performance comparison between Scala and Java. See multi-language-bench.googlecode.com/svn/trunk/d...)

Fair assessment, but depends on the application by Russell Leggett

There's no denying that JavaScript still has a long way to go when compared to native for many types of programs/algorithms, and it is unlikely to ever come within a factor of two for the general case. The case against garbage collection is also fair, again, for certain types of applications. However, I think where it falls short, is the bigger picture.

To begin my own thinking, I would point out Node.js as an interesting study. I am trying to remember the details of the quote, but I believe I remember someone saying something along the lines of: "Don't think of Node as JavaScript - think of it as a JavaScript layer on top of some great low level C code". JavaScript has always been a language that has been used to orchestrate lower level pieces. It's true that more and more business logic is being done in JavaScript, and that's ok, but the point is that have always been, and likely will always be, ways of offloading some of the most resource intensive work to another more "native" technology.

Working through the DOM, CSS, and WebGL allows for deferring a lot of computation to native code, and even hardware acceleration. I see asm.js in a similar vein, only now developers are able to control what is available instead of just relying on the browser to implement it. You can call it cheating, but its not so different from the way apple uses Objective-C to interface and orchestrate C code. And keep in consideration that most dynamic languages commonly use FFI to talk to native libraries for things like image manipulation.

I think all of the points listed in the article have been well thought out, with good evidence behind them, but its making a case against JavaScript when it should be thinking about the full web stack instead. If all web pages were pure JavaScript writing out to the canvas tag, I would agree with the article completely, but that's just not the situation. Pragmatic choices are being made to evolve the language and environment to something which can compete.

"Never bet against the open web"

Did you notice what the benchmark actually compares? by Mirko Stocker

An interesting article, but it doesn't look like the author understood Hertz and Berger's paper he quoted. On first glance, it seems that it benchmarks GCs against traditional non-GC memory management, like reference counting. But that's not true, the benchmark compares GC to memory management with an oracle:
first executes the Java program to calculate object lifetimes and generate the program heap trace. The system processes the program heap trace uses the Merlin algorithm to compute object reachability times and generate the reachability-based oracle. The lifetime-based oracle comes directly from the lifetimes computed during the profiling run.

So this is not something that could be done when running your regular programs. And is says nothing about the performance of reference counting.

This Angers Me by Stephen Green

You presented no new information, opinions, or value to the original article. Please do not simply re-post other people's work.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

4 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT