Swift Might Not Be As Fast As Apple Claims It To Be: First Benchmarks
[August 19, 2014 Update] As developer Jesse Squires reports, Xcode 6 beta 5 release has greatly improved Swift performance. "I've been running these trials since the first beta, and this is the first time that Swift has performed for every single algorithm, with standard optimizations", he says. Even better results are achieved by enabling the new
-ounchecked optimization, which has replaced
-ofast and made more explicit its real behaviour. [End update]
Performance is one of the benefits that Apple claims its new Swift programming language should bring to OS X and iOS developers, and being in beta hasn't prevented independent developers from running benchmarks and reporting their findings. Perhaps unsurprisingly these show that in some cases Swift performance is not yet satisfactory.
Developer Jukka Suomela wrote a post on Stack Overflow describing its findings when implementing an algorithm in Swift and noticing that it had very poor performance. Narrowing down his analysis, Jukka has been able to find out that the major bottleneck in his code came from a task as simple as sorting an array.
Indeed, sorting a 1 million random integers array took 6 seconds when using Swift, versus 0.06 seconds when using C++, and 0.6 seconds when using Python. Those results were obtained compiling with the
-O3 optimization level, which is commonly used by Xcode for release builds. When compiler optimizations where disabled altogether, corresponding to the
-O0 flag used by Xcode for debug builds, the above described test took 88 seconds, according to Jukka.
Jukka's findings where confirmed by other developers joining in the conversation on Stack Overflow. Developer sjeohp implemented the quicksort algorithm to find out that Swift was about 1000 times slower than C with no compiler optimizations enabled (
-Onone). On the other hand, he found that Swift was slightly faster than C when aggressive compiler optimizations (
-Ofast) were enforced. Similar findings were highlighted in a second post on Stack Overflow, describing an image processing test.
Aggressive optimizations, according to LLVM documentation, disregard strict standard compliance.
-Ofast enables all -O3 optimizations and additionally turns on
-ffast-math, which relaxes IEEE or ISO specifications for math functions, leading to possible incorrect output for programs that require the guarantees of those specifications. Furthermore,
-Ofast disables the checks for integer overflows and array indexing overflows, thus harshly reducing Swift safety features.
Going deeper into the analysis, Jukka inspected the assembly code generated by the compiler in a different test and found out that a simple loop over an array would generate a lot of memory management (retain and release) calls that were completely unnecessary. No math was involved in this test, showing thus that the main performance bottleneck likely came from those astray calls.
As noted in the opening paragraph Swift is still beta software, a point that several developers have already pointed out. This could be a good enough explanation of the current behaviour of Swift programs. Specifically, the described behaviour with unnecessary release/retain calls hints at some ARC optimizer bugs that could be fixed without requiring aggressive optimizations.
While the language is in beta, Apple will not allow developers to submit apps for review that have been built using Swift. The final Xcode release is expected next fall.
That is really silly...
Re: That is really silly...
sergio de simone
This post aimed at reporting on the discussion that early Swift benchmarking triggered among developers. I tried not to take any side and, towards the end of the post, I took care of noticing that Swift is just in beta and that could explain bad performance.
In any case, Swift has improved a lot in Xcode 6 beta 5 and I have added a short comment to the beginning of my post to highlight that and link to the relevant source.
I hope this will not contribute to labelling Swift in any way.
John Krewson, Steve Ropa and Matt Badgley Nov 24, 2014