Dynamic Language Roundup: Python's GIL Gets Overhauled but not Removed, Squeak Comes to Android

Posted by Werner Schuster on Jan 24, 2010

Sections

Operations & Infrastructure,

Enterprise Architecture,

Development,

Architecture & Design

Topics

Linux ,

Architecture ,

Performance & Scalability ,

Ruby ,

Platforms ,

Runtimes

Tags

Threading ,

Performance Tuning ,

Python ,

SmallTalk ,

Concurrency ,

Virtual Machines

It's easy to annoy a Python developer: complain about significant whitespace in the language or point out the Global Interpreter Lock (GIL) which restricts the interpreter to running only one Python thread at a time.

With the advent of multicore systems, the GIL has come under even more fire.  Dave Beazley's talk on the GIL explains the problems in detail (PDF Warning). It shows how the implementation of the GIL manages to slow down multithreaded Python programs due to bad interactions between the Python GIL and OS thread scheduling.
In short, the Python interpreter runs a thread for a certain amount of ticks (executed bytecode instructions); then it signals a condition variable that wakes up other threads which now have a chance to grab the GIL. On multicore or SMP systems, the original thread keeps running and also competes for the GIL - which poses a problem: the threads waiting for the condition variable's signal needs to be woken up before they can grab the GIL. Waking up a thread takes a bit of time - and often that's long enough to allow the thread that just released the GIL to be quicker and re-acquire the GIL. The effect: waiting threads keep getting awoken over and over trying to grab the GIL, only to be beaten to the punch by the already running thread.

The current Python trunk, which will become Python 3.2, contains a new algorithm, by Antoine Pitrou, where scheduling is based on time slices instead of ticks. Dave Beazley explains the new GIL algorithm and the tweaks that make it fairer and avoid the problems of the old approach.

A complete removal of the GIL was an item on the original todo list of the Unladen Swallow project, which has recently submitted PEP-3146: "Merging Unladen Swallow into CPython".

The GIL is also a problem in some Ruby implementations, mainly Ruby 1.9. However, JRuby, IronRuby, and even MacRuby, which was originally based on Ruby 1.9, allow parallel execution of Ruby threads.

Squeak, an open source Smalltalk, has been available for the iPhone for a while, and now Squeak has been ported to Android. The project is still at an early stage, but some of the details of the port are worth mentioning: Squeak on Android's only really possible because of Android's Native Development Kit (Android NDK), which allows to use native code in Android applications. With this, porting Squeak meant getting it to work with the ARM tool chain and  porting some of the system integration to Android's flavor of Linux. Andreas Raab, who created the port, describes what's necessary to compile Squeak for Android.

A bigger problem is the current nature of Android NDK: it only interacts with Linux part of Android. Any Java-based Android libraries and APIs are not directly accessible - native code needs to route the calls through JNI and custom Java classes that perform the calls on behalf of the code. Another option can be seen in the Android Scripting Environment: Android APIs are exposed via JSON RPC. Native version of Python and Lua already use this method to access Android features with the ASE.

ASE's supported runtimes and ports like Squeak for Android also show something else: native versions of language runtimes, compiled using Android NDK, seem to be the quickest and most efficient way to bring languages runtimes to Android. While Android's Java based APIs seem to be a good argument for JVM based language implementations, they're limited by Dalvik VM, both by its execution efficiency and its use of a custom bytecodes instead of Java bytecodes.

More and more native language runtimes become available for Android, the already mentioned Python or Lua for Android, or Ruby for Android, are just a few examples.

1. Back to top

   IronPython & Jython

   by Dave Kirby

   "The GIL is also a problem in some Ruby implementations, mainly Ruby 1.9. However, JRuby, IronRuby, and even MacRuby, which was originally based on Ruby 1.9, allow parallel execution of Ruby threads."
   
   It should be pointed out that Jython and IronPython also allow parallel execution of threads, since they use the JVM/CLR thread models.

   Reply

• Older >