More on Parallel LINQ

| by Jonathan Allen Follow 578 Followers on Sep 20, 2007. Estimated reading time: 2 minutes |

LINQ, or Language Integrated Query, is the poster child of Visual Studio 2008. It allows for a SQL-like syntax to be used for querying any kind of data. Out of the box it supports SQL Server, XML, and in-memory objects and the extensible framework allows developer to support other providers including MySQL, Amazon, and Google Desktop.

Queries are special in that they are inherently parallelizable. While queries can have side effects that prevent parallelization, they are rare. This makes them an ideal candidate for parallel processing libraries like PLINQ.

PLINQ, formally known as Parallel LINQ, operates on XML and in-memory objects. Queries that are executed on a remote server such as LINQ to SQL are obviously not candidates.

To turn a LINQ query into a PLINQ query is surprisingly easy. Just add the call ".AsParallel()" to the end of the data source specified in the From clause. The Where, OrderBy, and Select clauses are automatically changed to call the parallel version, as are any joins.

According to MSDN Magazine, PLINQ can execute queries in one of three modes. In pipeline processing, one thread is used to read the data source and the other threads are used to process the query. The results can be used while this is going on, though the single consumer thread may have difficulty keeping up with the multiple producers. But when the workload is well balanced, this can have a significant reduction in memory consumption.

The second mode is called "stop and go". This is used when the entire result set is needed at one time, such as when calling ToList, ToArray, or when sorting the output. This completes all the processing at once and then turns over all the resources to the consumer thread. This can have better performance than the first method because the need for synchronization between the query threads is reduced.

The final method is called "inverted enumeration". Instead of collecting all the output and processing it on a single thread, the final function called is passed to each thread via the ForAll extension method. This is by far the fastest method, but it requires that the function passed to ForAll be thread safe and preferably free of locks and side effects.

An exception in any PLINQ thread causes all the other threads to terminate. If multiple exceptions occur, they are all bundled into a single MultipleFailuresException with their original stack traces preserved.

Rate this Article

Adoption Stage

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread


Login to InfoQ to interact with what matters most to you.

Recover your password...


Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.


More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.


Stay up-to-date

Set up your notifications and don't miss out on content that matters to you