The Dark Side of Closures

Closures are not a new concept and in LINQ have proven to be incredibly useful. But they do have a dark side when used to break encapsulation. When two seemingly independent functions are tied together, unexpected results can occur.

Closures allow functions to share their local variables with anonymous functions defined within them. These anonymous functions, often referred to as lambdas, are essential to creating the strongly typed queries exposed by LINQ.

In his experiments with LINQ, Dustin Campbell discovered that queries could be altered while they were running. That's right, by changing a local value that was closed over, the function used in the where clause is modified. If this is done while the query is running, the query may acknowledge that change and respond accordingly.

Dustin uses this trick to create a query that only returns distinct items. To start with, the where clause is set to the expression "m.Name != filter". Each time an item is returned, the value of filter is updated to be the last name returned. Under the right circumstances, this can effectively create a list of distinct items.

However, it is an incredibly fragile technique. In Dustin's case, the list had to sorted prior to applying the where clause. If done the other way around, all the filtering will occur before any items are returned and there will be no chance to alter it. This can be done in LINQ because where and order by clauses can appear in any order.

Something Dustin did not mention is that this will not work for all providers. Those that send the query to an outside engine like LINQ to SQL will also not get a chance to alter the where clause. It gets even worse if you try to use Parallel LINQ. Because it is running on multiple threads, any change to the where clause will result in a race condition.

Of course the right answer is to simply call Distinct() and be done with it. While these tricks are academically interesting, they are certain to lead to subtle bugs and are very susceptible to changes in the framework.