BT
x Your opinion matters! Please fill in the InfoQ Survey about your reading habits!

Is IQueryable A Bad Choice For APIs?

by Roopesh Shenoy on Mar 30, 2012 |

Mark Seemann suggests in his article “IQueryable is Tight Coupling” that designing an API such that it exposes an IQueryable<T> interface is a bad idea for several reasons.

Why?

..the IQueryable<T> interface is one of the best examples of a Header Interface that .NET has to offer. It's almost impossible to fully implement it.

This simple fact means that you can never be sure that the method will return a complete implementation of the IQueryable interface.

Even so, the interface is so flexible and expressive that with a single exception, it's always possible to write a query that a given provider can't translate.

The only implementation that fully implements IQueryable<T> is the in-memory implementation

According to Mark, this makes the abstraction leaky because more often than not, the IQueryable will be implemented by a Store provider inside your code. 

While the argument makes perfect sense, a counter point could be, how bad is the leaky abstraction? Does it make writing code at least somewhat simpler that we can bear to live with it?

For instance, ASP.NET Web API magic makes it easier to use the same webapi for filtered data through request parameters, if the return type is an IQueryable. Similarly we have WCF RIA Services where the DomainContext methods returning IQueryable can used by the client to fetch only required data by using filter descriptors right in the XAML or from your JQuery Client

What do our readers think? Do the benefits of the flexibility outweigh the leakiness?

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Totally Agree by Jason Young

Totally agree, this is a very leaky abstraction, even if a provider does support every query you present to it, it may not implement it in a very non-performant way, and exposing it directly from the WCF service layer could be a disaster as end users are generally not savy enough to know what types of queries could tie up the server.

This reminds me of another common, but IMO flawed approach to implementing the repository pattern. You will often see people advocating creating a general IRepository<T> and a Repository<T> that exposes a single IQueryable, or a Find method that uses an linq Expression. These techniques completely lose the point of the repository pattern, the whole point of the repository pattern is to invert the dependency on your data, the contract for the repository should be defined in the business layer, it's implementation should reside in the data layer. By exposing a single method for finding entities you get the following issues.

a) It is very difficult to understand what queries are actually being performed
b) It is difficult to optimize a particular query, e.g. replace with a store procedure, of course new methods can be added, but they will break the existing contract, and tracking down all the places in the code that might be using the "old" un-performant query is then a manual process.
c) Most importantly, you have NOT inverted you dependency on the data layer, you have tied yourself explicitly to it.

With all that said, the query side of CQRS can quite effectively make use of IQueryable via specially designed query models, again, as long as they are not exposed directly as a WCF end point.</t></t>

Totally Agree too. by Sake .

Few seconds after skimming over the article, the phrase "Totally Agree" come first to my mind.
And it's already here in the comment too.

Microsoft OData is another step in very wrong direction.

Agreed in almost all general header interfaces cases too! by Ethar Alali

The only exceptions to the general rule on header interfaces that comes to mind are message/service contracts, but you should not be exposing IQuerable through a standard WCF service contract anyway. Data contracts, one could argue, may need to use it. However,again I would argue a good, implementation specific, exposure through a WCF service would do the job better.

In the specific case of message contracts, the trade off is that the system doesn't necessarily scale well if each individual 'role interface' is required for processing units. There is a lot of extra effort to develop architectural units to deal with them and if individual services require more than one such interface to complete the necessary dataset required for processing and it is is not explicitly defined in any greater contract, then it may require multiple calls which, as with all RPC/RMI mechanisms, will impact performance severely.

Indeed, role interfaces can increase coupling in a different way, by potentially adding to the number of required processing units needed to deal with the specific interface implementations, even if the same object is exposing two or more role interfaces, if needs be.

Anyway, I digress. But as usual, careful thought and some trade-off analysis is necessary, given specific contexts.

Re: Totally Agree by Steven Pena

Totally agree with "Totally Agree". Just adding to the agree-ness so we can amass in numbers and do away with this horrible pattern I keep seeing in code examples and in projects.

IQueryable rules by Olmo del Corral

I can see the article has a point:

* Implementing IQueryable is also incredibly hard. I'm the developer of Signum Framework Linq provider and I've been working on it for 3 years already. You can never fully implement it, at the expenses of some runtime exceptions for un-supported expressions.
* IQueryable<T> and Repositories don't work together: I'm not an expert of DDD, but AFAIK the whole point of Repositories is to abstract away where entities are saved, so you can mock it ant test it.... and mocking an IQueryable is hard.
* Also, because lambda quouting is transparent, most developers don't really know if they are worknin in SQL or memory.


But IQueryable is the sweetest API ever.

* It's a perfect abstraction over a queryable store. The LINQ operators are incredible expressive, I haven't seen any other API that is able to express such complex queries in such an elegant way. Any other API fights the C# language, or are more limiting, while IQueryable beats SQL readability by far.
* It's true that there are IQueryable implementations that produce bloated queries, but very often the reason is that SQL is less expressive, not a translation issue.
* Even if a complete implementation is not possible, the partial implementation is familiar to the API consumer. I think this is more important.
* At the end, any other implementation lacks projection operators, they only filter, so there are important performance consideration on reporting scenarios.


So back to the main point, if Repository don't work with IQueryable, then Repository is the one that should be removed.
* IQueryable protects your queries from adding removing and renaming database fields and tables (since it has refactorings and compile-time checking), something that happens very often on live applications.
* Repositories allow you to change the ORM... I've never done that in an application. Did you?

The other use of repositories is to keep your application more modular, so you can replace your data store from a Sql Table to a web service.... Bullshit!:

At the end you are going to join Orders, Customers and Products, how can you do that if Orders is implemented as a MongoDB collection, Customers is a web service and Products a Csv file? At the end you end up writing most of the code below the repository abstraction, so your code is not abstract anymore.


My final point: An interfaces is a door that keeps two rooms separated. Making five doors just inside a bathroom creates more problems that it solves. All this inversion of control paranoia costs money. </t>

Disagree by Eugene Tolmachev

Didn't read the original article, but the offered highlights read like purist complaints lacking constructive criticism.
Seems to me the argument boils down to static verify-ability. Is that really what we are asking for in deferred evaluation?
I admit mocking it is hard, so I've got a small linq-to-objects implementation that stubs out a given repository for me. Sidestep the issue and move on.
In exchange it gives me a way to traverse tiers in very efficient and flexible manner.

All abstractions are leaky, the question is "does a given abstraction offer enough benefits to significantly outweigh drawbacks?" and to me, IQueryable does.

Re: IQueryable rules by Francois Ward

How is mocking iqueryable hard? If the iqueryable itself and its components are what you're testing, yeah, its hard. But if you're making an application that depends on iqueryable return values, just mock them and return a list or something. Problem solved. Then again, one definately could argue that iqueryable makes repository obsolete aside in a select few edge case anyway...

Re: Totally Agree too. by Daniel Fisher

>> Microsoft OData is another step in very wrong direction.

OData is a very nice HTTP Query Protocol.

It has nothing to do with IQueryable - beside the fact M$ is using the interface in their implementation (WCF Data Services). WCF Data Services and the OData Project (odata.codeplex.com/) are so thightly coupled that it's not possible to 'just use the OData-Parser' cuz there is none...

Thats leaky abstraction

--Daniel

Re: IQueryable rules by Daniel Fisher

iqueryable is not repository. iqueryable is context. context is unit of work...

Re: IQueryable rules by Daniel Cazzulino

IQueryable is not unit of work.
IQueryables are exposed from a context, which is the unit of work. That unit of work/context might be called "Repository" as it abstracts the storage, I think...

Totally Agree too. by Nikos Baxevanis

All of these make a lot of sense. You don't need to expose IQueryable and if you do you implicitly tight yourself with something concrete; that is either the query provider that is associated with the data source or the expression tree that is associated with it.

Re: Totally Agree too. by Jacek Helka

Actually, you can mock the provider. It's not trivial, but not so dificult also.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

12 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT