Debate: ODBMS sometimes a better alternative to O/R Mapping?

I think OODBMS can be a good option when the data requirements are modest.
I've successfully used db4o in a couple of project and there are some other nice OODBMSs our there. However I wouldn't use them for larger project, esp. if the project is data intensive. I also talk about that in a paper I wrote on OR/Mapping when/why

Arnon

> And what about the case in which data is shared between
> applications having slighty different object models?
> Are OODBMS able to handle those kind of case?
Well this depends on the vendor. Some can better, some can not.

For example, db4o can handle different classes with different names
and partially different data if you use an alias. This allows e.g. the
usage of Java and C# classes in different apps but on the same database.
(That can run under MS or Java).

By the way: This is a nice extraction of the "meta discussion" between these groups as Gavin King and Ted Neward. And a perfect forum for the discussions
of the "when and why" and "the right tool for the right task" might be the
Object Database Conference that will be held 2008 in Berlin with my support:
www.icoodb.org

Try to contact or join!

Best
Stefan Edlich

And what about the case in which data is shared between applications having slighty different object models? Are OODBMS able to handle those kind of case?

Apologies if my post came across as aggressive or critical at all. That wasn't how it was intended, although reading it back now I can see that it might be read that way.

Now, perhaps if all the projects you've ever done are projects where the developer gets to define both, then the problem doesn't appear, but if you're in an "enterprise" world where the database schema is managed by a team of DBAs and is shared across projects, you don't have the flexibility to "refactor" the schema like you can your object model.

I find myself violently disagreeing with myself, or at least the way I've been quoted ;-)

I don't believe that the correct way of looking at an OODBMS is that it minimises the need to keep an object schema and an SQL schema. To me this is spurious thinking.

The key (IMO) to using an OODBMS correctly relative to an RDBMS is to note that they have very different (performance) characteristics.

1. OODBs have a very close match to single link OO navigation, a good match to OO business logic. they are optimised for that.
2. OODBs are very good with complex graphs -- like the ones required by CAD tools etc.
3. OODBs are very poor at data independence. Use an RDBMS for any business data.

Areas in which I've used an OODB very successfully are with a CAD tool, and also in telecomms where there were millions of customer records with some complex graphs attached. Also, for batching and queuing messages spooled straight from a financial exchange where realtime operation was paramount.

As to Gavin's claim about performance, I respectfully disagree that an RDBMS is always faster all other things being equal. The key is that OODB's excel at single link navigation, and going against the grain of these links will produce terrible performance. If you keep to this rule then, and don't require set based operations, performance in an advanced OODB is generally quite suprisingly spectacular over many millions of records.

It is correct however in that OODBs are very much more immature than their RDBMS counterparts. In addition, the tight coupling of the object model and the schema actually produces problems in an enterprise when adding fields and morphing a schema...

As for the use of embedded OODBs, most OODBs like Versant and others are client-server. Even "small" OODBs like objectdb offer a superb client server mode supporting thousands of transactions a second on modest hardware. I've never used an OODB in embedded mode except for the simplest of apps.

Andrew

As to Gavin's claim about performance, I respectfully disagree that an RDBMS is always faster all other things being equal.

As to Gavin's claim about performance, I respectfully disagree that an RDBMS is always faster all other things being equal.

I most certainly never made that claim, or anything remotely like it. All I did was point out that there are many complex variables affecting performance, and, a priori, the dominant variable is *not* whether the underlying conceptual model is relational or object-oriented.

OODBMS's are no panacea, but neither are RDBMS's. While there are some very solid reasons for not using an OODBMS in certain situations, like data warehousing, every argument I have seen that argues against the use of OODBMS's in any situation boils down to one sentence:

"OODBMS's are not mature enough"

There is nothing that precludes and OODBMS from having data integrity, ad hoc querying (couldn't they just support HQL?), or interoperability.

Furthermore, while it is true that another application may need to get into the database at some point in the future (which I agree, is an important thing to have), what would prevent them from using it in the same manner as the java application? At least assuming a driver exists for the language, which it would -- if the OODBMS were mature.

Hesitation to use something off the beaten track is understandable, but that does not mean the entire concept is invalid.

Sure, an OODBMS is not a mature (in terms of widespread usage), battle-tested concept, but neither was a RDBMS at one point (or, more recently, ORM). It is good to see some more practical effort going into the field -- it will only help expand and improve the tools we have available.

Well my concern is not about the platform but about the OO model. I have never seen two applications use the exact same domain design (or seen any domain based libraries succeed). After all, a design is just an approximation of the world out there based on the application needs. Therefore I always though it was quite normal to have some kind of mapping (as light as possible) between the data storage model and the domain model in case where the data is or may be shared between 2 applications.

To see the pros and cons of an OODBMS approach, you really need to look into the actual database model. It's not really anything to do with maturity. OODBs have a model very similar to java objects, where links are directly encoded and they are (almost always) one way. i.e. if you have a reference in class A to class B, then you can navigate from A to B but vice versa is very difficult.

so, what makes an OODBMS fast also makes it tied to the application logic that the links encode... That's why adding ad-hoc queries is so difficult.

Interoperability and data independence is limited by the navigation paths you have encoded.

Andrew

Well my concern is not about the platform but about the OO model. I have never seen two applications use the exact same domain design (or seen any domain based libraries succeed). After all, a design is just an approximation of the world out there based on the application needs. Therefore I always though it was quite normal to have some kind of mapping (as light as possible) between the data storage model and the domain model in case where the data is or may be shared between 2 applications.

To see the pros and cons of an OODBMS approach, you really need to look into the actual database model. It's not really anything to do with maturity. OODBs have a model very similar to java objects, where links are directly encoded and they are (almost always) one way. i.e. if you have a reference in class A to class B, then you can navigate from A to B but vice versa is very difficult.

so, what makes an OODBMS fast also makes it tied to the application logic that the links encode... That's why adding ad-hoc queries is so difficult.

Interoperability and data independence is limited by the navigation paths you have encoded.

But must OODBMS's have a java-like model? Is having one-way associations a requirement? If you need to go back and forth, what's stopping you from putting in that other link in the form of a collection or map?

There might be performance problems with that, but such a thing is dependent on the implementation. It shouldn't need to have any more overhead than a hibernate many-to-one relationship.

yes, performance evaluation and estimation is complex. however, the 2 models of db are very different under the hood resulting in different performance characteristics. ... I haven't looked at the technical side for over 5 years, and I've never implemented either product fully myself, so bear with me, but OO dbs (under the covers) have effecively a pointer concept -- the ODMG used to call them "swizzled pointers". Under the covers of a RDBMS are set structures and indexes.

Integrating link navigation facilities of an OO database into a relational, set-based model is apparently difficult and carries tradeoffs.

Personally, I think the role of OODBMS has been hampered by the fact that there isn't a good, clearly-recognized, free (as in beer), commercial-friendly-licensed (Apache, BSD, etc.) OODBMS. There are a number that match one or more of those criteria, but none that I'm aware of that gets the whole set.

Oh c'mon, that's silly. Every implementation of object references boils down to a "primary key" under the covers of a simple programming-level abstraction. Even Java layers object references over some kind of memory address.

It's called ORM. It's a *way* more successful technology than OODBMs. It's not perfect by any means, but it solves the problem for most people.

Personally, I think the role of OODBMS has been hampered by the fact that there isn't a good, clearly-recognized, free (as in beer), commercial-friendly-licensed (Apache, BSD, etc.) OODBMS. There are a number that match one or more of those criteria, but none that I'm aware of that gets the whole set.

Oh c'mon, that's silly. Every implementation of object references boils down to a "primary key" under the covers of a simple programming-level abstraction. Even Java layers object references over some kind of memory address.

Umm, no, it's not silly. In an object database these are literally "pointers", not some a set of numbers that must be further resolved into a set of addresses through a filter/join. .... Have a look under the covers of how these things work in the products, not just in the mapping layers. Single link navigation and set-based join are very different primitive operations with different performance characteristics and a different conceptual basis. They are much closer to the old hierarchical databases which preceded relational.

• An object database stores a to-one association as a "pointer", by which I understand you to mean an address of a disk location. (I'm skeptical that there is not some additional indirection there, but I've not implemented an object database so I'm not sure of that.)



• A relational database stores a to-one association as a primary key values which is resolved to an address of a disk location via an intermediate (efficient, pure in-memory) index lookup.

Further, caching is often problematic for horizontal scaling in large enterprise apps, requiring classification of different data types and expiry times. (as an aside, in all the object dbs I've used, they've never had a need for a cache for performance reasons).

So this index lookup is what is making the relational database so much slower?

I don't get it. I've simply never met an application where performance was bounded by the
cost of index lookups used for resolving to-one associations. Every application I ever met was bounded by the cost of server round-trips.

Now you've really, really lost me. Why on earth should the caching be a different problem? I have a server, with data on disk. And a remote client, which needs that data. Ergo I need a cache. Different types of data in my system are accessed in different ways, with different consistency requirements. Ergo I need different expiry policies.

You're now deep in implementation details. So I'll return to my original argument: nothing you are describing here is a fundamental attribute of the object-oriented or relational conceptual model; what you are describing is how (some) existing systems happen to implement that conceptual model. A priori, I don't see any reason why a primarily relational database could not make the optimizations you are describing. Likewise, I don't see why a primarily object-oriented system couldn't provide great reporting and data integrity, just like a relational database.

It's not about "sets" vs. "pointers". Those are just abstractions that exist at the API level and at the conceptual level.

Of course, it turns out that most relational systems that exist today happen to not implement optimizations for operations on the kind of hierarchical data that crops up in a number of niche usecases (CAD being the example that is often given). I think that's primarily because those usecases are rare in business applications. Which is not to say they're unimportant.

Most importantly, none of this has to do with any "dual schema" or "paradigm mismatch" problem.

Yeah but an person doesn't mean the exact same thing to both application even though they use the same date. In application #1, the person may have an association to a department and 5 or 6 subclasses while in the other application, this association to a department has no meaning and shouldn't exist (even though the data is present) and doesn't support any person subclass.

If it's not a case an OODMS can handle then I think morst applications should stay away from this technology because nobody can't predicate the enterprise requirements. Programming for phantom requirements may be a bad habit but your architecture has to be able to scale up or down.

You're now deep in implementation details. So I'll return to my original argument: nothing you are describing here is a fundamental attribute of the object-oriented or relational conceptual model; what you are describing is how (some) existing systems happen to implement that conceptual model. A priori, I don't see any reason why a primarily relational database could not make the optimizations you are describing. Likewise, I don't see why a primarily object-oriented system couldn't provide great reporting and data integrity, just like a relational database.

It's not about "sets" vs. "pointers". Those are just abstractions that exist at the API level and at the conceptual level.

Of course, it turns out that most relational systems that exist today happen to not implement optimizations for operations on the kind of hierarchical data that crops up in a number of niche usecases (CAD being the example that is often given). I think that's primarily because those usecases are rare in business applications. Which is not to say they're unimportant.

Most importantly, none of this has to do with any "dual schema" or "paradigm mismatch" problem.

There is nothing that technically limits an OODBMS to the same limitations of, for example, Java. That in mind, it is quite possible for an OODBMS to provide a view to each application that satisfies every concern you just stated.

Are there any implementations like it? Don't know. But until there is a widespread standard (i.e. the equivalent to SQL for OODBMS), we shouldn't write off what an OODBMS can and cannot do.

I disagree with the statement that "nothing you are describing here is a fundamental attribute of the object-oriented or relational conceptual model". Look at the maths in Codd's paper if you haven't had a chance to already:

www.seas.upenn.edu/~zives/03f/cis550/codd.pdf
(it was a significant enough shift to warrant the Turing award for the guy in '81)

The relational model has a conceptual foundation in set theory in maths. Object dbs do not. This conceptual link is what gives the relational model its power. To add in hierarchy, it needs to be done at this level, where sets are the main construct. How do you get a link? It's a set intersection. Hierarchy is subordinate to sets, and essentially missing from the base concepts in the model.

Objects see databases as memento and object-graph storage. Databases see objects as data exposed in table rows. RDF databases see objects data exposed in schema-constrained graphs. The private of one is the public of the other. The benefits of each conflict with the design goals of the other.

Perhaps REST is the middle ground that everyone can agree on. Objects interface easily using REST. They simply structure their mementos as standard document types. Now their state can easily be stored and retrieved. Databases interface easily using REST. They just map data to data. So the data in an object and the data in a database don't necessarily have precisely-matched schemas. They just map to the same set of document types and these document types define the O-R mapping. The document type pool can evolve over time based on Web and REST principles, meaning that tugs from one side of the interface don't necessarily pull the other side in exactly the same direction.

If O-R mapping is the Vietnam of computer science, perhaps we should stop mapping between our object and our relational components. Perhaps we should start interfacing between them, instead.

Benjamin.

OK, here's where I'm checking out of this discussion. My eyes glaze over the minute people start talking about the supposed magical set theoretical foundations of relational databases.

Relational databases have almost nothing to do with what mathematicians call set theory - which is primarily concerned with the study of transfinite numbers (a la Cantor), or with reducing mathematics to primitive axioms (RW, ZF, etc).

Relational databases bear the same relationship to set theory that arithmetic bears to number theory: ie. the most trivial, pedestrian application of the most basic definitions.

I understand that most computing professionals have not studied set theory and are easily intimidated by the invocation of this term, but my major was pure mathematics, and I'm not intimidated, indeed I'm well aware that set theory has almost nothing to do with the practice of data management.

The biggest problem with Gavin's position is that he assumes everybody sees the world like him. Apparently, though, there are 10,000s of Java and .NET developers who see that limiting one's options is usually not the best strategy to improve the outcome.

The main flaws in the ORM vendors' arguments are:

spend hundreds of millions of dollars re-educating developers and data management professionals

[ODBMS lack] ad hoc querying

even a partial migration carries many costs and inefficiencies due to the exchange of one data management technology to two. Can you imagine the pain and suffering in the inevitable need to occasionally migrate data between your OODBMS and your RDBMS?

As to Gavin's claim about performance, I respectfully disagree that an RDBMS is always faster all other things being equal.

I most certainly never made that claim, or anything remotely like it. All I did was point out that there are many complex variables affecting performance, and, a priori, the dominant variable is *not* whether the underlying conceptual model is relational or object-oriented.

That's not to say that any particular OODBMS is not faster than some particular RDBMS for some particular task.

It is not even to say that, with their emphasis upon particular kinds of tasks, existing OODBMSs in general are not faster than existing RDBMSs in general. The usecase of CAD tools that you quote is a good example of where existing OODBMSs have a strong featureset, and existing RDBMSs are weak.

Object Oriented deep graph models, pojos specifically, cannot be easily reasoned over declaratively - which can create a reliance on imperative programming. With the work that W3C has done to standardise Description Logic with OWL-DL combined with declarative reasoning sytems like Drools (we'll be adding Description Logic based modelling after 4.0) you have a much more powerful metaphor for application development (although probably not framework/subs system development) - also forget OWL Full, its an academic exercise, and RDF triples are unfortunate, but luckily can just be considered a transport mechanism. Declarative relational programming obviously has a much closer 1 to 1 mapping with the database itself.

For a simple example look at the Conways Game of Life example we provide (will be updated to ruleflow soon, instead of agenda groups, which will make it more declarative). In this example we have a large NxN grid of Cell objects, the previous approach was for each Cell to have a HashSet of its surrounding Cells. The only way to calculate the number of surrounding Dead/Live cells was to imperatively iterate that HashSet for each Cell. This would create repeatedly redundant work, as we don't know what has and what hasn't change, we could track that, but again it's more imperative code and tracking we have to do. The updated Conways example uses a relational approach, no nested objects,(Although no DL yet, that is post 4.0) instead we use a Neighbour class to bi-directionally relate each surrounding cell; this means we simply declare what we want it to do to track Dead/Live cells and the system, with its understanding of the relations and what has and what hasn't changed, does the rest for us.
anonsvn.labs.jboss.com/labs/jbossrules/trunk/dr...
rule "Calculate Live"
    agenda-group "calculate"
    lock-on-active
when
    theCell: Cell(cellState == CellState.LIVE)
    Neighbor(cell == theCell, $neighbor : neighbor)
then
    $neighbor.setLiveNeighbors( $neighbor.getLiveNeighbors() + 1 );
    $neighbor.setPhase( Phase.EVALUATE );
    modify( $neighbor );
end

rule "Calculate Dead"
    agenda-group "calculate"
    lock-on-active
when
    theCell: Cell(cellState == CellState.DEAD)
    Neighbor(cell == theCell, $neighbor : neighbor )
then
    $neighbor.setLiveNeighbors( $neighbor.getLiveNeighbors() - 1 );
    $neighbor.setPhase( Phase.EVALUATE );
    modify( $neighbor );
end

I also invite you to look at the "register neighbor" set of rules, so you can see how these Neighbour relations are setup declaratively, exploiting the cross products of the column and row fields in the Cell.

While this is just a simple example using propositional logic you can exploit these relations much further, especially when working with sets of data and first order logic using things like 'collect', 'accumulate' and 'forall'. For more info see What's new in JBoss Rules 4.0 which is released mid next month.

Mark
markproctor.com
markproctor.blogspot.com

I used to study math also but is saying this

which is primarily concerned with the study of transfinite numbers

I understand that most computing professionals have not studied set theory and are easily intimidated by the invocation

with reducing mathematics to primitive axioms (RW, ZF, etc).

As far as I understand, it's the accessing interface we are concerning, that is:
*SQL* vs *Object*

IMHO, ORM is not perfect coz it simply translates *Data Operations* via Objects to *SQL Transaction Scripts*, at CLIENT side. They can NOT map perfectly especially with *Data Semantics* seriously considered.

My approach is to take RDBMS as a storage engine, thus provide *Relational* interface to Analytical Applications which need readonly access via SQL. While keep an ever-running *Object Graph Environment* as a standalone server instance, thus allow Transactional Applications to send executable objects to this environment for data manipulation. The *Object Graph Server* manages *Object Transactions* itself (unlike ORM that tergiversates that to the RDBMS).

See:
www.ableverse.org/articles/fakeorm.pdf
tob.ableverse.org

You're now deep in implementation details. So I'll return to my original argument: nothing you are describing here is a fundamental attribute of the object-oriented or relational conceptual model; what you are describing is how (some) existing systems happen to implement that conceptual model. A priori, I don't see any reason why a primarily relational database could not make the optimizations you are describing. Likewise, I don't see why a primarily object-oriented system couldn't provide great reporting and data integrity, just like a relational database.

It's not about "sets" vs. "pointers". Those are just abstractions that exist at the API level and at the conceptual level.

I disagree with the statement that "nothing you are describing here is a fundamental attribute of the object-oriented or relational conceptual model". Look at the maths in Codd's paper if you haven't had a chance to already:

www.seas.upenn.edu/~zives/03f/cis550/codd.pdf
(it was a significant enough shift to warrant the Turing award for the guy in '81)

The relational model has a conceptual foundation in set theory in maths. Object dbs do not. This conceptual link is what gives the relational model its power. To add in hierarchy, it needs to be done at this level, where sets are the main construct. How do you get a link? It's a set intersection. Hierarchy is subordinate to sets, and essentially missing from the base concepts in the model.

[...]

Andrew