Joseph Yoder on Adaptive Object Model Architecture

1. My name is Srini Penchikala. I'm here with Joseph Yoder from the Refactoring Inc. Joe, can you tell us a little bit about what you are currently working on?

Basically I'm involved with a lot of things right now. I have a team of developers, we're writing a lot of software, as well as refactoring software. Specifically, we are writing a lot of software on adapting to changing business requirements. In the specific style of that, architecture is what we call the adaptive object model architecture, which is an architecture for allowing you to adapt to changing business requirements.

2. What is adaptive object model and how does it differ from a traditional object model?

For a traditional object model, when you think of modeling your objects you might use a UML diagram, you draw your different main domain constructs within the business. For example with the insurance domain you might have things like customer, insurance policies and you have different behaviors in your interactions between those types. A customer may have things like an address and things that they own that they might want to ensure, like a house. So, you have this classic object diagram that generally when you write in code, you would write classes for it, like Java classes - or whatever language you are using - to represent that model.

The thing with that is, if you are modeling things that way, whenever that object model changes, say in the insurance domain you want to change the insurance policies and types of products and services that you are offering, you have to write new classes and recompile it and release with that. With the adaptive object model, the difference between it and the normal one is you still want to model those types of things, those types of business entities, such as in the insurance domain you want to get your insurance policies, but, rather than representing those as classes, you will still be able to model them, but we are going to represent them with descriptive information about your business domain, so that we can change that, even at run time.

You are taking the descriptive information and putting it into XML or a database, or some kind of metamodel that you can interpret at run time. The difference between adaptive object model and your normal way of object oriented development and when you are doing object models generally is, rather than representing your business domains as classes with attributes and behaviors, you represent the types of models that you need as data so your classes and your methods on classes and all the attributes and relationships between classes are representing as data so we can change and add those without changing code.

An adaptive object model is very useful if you're saying you are going to have these new types of products and services and your business is going to change within a domain specific way, you can represent them that way and it will allow users to adapt more quickly to that changing requirement. It usually evolves from frameworks and domain specific languages - they have a lot of related patterns that you see arriving from them.

3. You mentioned about the insurance domain as an example. What are the other domains or the other real world examples where the adaptive object model can be used?

I've seen it used for a lot of different places. The main thing is you can use it wherever your business requirements are changing very rapidly. There is a time to do it and a time not to do it, but the different types of domains where I've seen it very useful for are insurance is obviously one - I've done some work there -, the banking industry, in Wall Street people have done it a lot, but we've also seen it where people have done it in manufacturing. I'm working now with a client where we are using it in a couple of different places, one is in invoicing. With all our different customers we do our invoices differently, so we have ways of describing the ways that we do that pushing that out into the database and then we can interpret that at run time and really quickly adapt to the changing way they do that.

Another example is we are importing different types of information from our clients so we can describe all those differences. A good example of this, one of the early on, when Ralph Johnson and I were really looking at this and starting to write about it, we noticed a lot of people making these cool systems for adapting. They had a similar architectural style and even something as simple as part number, I know of an organization I've worked with, cost a lot of money to change when they bought another company and had to adapt something.

Their software was something as simple as part number, maybe you went from 10 digits to 12 digits or maybe there is some different rules around alphanumeric. If you could model that by describing things about part number, rather than hard coding that in, then you can very quickly adapt to the changing business requirements in that. That's the adaptive object model's useful for. It can be used for a lot, but there is a lot of domains, as I mentioned and I can go on and on about it. I do have a lot of information on adaptiveobjectmodel.com or people can go look a lot of other examples of different domains people have worked on.

4. Regarding the dynamic nature of the adaptive object model you mentioned, for example, that the properties of an object can be changed at run time. What are the constraints in that? How far can the developers go to exploit the dynamic nature?

That's a danger, by the way, when you talk about how far developers can go. A lot of us, computer scientists specifically, like languages and we like inventing cool things, so you could really push this as far as you want, but there is also the downside, the danger of it. In some of the patterns that we've been writing about, I've been trying to write the best practices of what works versus what doesn't work, when you should do it, when you shouldn't do it, because there is this trade-off. It costs a lot when you are inventing a new type of domain specific language, so you should really have good user scenarios.

You use your standard Agile principle for developing it, a very iterative approach, get feedback often and only put the flexibility in where you need it. If you will need it for part number, there is no reason in pushing it too far. You could push it all the way, by the way, you can make everything in your system totally adaptive and that means that we're describing everything in data, but if you go that far, what have you done? You're, in a sense, almost inventing a new programming language. Do you really want to reinvent Ruby or Java or some other language? We have languages for that!

What you end up doing - and this is a common practice - is you put the flexibility where you need it and another pattern I've commonly seen as I've been working with building these systems is you have dynamic hooks at places in your code whereas if you need to add extra behavior that your adaptive object model you did need at a time, you have a place where you can still write code to overwrite some of this behavior without messing up the language. The pattern is, if you start seeing 2 or 3 of these types of similar hooks you happen to overwrite, than you can refactor your adaptive object model to allow for this new type of dynamic behavior.

5. It's a good point, because too much flexibility always brings in complex DSLs also.

Some of the workshops we've had, by the way, have looked at that because some of the other people that have done work - you look at the MDA people, the model driven architecture - have more of a generative approach a lot of times, also when we were looking at the math, when they were looking at the 4 layers there. A lot of people have been, for example, focusing on a language to define your domain specific language, so a language for language. They have more of a top-down approach. The adaptive object model architectural style was really more of a bottom-up approach.

You have a real problem right now that you need to solve and you need this type of flexibility and there is a good business case for you be able to very quickly adapt your system to this changing business requirements quickly, so add that flexibility in here. They are related and, in fact, a lot of domain specific languages can do some very similar things to produce some of the same results of what an adaptive object model is, but there are some different ways of doing it. You don't have to use an adaptive object model architectural style to get flexibility. You could do scripting, you could do generative approaches, there are all kinds of competing the related types approaches.

6. What are the main elements of adaptive object model if someone wants to start learning about AOM in more detail?

The main architectural elements and we've written them up as patterns, which you can find off our website - adaptiveobjectmodel.com - is, in a sense, you still want an object model, you still want things that look like classes and have attributes and still have relationships between classes and still have behaviors, some kinds of functions or methods behind it. A type object, for example, which Ralph Johnson and Bobby Wolf wrote up, as a pattern back in the late '90s, is a good way of describing new types of things, new types of entities for your system.

That gives you new types of entities and if you entities aren't varying much, like they don't have different attributes and they don't have different behaviors, than that's good enough, but, in general, in more interesting systems your different products and services are going to have different attributes. Then you can use the properties pattern, which is another one that Brian Foote and I wrote up, that gives you dynamic properties, so that you can say that, if, for example I'm at a store and I'm selling different things - videotapes and sweaters. Videotapes have attributes such as rating and maybe the length of it and the category, whereas sweaters might have things like color and size, different types of attributes.

The properties pattern can help your different types of objects that you've used have varying properties. Normally you would have written this in classes, but now that we've pushed it out for being able to describe it, we can quickly just have an editor or GUI for describing these new types of objects with their attributes and you can instantly reflect those new types of things. Then, you can use the accountability - Martin Fowler wrote about that - for relationships between objects and then, ultimately, you need behaviors. So, we have objects, we have new varying attributes, we have relationships between those objects, but the more interesting thing is user behavior varies between these objects.

You can start off with a little simple set of strategies for defining your different types of behaviors and you can associate your type objects with those different types of strategies. Ultimately, these strategies can grow and maybe you have ways of composing them when you say "When this is true and that is true" or you end up with a domain specific language in a sense that varies within the domain - that's the part of adaptive object model that would be very hard to generalize. But it's still very similar: you have primitives in your languages, just like Java or other languages that have primitives and you have a way of composing and building that up.

The differences are it stays very domain specific, so you only focus within that domain, where there are domain constructs and things that you need for that. Those are the main architectural elements. Of course, there are a lot of other patterns and complexity that can come in the versioning history, your rules can change over time, you need dynamic GUIs, you may need ways of dealing without those types of issues. I'm waving my hands at a lot of those. We are trying to write up a lot of these. I see these as the core things.

The software community has really evolved through the 90s and into the new millennium, things started changing so quickly, so that we really need to work with these types of systems. These systems are cool and they are not that hard to understand, but the problem is a lot of people don't know them that well. By knowing the patterns and knowing how it works and when to do it and when not to do it, will really help people build these types of systems and be successful with them.

7. Speaking of learning the model and also documenting the model, usually we use the UML - Unifying Model Language - to document the traditional object models. Can we still use UML for the adaptive object model or are there other modeling notations?

Absolutely, you can still use UML. One thing I like about what Charles Simonyi talked a lot about is Intentional Programming. You can end up with a domain specific languages related to the intents of what the domain expert needs. You can create some kind of modeling technique within that domain. Something that Dirk Riehledid quite a few years ago and published a paper on was he wrote a UML Virtual Machine, which is an instance of an adaptive object model. It exactly is an adaptive object model, but there he did it!

He used UML as a modeling tool and then under the hood he had an adaptive object model that, when you describe new types of classes, you could actually do run time and get instant feedback and run tests on it. At that case you could even make your visual language, your editor look like UML, if you want it, or you could have UML over here and then draw that up, if that makes sense. A lot of times, what I've seen is every domain has things that those domain experts are used to. If an object model works for you, it's fine! But our main goal, lot of times when I'm working with clients, is to give them something that they can relate to that the domain experts can look at. I don't want them to have to learn UML.

If I'm the one maintaining and growing it, maybe I'll use UML, but for example, with our invoicing language, we gave something that was related to setting up different types of client charges for the invoices that mapped down to how they got all those orders for the math and then they can sum up more mathematical. That's what the people that were working with the invoicing language were used to. We did not use a UML there, but we could have.

8. In more traditional applications some of the properties of the objects are stored in database tables and fields and some of them are indexed and have constraints to enforce some business rules. How do you match that with the greater flexibility of an adaptive model in which all properties are stored in the same way, so you can't distinguish between one particular property that needs certain constraints or needs to be indexed for performance?

When you really start going to the next level, which you do need to store in a database - like you were pointing out - usually you need to persist these values and if you are persisting them in a relational database for example, you can start having some of these issues: there could be constraints, there could be extra business rules. Generally, if you are having to deal with that, you'll have to have another layer for dealing with some of those types of complexities. Your adaptive object model, then starts to add in extra complexity to handle some of those constraints. Constraints, by the way, is something that I didn't mention, but you could use a constraint-based language as part of your rule language, for example, within that.

Within the property itself, you could think of "You are my supervisor if and only if we work in the same department and you've been assigned a certain role within that". There could be other attributes or properties that you have before you can have this other property - for example, the medical domain. We wrote an adaptive object model for it, for newborn screening, for newborn infants in the country and you can screen for certain genetic diseases and if you got it right away, they live a nice, healthy life. There was a rule such as "This person could be my mother if and only if she was not male". Before you'd think it would be female, but I found out there is all kind of variations, so the rule is starting to get constraints: "Plus, she had to be at least 8 years older than me" - those types of constraints.

When I'm storing into the database, you start having other additional problems that you are starting to talk about, too. You might constraints between the data, so a lot of times you might end up with a generic set of tables for doing that, but that might be hard to report on and you might have to write something on that, but you might also need a layer too, to help maintain constraints that do your indexing and deal with some of the performance. It depends on how you're storing that. Usually, those become very domain specific. You can somewhat generalize that, but if you make it too generic, then you can make the reporting team go crazy. To support them, you might write views or create a layer that can put something out for the reporting format. I don't know if that answered your question, but I was trying to focus on it.

9. How do you validate the adaptive object model? Someone starts using it and to complete the feedback cycle, is there a way to validate the model?

Validation and testing - I guess I haven't mentioned that at all. That's very important. Say, for example, you are developing it in an Agile way. We should be having some iterative cycles and do lots of testing and getting regular feedback. The best way to validate it is we want the real business person working closely with them, but usually, in a normal object oriented system, you have real domain objects that you can sit there and write your unit test cases on for validating it. Now, with an adaptive object model, you rather than having real domain objects, you have instead one of these things like type objects, properties, strategies, rule objects, maybe some constraint typed things to help with this.

You might have a persistence layer to help the mapping between these. You have to not only validate that model, but then you might have to write support tools, so that the person describing the adaptive object model, the new types of entities with your system, that they can get feedback and validate it before you push that out into a running system. Just like we normally would do that in code, with our unit tests and other types of things, now we need to write ways that they can validate that model. Usually, the GUI - or some kind of editor, or some kind of way of describing the domain specific language - helps make sure that you give a way of doing some validation.

For example, with the invoicing language, we had to have ways where they put invoices together that they could run against the test database, they had stuff they knew was in it and could give it back in. Was that what they were expecting or not? So they could play around with it and say "Well, that got most of what I wanted, but how come I got these weird charges here? I didn't expect to get those weird charges for this math when I made these changes. Oh, I entered the number in wrong!" It is data, so if you put the number in wrong, this comes back to code as data. In fact, everything is data.

Ultimately, before you had an adaptive object model, you were writing it in code. That was being interpreted and compiled and then you were running it. Now, we are interpreting it and running it - basically compiling it - at run time, but we've put it at a higher level, so you need those types of checks before you go into a running system. Since you are, in a sense, inventing the language, you also need to have version control. Think for example in the banking. Maybe I've invested some money in the stock market and I figure out that I forgot to calculate that I did this investment last month and I have to go back and replay it. The rules for today could be quite different than the rules for way back then, so you even have to keep a version of your rules and how they change. You have to replay things or go back and figure out what happened and when.

10. That's a good point, managing the versions of the adaptive model. How do you usually do that? Is there a way to store the snapshot of an object model if there are any changes?

Yes, there are some pretty well known techniques. In fact, one thing you'll find out with adaptive object models is there some known techniques that we've learnt throughout history of software development, including programming languages, that apply well. For example, if they perform slowly, you use caching techniques. For versioning and history, Francis Anderson wrote some history patterns at one of the patterns conferences for an adaptive object model that he worked on in a telephony domain - that's another domain that can be very widely used for.

There you talk about adding version and your history information not only to your different rules and your different types of objects, but the data itself, so it can be associated with its version of where it was validated from. That way, it could be that the rule changed, so that this object is no longer valid, so you might have to go back and see what was it back then and then, if you want to update it today, you might have to apply new rules. There are some very well know techniques. You might have a begin date - end date associated with your specific rules, keep the history of when this happened. There are also other types of techniques that you can use.

11. Going back to financial applications, have you found that those types of clients are open to the idea of having the functionality and the behavior of the application able to be changed once the system has been deployed? Thinking in terms of security, are they concerned that the possible malicious individual would enable or would insert some rules into the system that, at least for a short period of time, would break havoc with the data?

That's an excellent question! Especially in the financial domain, people change because it becomes a legal thing, there are responsibilities, plus, when you are talking about people's money, people also change the way they are going. I can think of a good example: I was doing work in the research area, funded by a very large company and it was within the financial domain and certainly wanted to go into production, they were all excited about what we were doing because of the flexibility we gave them. But then, all of a sudden, the auditors came in and new rules applied because "You are going to deploy and this is about our money" and things along that line, things changed a lot.

The main thing is there is pretty much that some people within the financial domain have been forced to have flexible systems, but security is key - that's where the history definitely comes in, too. They want us to be able to prove that we can track every change it was made, why it was made, when it was made, by whom and also control who could change what. Before "who could change what" it was a little bit slower because it had to come through the business person, the financial person, go through a business analyst, ultimately go to code, come back and be validated. That process still needs to run, the difference is that they can do that much faster now. They can make those types of adaptations much more quickly, but they still need to be validated and controlled who changed what and when.

You still need the same types of checks and balances, specifically in the financial domain, but not only that domain. The medical domain is another. Anything that you are really concerned with. Security is becoming a big issue. You are right, if you make it easy to change, people can change it more easily, they can mess things up more easily, plus you might get a malicious person for storing in data and somebody hacks in and they are using SQL injection because you are on your intranet site and figure out a way to mess up stuff. You have to be concerned about that, too and how do you protect for that. The same rules apply, you just have to build the infrastructure there to do that.

12. What do you see as the future of adaptive object modeling and where it's going or what might be coming up in the future?

I definitely see people continuing to grow to do this. People are just forced to be able to build their systems so that they can adapt more quickly. One thing though: I think adaptive object model is an architectural style, a specific way of making a system flexible. There are many ways to do this and we are trying to document and describe the best practices. It's highly related to domain-specific languages, so one of my goals for the future is to really document the best practices, how to do it correctly, talk about the gotchas like the security issues or the database side and how to map that.

The main thing is I think we are going to see more and better tools for being able to evolve and build your type of flexibility in your system, including maybe even some good domain-specific languages that are already there, maybe some adaptive object models that you can buy if you are in the financial domain or in the insurance domain. Part of the problem, so far, is people are being very successful with this, but they are very proprietary with it because at this point they really see the competitive advantage, but ultimately is more and more of that happening and you see a lot of companies inventing the same thing, maybe a little variation on it. If that could grow and there could be good support for it, I think it would be very good for the future of it as well.