SOA Equals Integration?

Posted by Boris Lublinsky on Jan 04, 2009 11:42 AM

Community

Architecture,

SOA

Topics

EAI ,

Enterprise Architecture

Tags

Best Practices ,

SOA Adoption

According to the report by Jack Vaughn at SearchSOA, Yefim Natis asserted "SOA is integration" during last week's Gartner AADI Summit. In all fairness, Yefim did not mean to equate SOA to integration:

You can only do [SOA] in parts of a domain where you have control. Many companies are instituting SOAs, but they are doing so without a singular architectural blueprint for all IT. Some people are starting to try to federate their 'domain SOAs' based on agreed-to interoperability protocols and transports that span the full organization.

So, although using the world "integration", Yefim was effectively talking about "federated" SOA, where multiple SOA implementations can be started independently by different groups/divisions within a given company and later integrated/federated into a cohesive whole. Nevertheless, his comment immediately started discussion on Yahoo! SOA discussion list. The discussion was started by Michael Poulin who asked the question:

What can we do to slow down spreading such Integration SOA madness?

Steve Jones dismissed Michael’s question and asserted that the approach described by Natis is similar to a BSB (Business Service Bus)/DSB (Domain Service Bus):

• The BSB/DSB model I've talked about for yonks is exactly about the federated SOA model.
• It’s the MODELS that matter and the TECHNOLOGY that integrates.

Anne Thomas Manes’ response, on another hand, concentrated on the differences between SOA and integration:

Many organizations mistakenly perceive SOA as an integration strategy. But it is not. SOA is about architecture. To achieve SOA, you must rearchitect your systems. You must remove the deadwood. Every organization has too much stuff - too many redundant applications and data sources. SOA is about cleaning house. You will not simplify your environment, reduce costs, and gain agility until you reduce that redundancy... I think it's important to distinguish between integration and architectural activities. It's fine to use service oriented middleware to implement integration projects, but then you need to readjust your expectations... Service oriented architecture is hard work. It's disruptive. It's a political minefield. It involves going through the application portfolio and identifying redundant applications that can be decommissioned and replaced by a single service. But no one ever wants to open that can of worms. Many folks live by the adage, "If it ain't broke, don't fix it." There's way too much other stuff to do.

Rob Eamon joined the discussion with the observation that:

While I wouldn't say "SOA is integration" per se, I'd say that integration is one of the core values of the SO approach. Services have 1 or more interfaces. Interaction with services is via those (and only those) interfaces. Services (and other components such as service clients) exist in independent ownership domains. Those characteristics are the heart of integration. SO demands that one consider integration up front rather than as an afterthought.

Miko Matsumura shared his experience in Software AG by noting:

Working at Software AG/webMethods, who are in some ways culprits of the old integration world, we are seeing the demand for true federation, but it's not across the single dimension of interfaces. Moving from Integration to Federation to be sure, from interfacing systems to interfacing tribal organizations. Our strategic customers are looking for ways to manage cost, complexity, heterogeneity, siloism, tribalism, consultant-ism and vendor-ism. But they are doing so across business processes, schemas, interfaces, contracts, policies, profiles, assets, infrastructure, VMs, etc. It's the natural pattern of the regional power to rapidly create variation (in the name of agility) and it's the natural pattern of the central power to consolidate, normalize, govern and otherwise rein in the regional powers to the extent possible (and sometimes more).

Michael Poulin added the following to the discussion:

What is the difference between integration and interaction? Maybe this is the way to finally find if SOA is about integration or not. When we gather services into the orchestrated process, it this an integration or interaction? I would agree with "integration strategy is a side-effect of applying SO principles at the enterprise level" after we find the answer to my question above.

Anne Thomas Manes continued discussion by explaining the difference between integration and SOA:

Integration is driven by individual projects, i.e., taking lots of small steps, but not bothering with the "thinking big" aspect. If you combine SOI with strong application portfolio management effort, then I don't think the difference is anything to be concerned about. The execution of specific projects tends to be equivalent.

Rob Eamon’s comment reemphasized the importance of the enterprise approach to SOA:

I agree with point about focus. Focus on the right level for a given situation. Focus on the right services to be built/exposed, not about tying applications together. But I don't think that changes whether or not integration is a core part of SOA. SO principles are about defining services and their interfaces with the "outside" world. Again, I agree that integration isn't *the* thing to focus upon, but it is an SOA thing, IMO.

Steve Jones added one more dimension to the discussion by trying to separate architectural and implementation concerns of SOA:

... the argument appears to be more about what is integration, for instance whether process and choreography count as integration and whether more dynamic interaction models count as integration. I think that most people on this list agree that SOA is predominately a governance/organisational/business/thinking thing, but that there are SOA technologies which are related directly to implementation. One of the on going challenges in this group is the two different worlds of SOA.

Anne Thomas Manes continued discussion by explaining the difference between integration and SOA:

Integration is driven by individual projects, i.e., taking lots of small steps, but not bothering with the "thinking big" aspect. If you combine SOI with strong application portfolio management effort, then I don't think the difference is anything to be concerned about. The execution of specific projects tends to be equivalent.

Mike Nibeck quoted Zapthink to explain the differences between integration and SOA:

Zapthink has a very specific take on SOA and integration. They state the following:

• One goal of SOA - Integration as a byproduct of Service composition.
• One Goal of legacy integration: building Services to support this goal, NOT connecting systems to address a particular business need.

Their primary point being that in a SO architecture, integration is simply one of the steps or parts of a composition, and it no longer gets seen as a distinct and separate set of processes or technologies. In most cases, integration efforts are designed to somehow "join" two or more disparate systems. If however the point of interaction is a higher level business service contract, the individual integration points become less relevant. You will always have the need to interact with remote systems, and the lower level details will still be very similar to traditional integration efforts, but these efforts will exist in a larger context, the service model that will hopefully not be directly impacted by the individual integration efforts.

Loraine Lawson addressed the issue outside the Yahoo group discussion on her own blog:

If you haven’t been keeping tabs, this [relations between SOA and integration] is a hot-button issue for SOA. The argument is that SOA is an architecture and about so much more than tying things together... But the facts are these: Most companies aren’t getting into SOA for a complete rebuild. Most companies deploy SOA because it’s so darn helpful with simplifying integration... Although David Linthicum and others believe that agility is the ROI for SOA, many companies are realizing SOA ROI through integration... a lot of SOA people do themselves and SOA a disservice by disavowing integration as a real reason for SOA. Hey, SOA works for integration. Why not embrace that?... So, maybe instead of saying, "No, SOA is not integration," and then advocating a complete overhaul, maybe SOA experts could try this: "Sure, great! Deploy SOA for integration" then come back to me in six months so we can talk about what else you can accomplish using this approach.

So what really is the relationship between SOA and integration? From the architectural point of view, Enterprise application integration (EAI) is:

...the process of linking applications within a single organization together in order to simplify and automate business processes to the greatest extent possible, while at the same time avoiding having to make sweeping changes to the existing applications or data structures.

while SOA is:

... an architectural style promoting the concept of business-aligned enterprise service as the fundamental unit of designing, building, and composing enterprise business solutions.

There is nothing in SOA definition that states the requirement for an overhaul of the existing IT systems. On the contrary, the majority of the successful SOA implementations are based on the reuse of the existing applications (through integration) and introduction a fairly thin layer of (service) rationalization on top of them. The basic difference between integration and SOA:

Although the goal of both EAI and SOA is often similar - support of enterprise business processes with the existing application portfolio - they achieve it in radically different ways. EAI focuses on exposing the applications' functions through integration services, effectively exposing an existing application portfolio as an enterprise business model. In contrast, SOA focuses on hiding the existing applications and exposing a set of application-independent business services instead, projecting an enterprise business model on the existing applications portfolio.

From the implementation point of view, with the current advances of Web services as a technology for providing transport solutions, they are often viewed as standards-based integration solutions. This makes them an extremely attractive (and vendor-independent) alternative to EAI implementations. The introduction of Enterprise Service Bus (ESB) products make Web services-based, standards-based integration solutions, even more popular.

1. Back to top

   Integration is the anti-thesis of SOA

   Jan 4, 2009 12:21 PM by Jean-Jacques Dubray

   Boris: I find this debate disheartening as SOA exist for minimizing Integration. Integration became necessary as system of records needed to duplicate information across the enterprise. Because application architecture never allowed for "composite applications" we had to come up with a technology that synchronizes and replicate information. This is integration. SOA is about creating services such that a new solution does not need to build overlapping systems of record. It can instead consume these services, as a composite application. The need for integration is removed. Of course the confusion comes from the fact that both SOA and Integration use similar technologies and patterns. However, an integration endpoint is not a service. It only becomes a service in the SOA sense when it offers a normalized interface (i.e. it is the only endpoint in the enterprise that offers this service). I find this paragraph a bit misleading: "There is nothing in SOA definition that states the requirement for an overhaul of the existing IT systems. On the contrary, the majority of the successful SOA implementations are based on the reuse of the existing applications (through integration)". I of course deeply and totally agree with the entire sentence except when you say "through integration". I think it would be more precise to say "leveraging technologies and pattern similar to the one found in integration", otherwise, I think it keeps the confusion going. SOA has nothing to do with replication and synchronization.

   Reply

2. Back to top

   Re: Integration is the anti-thesis of SOA

   Jan 4, 2009 2:06 PM by Boris Lublinsky

   JJ, from my point of view the main benefit of SOA is the ability to hide existing enterprise applications spaghetti and expose a “clean” enterprise-wide model that can be used as a foundation for further enhancements, most notably enterprise-wide business processes. Such approach to SOA allows to for the most cost-effective transition to SOA. In this case companies are not forced to implement “big bang” overhaul of existing IT systems, while transitioning to SOA. This approach also means that the actual service functionality is implemented through integration of existing enterprise applications. I do not see anything wrong with using integration for this purpose, because they are, in this case, are part of a service implementation and should not be accessed directly, but through the service. SOA introduces a lot of new things, including service composability, but it should not throw away things that do work, including integration. I do not think the debate should be on SOA vs integration – it should be on the role of integration in SOA implementations.

   Reply

3. Back to top

   Re: Integration is the anti-thesis of SOA

   Jan 5, 2009 11:53 AM by Mark Griffin

   Boris, A good synopsis of the on going conversations. The following definition you used: "Although the goal of both EAI and SOA is often similar - support of enterprise business processes with the existing application portfolio - they achieve it in radically different ways. EAI focuses on exposing the applications' functions through integration services, effectively exposing an existing application portfolio as an enterprise business model. In contrast, SOA focuses on hiding the existing applications and exposing a set of application-independent business services instead, projecting an enterprise business model on the existing applications portfolio." It's a good attempt at contrasting the styles but I think a lot of us that came from the EAI world did hide the underlying applications from each other. We used CIM's to talk about services instead of the underlying data models providing by the vendors. These CIM's were focused on what we were trying to do as a business service rather than api specific data models. Yes at the last mile we still had to speak that native api that the vendor of Application A spoke but so do the so called agnostic services in the SOA world. Unless your service contains all the logic it needs to perform whatever business function it needs to do which seems highly unlikely for all but the simplest services. I think this comment of yours sums it up nicely. "I do not think the debate should be on SOA vs integration – it should be on the role of integration in SOA implementations." markg http://www.greylines.net

   Reply

4. Back to top

   Re: Integration is the anti-thesis of SOA

   Jan 6, 2009 4:49 PM by Eric Roch

   SOA is architecture - the architecture should be business driven. If a company has mostly packaged applications SOA will have an integration flavor. Also, I have never seen a SOA transformation that did not include some integration. I have never seen everything get thrown away with a fresh start and don't suspect I ever will. I have worked for several global companies and it is impossible to get every business entity in every country and LOB to agree to services, semantics, processes and so on. You have to break these problems into domains and integrate the domains or you will never get started much less finished. If you don't want to do integration then SOA is not your answer. Go to a global ERP template and force it down the user's throats. But I thought that is want we are trying to avoid with SOA and SOA for integration. It is also fine to use SOA for information access (entity services) and within BPM (consuming services or creating and exposing composite services). But SOA does not "equal" information access or BPM or integration - SOA is architecture! These arguments on what SOA is are pointless without some context of the problem to be solved. See the W3C definition of SOA and also my comments in this post. http://it.toolbox.com/blogs/the-soa-blog/what-is-soa-23569

   Reply

5. Back to top

   Re: Integration is the anti-thesis of SOA

   Jan 7, 2009 12:49 AM by Stefan Tilkov

   See the W3C definition of SOA

   I'm not aware of such a thing. Are you referring to the OASIS definition?

   Reply

6. Back to top

   Re: Integration is the anti-thesis of SOA

   Jan 7, 2009 11:10 AM by Eric Roch

   Web Services Architecture W3C Working Group Note 11 February 2004 http://www.w3.org/TR/ws-arch/#service_oriented_architecture

   Reply

7. Back to top

   Re: Integration is the anti-thesis of SOA

   Jan 8, 2009 2:24 AM by Stefan Tilkov

   Ah yes, I had forgotten that document even mentions SOA. Thanks. Personally, I find it sad that the whole W3C effort around this document ended without a formal result.

   Reply

8. Back to top

   Re: Integration is the anti-thesis of SOA

   Jan 10, 2009 10:39 AM by Boris Lublinsky

   More then that, the whole definition equate SOA with distributed systems and does not even try to address business side of SOA

   Reply

• Older >