Sooo...
You're at your desk, staring at the line of SOA books. Your eyes wander over the boxes of SOA tools your CIO let you buy. Wow, how much money is sitting on that shelf?
Sure, you got those wretched acquisitions to cough up their customer data via a SOAP service, in one format no less. Not only that, it's even being used! By who again? Oh yeah, that cute girl from Marketing. Geez, you never would have thought this SOA stuff would get you a date- or at least a lunch with her.
Yeah, SOA sure delivers, doesn't it? Good thing you got those message header standards established. Yah, like, what-EVER!
What a pity many IT organizations do not get past the point of installing some tools and deploying a few services. Quite frankly, it's that rare firm that actually is getting solid and consistent benefit from their Strategic Enterprise SOA Effort.
Why is that?
I would like to, if I may, take you on a journey where you will suspend disbelief. That journey is AgileSOA. Like SOA is not buzzword enough, right?
Just to be clear, we are going to talk about the convergence of Agile software development practices and building applications as a collection of services. As you will see, the two mesh together nicely, as if they were made for each other. A little Agile gets your first services going. And those services will allow you to apply some more Agile, et cetera. The word that comes to mind is "symbiotic", but "most excellent" and "lovely" apply as well.
In most cases, a firm's SOA effort is a more hype-overloaded EAI effort requiring more standards, more process and more Big Upfront Design.
Well, let me tell you my friend, this BUD's not for you.
What goes wrong here is that enterprise architects apply age-old engineering practices to something that defies description, planning and upfront design. It's the full-scale confluence of business and technology. Such uncharted territory needs to be approached with the most agile of methodologies. After all, do you understand all the business processes? Do you know exactly which pieces of functionality can be separated out, allow for an unambiguous interface definition and can operate stateless and without knowledge of their invokers and therefore will be best suited to become services? DO YOU?
No, you don't. And it's about time you simply admit it.
Engineers are brainwashed with the doctrine that they have to have all the answers, at all times. Not knowing means discredentialization, i.e. getting a wedgy in front of the whole class. Saying the wrong thing is not as bad as admitting to ignorance.
Scientist however know what they don't know and accept that to be the case. In their quest for knowledge, they devise experiments to either prove or disprove their assumptions. A failed experiment is not one that proves their thesis wrong; it's the one that is inconclusive. The lack of knowledge about the outcome and correctness of approach does not preclude the scientific team to apply the utmost rigor in planning, executing and recording the experiment. The findings from each experiment are used in each subsequent iteration, until either an answer is found or funding runs out.
Similarly, the Agile methodology advocates Doing over Pondering. With SOA, as with most software design, there are so many unknowns that most people who endeavor it don't even know which questions to ask. Let alone having any answers.
So I ask you: how can you design, plan and implement what you do not know? You can't. So you might as well design around just what is in front of you and plan for half of what you know to be invalid tomorrow.
You can't learn what it's like to make, own and interconnect services from a book, you have to DO it. Only when those messages start flowing and trigger events will you understand. Only when you invoke the same service from two entirely different pieces of code will you understand the power of what you have created.
Do you need to have your message headers standardized to do this? Do you have to have your governance set up? Do you need to have an SOA infrastructure up and running?
No, and you know it.
But just sending wild XML without schemas over JMS sounds so.., so... impulsive! Yes, and isn't it beautiful?
So, now that I have you convinced that there is an alternative way to approach SOA, let's look at how you can actually get true SOA off the ground, using Agile/XP practices.
First, select a team of 4, 6 or 8 developers. The more varied their skill set, the better. Reserve a room for half an hour at the beginning of every day. This will be your AgileSOA project room where you will discuss lessons learned, designs and practices and where you will plan your next steps. Get an easel with a flipchart, so you can carry your artifacts in and out at will.
Select a mildly complex process flow from your company's core business. To name some examples, try implementing a simple loan application. A basic stock trading system is a good candidate as well. Remember, you don't have to solve the entire business problem, just make your solution do something that appears useful to your business types.
Chart out the simplest path through the flow, initially leaving out all exceptions, errors and decision points (the so-called Happy Path). Now go and implement the first pieces, pairing developers with the largest difference in skills and experience. This may seem unexciting, but I challenge you to build that first consumer of MSMQ or JMS messages, get a few running and have a producer successfully send some XML data to them. (That means ALL of them, not just one)
So what if this first step is trivial and mundane? Did you write unit tests for the producer and consumer, so that you can verify that the data comes across intact? Did your pairs develop together, without walking back to their own desks? No? Thought so.
Figure 1: Most basic process flow services
Figure 2: Add validation service
Next, introduce a few more Happy Path services. Try to mix synchronous, asynchronous and message-based request-reply. Be adventurous! And by all means, write your tests first. After all, it's much easier to write and refactor code when you have unit tests in place.
Figure 3: Expand validationNow that you have your Happy Path running it is time to introduce an exception path. This may require severing the ties between one or more producer-consumer pairs, and that's OK. In fact, it's great, because now you're experiencing how easy it is to change the behavior of your "application" while leaving most of it intact and running! At this time, you may want to add some metrics gathering pieces. Good data to collect are service response and latency times, throughput, uptime, message counts, etc.
Figure 4: Add validation exception
As you continue to refactor the services and the flow to accommodate the remaining or new requirements to finalize the business workflow, you will notice that it gets easier at every pass. You will also notice a chance in your team. They have now morphed into a cohesive blob of knowledge and understanding, almost moving and thinking as one. One would say they're "gelling"...
In and of its own, this exercise may not impress your CIO. But when you tell her/him the time within which your team completed it, plus how many times you actually changed the design, she'll think you have had a few too many cups... or she'll ask you to share the dope, dude.
...and that's exactly what you're going to do!
Now that you and your posse have set a precedent, you can move forward and undertake an actual implementation of new functionality (or a rewrite of existing stuff). You now have the experience and confidence to simply begin and work in close cooperation with the "client" to build what they have in mind. You no longer scowl, frown and sigh when they proffer the so-many-eth change. You oblige with delight, because your unit-tested code is rock-solid, your system scales nicely, your service-oriented approach allows you to deploy and redeploy pieces ad infinitum and you can do so with a running system.
As your service population grows, the need for some infrastructure and standards will emerge. Now is the time to identify services and formats that are used by many SOA participants. These become the de-facto standards, based on what you actually use and derive value from. Your services will be reliable, scalable and easy to maintain. This will attract new business users who now see the previously doubted promise of service reuse gain new momentum. Coupled with the basics of SOA governance, they will feel empowered to demand from their development teams to use existing services rather than implement their own.
And so, your SOA has come to life at last.
Think this is fantasy? Just ask our clients. Or even better, why don't you try it for yourself.
Yeah, you, staring at that row of unopened software boxes...
About the author
Carl Ververs is an SOA Strategist for ThoughtWorks, Inc. He speaks and blogs regularly about Agile and SOA. See http://carlaugustsimon.blogspot.com/