InfoQ

InfoQ

Presentation

My Bookmarks

Login or Register to enable bookmarks for unlimited time.

The content has been bookmarked!

There was an error bookmarking this content! Please retry.

Recorded at:
Recorded at

Enterprise Integration–The Seriously Nasty Stuff

Presented by John Davies on Jan 06, 2012 Length 01:33:00     Download: MP3
     Slides
Sections
Enterprise Architecture,
Development
Topics
Spring ,
Java ,
Dependency Injection ,
SpringSource ,
VMWare ,
Websphere ,
Languages ,
Design Pattern ,
Programming ,
IBM ,
Application Servers ,
Patterns ,
Object Oriented Design ,
Design ,
SpringOne 2GX 2011 ,
Agile in the Enterprise ,
SpringOne ,
Companies ,
Agile ,
Enterprise ,
Conferences ,
Spring Integration ,
Integration
 

How would you like to view the presentation?

In case you are having issues watching this video, please follow these simple steps to help us investigate the issue:
1. Right click on the video player and select Copy log
2. Paste the copied information in an email to video-issue@infoq.com (clicking this link will fill in the default details in most email clients).
Note: in case your email client hasn't automatically picked up the email subject, please include in your email the URL of the video too.
3. Done.
We will investigate the issue and get back to you as soon as possible. Thanks for helping us improve our site!
Summary
John Davies discusses enterprise scenarios where Spring Integration fits and some where it needs additional help from solutions such as SWIFT from C24 Integration Objects.

Bio
John Davies is co-founder and CTO of Incept5, which is involved in implementing Visa's new capabilities and initiatives around the world. John's past includes Global Chief Architect at JP Morgan and BNP Paribas, co-founder and CTO of C24, later sold to Iona and then Progress Software where he was Technical Director. John specializes in high performance, low latency enterprise architectures.

About the conference
SpringOne 2GX is a one-of-a-kind conference for application developers, solution architects, web operations and IT teams who develop business applications, create multi-device aware web applications, design cloud architectures, and manage high performance infrastructure. The sessions are specifically tailored for developers using the hugely popular open source Spring technologies, Groovy & Grails, and Tomcat. Whether you're building and running mission-critical business applications or designing the next killer cloud application, SpringOne 2GX will keep you up to date with the latest enterprise technology.
  • This article is part of a featured topic series on Agile
not worth the time by Aravind Yarram Posted
Re: not worth the time by peter lin Posted
Re: not worth the time by Harsh Gupta Posted
Re: not worth the time by John Davies Posted
Re: not worth the time by John Davies Posted
Re: not worth the time by John Davies Posted
watched 40 min of video and almost 30 slides out of 43 by Harsh Gupta Posted
Re: watched 40 min of video and almost 30 slides out of 43 by John Davies Posted
  1. Back to top

    not worth the time

    by Aravind Yarram

    product sales pitch than related to integration

  2. Back to top

    Re: not worth the time

    by peter lin

    There was a little bit of product pitch, but overall it was interesting to me. Having worked with accord specification and other financial specs, I found the talk entertaining. Back when I worked on trading systems, a new developer asked "why doesn't everyone implement FIX and SWIFT protocol?" Just as John says, these specs are ugly and complex. What I found interesting is how they are using spring integration to chain together transformers.

    One could argue "why didn't john davies go into technical details?" Honestly, it's hard to get into details without diving into the specs. Do you really think that's going to appeal to the general public? I had to work with those specs for past jobs and honestly it wasn't joyful. Just about everyone that works with those specs finds them painful once you get into the nitty gritty. For example, try running .Net xsd.exe on accord spec and see what happens.

  3. Back to top

    watched 40 min of video and almost 30 slides out of 43

    by Harsh Gupta

    It was good watching a different aspect of financial systems. I wasn't aware of SWIFT/FIX/FpML etc protocols and after watching this video, i kind of started understanding the protocols & communication medium used by banks/traders. I think up-til this point he is trying to explain how to reduce the ugly details of these protocols/messages and is trying to promote Integrating Objects from C24.biz company. I think Integrated Objects api is paid too.
    Will watch rest of the video and update

  4. Back to top

    Re: not worth the time

    by Harsh Gupta

    Interesting point was when somebody in the audience asked "How to process 2 lakh messages?". What i understand is they read and process headers and then route the messages to different channels and they are using Gemfire where messages are replicated to distributed systems and each system then might process the remaining part of the message.
    Also he mentioned using C++ at one point. Increasing permgen size to load 5000 classes is eventually required. JaxB uses sax parser and is very fast but the object itself will require memory.
    Anybody's opinion on this part? Was expecting more on this front....

  5. Back to top

    Re: not worth the time

    by John Davies

    Every single bank on the planet has to process these sorts of messages, there are very few open source options here as for SWIFT for example everything needs to be certified and that costs roughly $1m in human effort to implement the standards every year.

    This presentation lasts 90 minutes, 60 of that was about integration in banking and the other 30 was a demo of a tool that anyone can download and use for free (from www.C24.biz). To use the SWIFT, ISO-20022, SEPA, FIX & FpML etc. in production you need to part with money, there is NOTHING in the open source realm that implement the SWIFT rules so I'm afraid you're stuck with sales pitches. This one is about as near as you can get to the open world as it was integrated with Mule, Service Mix, Spring Integration & Camel.

  6. Back to top

    Re: not worth the time

    by John Davies

    Thanks Peter, I try to entertain.

    It's always a dilemma to know whether to dive into detail and walk through the code or cover the problem at a higher level hoping that the people who want details will engage afterwards, these integration issues are so specific to large enterprises (HL7, Accord, SWIFT, ASN.1, ISO-8583, ISO-20022 etc.) that unless you know what the problem space is you're going to end up walking out after the first 15 minutes and that's not good for the conference.

    I have had a number of emails (to John dot Davies at Incept5 dot com) asking about this talk and I'm always happy to delve into details, supply code examples or reference to production examples.

    I'm doing a shorter (60 minutes) version of this talk (without the "sales pitch") at SpringIO in Madrid on the 17th Feb if anyone's in the area.

    -John-

  7. Back to top

    Re: watched 40 min of video and almost 30 slides out of 43

    by John Davies

    Thanks Harsh, I think you got the idea, my goal was to educate the attendees and in this case the listeners about how we deal with the complex messaging and protocols in the financial world.

    I've had a chance to do a little work outside the financial services and I found it very useful being able to apply my knowledge in these areas. The "Integration Objects" product was created as a result of our pain and those of people I worked with over the years (and decades). We must have done something right because it's been very successful and now used by the majority of the world's largest investment banks.

    It is free to download and use but you will need a runtime license for production use of the complex libraries.

    Regards,

    -John-

  8. Back to top

    Re: not worth the time

    by John Davies

    For those that are not familiar with Indian numbers 2 lakh is 200,000 :-)

    If we get something seriously complex to process then we often "pre-parse" the messages and then distribute them, an example would be processing a large number of complex SWIFT messages where we can only parse, validate & transform a few thousand per second per server. If we need a higher throughput we can simply distribute them but sometime it's more efficient to parse part of the message first and then distribute them.

    When we need seriously low latency then we simply have to move to C/C++, our very fastest FIX engines, used by the lowest latency traders are all C/C++, it's the predictability as well as the latency that rules out Java. By the time we get to this stage we're into FPGA, RabbitMQ, ZeroMQ and LMAX etc.

    Send me an email if I can be of further help,

    -John-

Educational Content

Eventually Consistent HTTP with Statebox and Riak

Bob Ippolito explains how to solve concurrent update conflicts with Statebox, an open source library for automatic conflict resolution, running on top of Riak.

Java.next

Erik Onnen attempts to demonstrate that Java is still the best programming language for the JVM if simplified idioms are used along with proper tooling.

Evolution in Data Integration From EII to Big Data

Approaches to integrating data are changing with emergence of cloud computing.

Winning Hearts and Minds: How to Embed UX from Scratch in a Large Organization

Michele Ide-Smith presents the lessons learned in the process of introducing UX principles and techniques into a large organization through a series of small steps.

LMAX Disruptor: 100K TPS at Less than 1ms Latency

Dave Farley and Martin Thompson discuss solutions for doing low-latency high throughput transactions based on the Disruptor concurrency pattern.

Thoughts on Test Automation in Agile

Rajneesh Namta shares his thoughts, experiences, and some of the critical lessons learned while implementing software test automation on a recent Agile project.

Actor Interaction Patterns

Dale Schumacher presents several patterns of actor interaction that can be used in collaborative programs written in any language.

Scalaz: Functional Programming in Scala

Rúnar Bjarnason discusses Scalaz, a Scala library of pure data structures, type classes, highly generalized functions, and concurrency abstractions to perform functional programming in Scala.