Transcript
Jim Gough: We're going to talk about APIs for agents. Really, what we've been doing at Morgan Stanley over the last year is really rethinking our API program in light of where the industry has gone. What we're going to cover today. We're going to do an introduction to MCP. We're going to talk about the core concepts of something called CALM. We'll get into that in a little bit. This is really the two things that we've paired together to give us platform architecture that enables us to move really quickly for the types of challenges that you'll all be facing. We're going to look at the demo from last year and really add MCP and other things to it. There are a lot of demos in this session. We're going to introduce governance and controls, that's a really big thing for us, and how that fits into both CALM and some of the work that we've been doing around gates. We'll then talk about how we're going to scale platforms, and how then we start to change for various different things that we're going to encounter.
Background
Who are we? I'm Jim. I'm a distinguished engineer at Morgan Stanley. I'm also a Java champion. All the examples that you will see are in Quarkus, except for the Python ones. I'm also the co-author of "Mastering API Architecture," another book, "Optimizing Java." I'm also the architect of Morgan Stanley's API program.
Andreea Niculcea: I'm Andreea. I'm the one that does the work. I'm a vice president at Morgan Stanley. I've been working with Jim for the past five years, building the API program. Started as a software engineer, focused a lot on onboarding tools. For the last two and a half years, I'm on the platform engineering side. Now I lead our self-service capabilities in the team. A lot of what we're going to see today is what I do in our program.
Demo (In the Recent Past)
Jim Gough: Last year's demo. About, I think it was a little bit less than a year ago, I spoke at QCon about API architecture. We had this great idea, and that was to use Architecture as Code to deploy all of our services, all of our infrastructure, across everything we do at Morgan Stanley. Last year, I was sat here going, yes, we're definitely going to do this. We had done zero things at that point. That was the beginning of the journey. Fast forward a year, we've now deployed over 110 APIs in production using Architecture as Code, continuous deployments around a very unique combination of architecture, platform, and security. We're going to talk a little bit more about that.
Andreea, if you want to flag up the demo. As things have changed, or really changed, so thinking about last year, I don't think I even knew what MCP was. I don't know if there's anyone else who would be willing to admit that at this point last year. It's gone really crazy. There's been a real big drive in terms of the change to API programs, in terms of the way we think about things as well. What you've got here is a small, almost preview, it's the Netflix trailer, into what we're going to cover, which is how we've gone from a traditional API program to supporting MCP, to then looking at future protocols, which this is using A2A in the background.
Introduction to MCP
Introduction to MCP. If you've not come across this before, there'll be a few questions that you'll want to ask. Straight away, I'll start with saying, it's an open protocol for connecting LLM-based applications to tools and data. That's the key headline story here. It uses a process of discovering what it's got available to it, invoking it, and validating in the background. There's a cycle of interaction that happens here. The key thing for me, as somebody who really believes in OpenAPI and API specifications, was the question, what does this give us above what you can get from APIs? If you think about it, we've already got descriptions around what APIs can do, what they can surface. That's been around forever. I'm going to say this now, I've not seen anybody get really excited about an OpenAPI spec in the business. There are lots of people jumping up and down about MCP specs and what that means.
The protocol itself, super simple. Protocol is just client-server interaction. It doesn't do very much above that end-to-end piece of communication that we're really used to and that we've seen in many different patterns and protocols in the past. Security and governance are also built in that are now tending towards first class. What is it at the core? I mentioned tools. That's about structured operations to perform a task. Think about your OpenAPI endpoints being more, this is what your API is capable of. Then the tool being, ok, this is how you can piece them together in a way that's reusable, helps you solve something that you're doing within the business context. It also includes two other things that are often overlooked, and this is maybe where it slightly differs from other protocols. It introduces something called prompts as well, which are reusable, parameterized instructions of patterns that help to guide the behavior. That becomes really important when we start to think about lots of things operating together. It also includes resources as well, so being able to bring documents and data to the agents to help provide that overall context. At the core of it, it's really not too difficult, but it gets pretty difficult quite quickly.
Now for me, I think it's important to answer what is really driving the MCP hype. I don't say that in a negative way or in that we shouldn't be thinking about this. It really is something that people can relate to on the business side. It's something as a developer you can expose really quickly, maybe without much thought if you're not careful. It's very easy now, LLM coding tool. Go away, say, wrap an MCP server around this thing, or even generate me an MCP server from an OpenAPI spec. It's quite easy to expose these things. Governance and everything else comes in with that in terms of control. The demand signal that we're seeing from the business is, I want to talk to x. Whether x is technical products, they want to talk to their trade information, the databases that we have large amounts of information stored in, or even just being able to query things that they didn't even know exist within your technical estate.
That's all using natural language, so being able to type in in, essentially, English, what they're trying to do, and then have the agent figure out what on earth they're trying to imagine. I always think when I get to this part of the slide, it's really good that we've chosen something that's really unambiguous for technical information, like English, to be able to get access to our various things. It's really new in terms of 2024, November, MCP was announced. 2025, OpenAI started to adopt MCP. That's around the time of QCon last year, so really brand new. Then GitHub got on top of it, and everybody, as a developer, was starting to see this in their day-to-day, and starting to see what this could really mean. In terms of later on, remote MCP became the big thing. That's really where we're going to talk about today, is remote MCP servers.
Where's the complexity? I've said it's easy. I've said it's like OpenAPI, but not quite the same. The complexity is less about the actual protocol itself. The protocol could be your dumb pipes, that's the plug for APIs, they're dumb pipes with the smart things that happen behind the scenes. What you've got is effectively the complexities around how your agents orchestrate around this new information. Tool selection, or having one to five tools is really easy. They've got a description. They've got a name. It's really easy to pick from those what you're trying to do. As you expand that collection of tools that are available to you, very quickly you start having overlapping definitions. It's that English prose again. How do you start to disambiguate between one tool that does something very similar to another? Then cost is driven up based on that, because you end up having to pick things.
You're even going to see in our very simple demo this chatty nature that you get through the protocol, which then means that you're driving up cost, because everything uses a token. Reading in those definitions, reading in the actual descriptions, but then also the responses can be quite large as well. We start to see this spiral quite quickly. What that's led to is specialized gateways and control planes. Where we have API gateways, and lots of nice talks about what they shouldn't do. They shouldn't know about business logic. They shouldn't know about this. This actually introduces a reason when they need to. That's also changing within architecture. It becomes more about your interaction design. You've got the protocol on one side, but then something almost intermediary needs to know a little bit more. That's MCP.
Core Concepts of CALM
What I want to talk to you now is about how we've been powering a lot of our demos that we'll show today, but also how we power our architecture deployments in Morgan Stanley. This is an open-source project. It's from FINOS, financial open source called CALM, which is Architecture as Code, basically, if you were to look for it in the repo. It's got several key components that allow us to build an ecosystem that's being driven not just by finance, but other contributors that are looking for that same kind of thing. It's a little bit more than C4 in terms of what it adds. The core model, if you think of the core model as being a JSON schema, you won't be going too far wrong, that allows you to model boxes and arrows. On top of that, you can start adding more typed information that then becomes quite important.
We'll show some of that typed information as we go through. You've also got a CLI, so CLI tooling that enables you to operate with CALM. We've got what's often called as a superpower, which is patterns. Think about in your organization how many times you redeploy the same thing over and over for different use cases. For us, we've got three or four patterns, I think, in total, and 110 deployments on those patterns. Again, setting out guardrails, setting out reuse, and make it really easy for developers to do the right thing quickly. We've also got a tool called CALM Hub, which we'll show. This is all open source, which allows you to model. It becomes like the Artifactory of your architectures, your patterns, your controls that live within your repo. It's got enterprise-grade features that, again, we don't sell out. We give you that in the product, and those have been engineered by various folks in the community. Of course, it couldn't be anything without some AI integration as well, so we've got various different prompts and chat modes that you can use with this alongside your IDE.
Some concepts that will help you with the demo. You've got the patterns, which are your templates, essentially, of how you want something to look. If I was a developer, I would be going, I want to build a new API. I'll grab one of Jim's patterns and create an architecture. That architecture now represents my specific requirements on how this should work. In the pattern, we convey, essentially, our platform opinions, and the architecture allows developers to specify those opinions. If they try to do other things, they're not able to. That gives us that control around what we deploy. That wasn't available last year as controls, so that then allows us to start building in key things, and we'll look at those as we go through, add some controls into our architecture and into our demo. I've got two things that live quite closely together, which are templates and bundles.
Templates are ways that you can specify what you want an output file to look like. If you've used things like handlebars, it's very similar to that. You define a template that takes in an architecture, pops something out on the other side. If you're thinking, I've got Terraform, I've got kube commands, I've got Helm charts, I've got Kustomize, you could do all of that, because those would live as your templates. The architecture pops into that, and you follow the same process that you've done before, just with this extra bit at the front. More recently, we've introduced something called decorators, and I'm calling those out because we will show those a little bit in our demonstration, because we found that architecture is just getting too heavy. People are adding lots of things in there, like metadata, information about just everything you want to know about an architecture. What decorators enable us to do is separate those things out a little bit, so we end up with a nice sequence of things, rather than just one big blob of JSON, which nobody really likes, to be fair.
Scenario 1 - Hello MCP
Let's have a look at our first scenario. We've got about five scenarios that we're going to do as demos. The first one is building on just having that simple API in the background, and we're going to add an MCP server in the way. We've got our agent host, which is Claude. What we've actually done in the background is we've taken the MCP and API architecture and deployed those things.
Andreea Niculcea: We're starting with our CALM architectures. We have two. We have our Trades API service, and we also have our MCP architectures. These are generated from their own patterns, and they define the intended state of our system. Now, in this example, we have something that we call composite patterns, because while the REST service is a system of its own, our MCP architecture references that one. We also have our CALM template, and this is how we're actually going to deploy connectivity. This is where we have our deployments, our services, ConfigMaps, anything that we need to actually set up this connectivity. On the right-hand side, we're going to do that with just kubectl apply in a minikube cluster. If I go back to our demo in here, you can see that everything is already running. There's some port forwarding going on, because I've got a secure tunnel to Claude.
Claude is our agent, and this is where we have set up a custom connector. If Claude comes up, and let's see. There you go, new chat started. Claude is thinking, is trying to figure out, how can I get your top 10 Vodafone trades? It found that I have a trades tool. It loads the tools, trying to get some information from them. Hopefully, eventually, it will start actually making some calls to our services. It found a GetTrades. There's no results for VOD. Let's try a different name. It figured that the first call that it made didn't actually quite work. There's more going on. It also tried VOD.L. It tried Vodafone. None of them worked. Thinking further down, it actually now figured a whole list of instruments. It found that we have a prefix, which is LSE, and now it found our trades. This is a very simple example of how we have deployed, based on Architecture as Code, Trades MCP Server that points to our API, and Claude connects to it.
Introducing Controls and Governance
Jim Gough: That's really cool. Even within that one question, I imagine your users are not going to type exactly what you want them to. You end up in that situation where it's now having to go backwards and forwards just to figure out what you're even trying to do with a simple query. That's something nice and easy, running without security, running without very much in the way. Those are the kinds of demos that you get to and it's like, "This is AI for real." What about when we start thinking about controls and governance? That's something that is really quite a big thing for us. In CALM, last year we had the same kind of question. It wasn't about MCP at the time. It was about security. It was, how do we introduce security architecture, security constraints into our patterns and architectures to make sure that they're followed and evidenceable as well?
We ended up building in something which is a model for non-functional requirements. The non-functional I've got here are an MCP, an observability, performance. There are many other things that you could add on here. What we're saying as an open-source community is we are not the experts to necessarily define those. I keep getting told I'm the API expert and I should define those. Other than that, it's an open model where people can contribute. Now based on top of that, we realized quite quickly that standards were a key concept. CALM is quite open by design, but in terms of in your organization, you probably want to make it more locked down and have things that you explicitly want to see on every single node or relationship that you define. Standards, although they were created as part of controls, they became part of the wider ecosystem, that ability to keep adjacent schema as a baseline effectively.
Now based on that, we have two things. We have a control requirement and a configuration. That enables us to specify, again, as adjacent schema, what you're expecting to see as part of your architecture. We're going to do an example where we look at actually putting a guardrail in place for MCP and looking to deny certain symbols. We don't want that information to come back. You can start to think about different situations where you want to do things like this. Control requirements can also be around network segmentation. We've used those quite heavily within our designs as well. The configuration specifies how you're going to meet that requirement. That can be different depending on your architecture, depending on how you actually go ahead and write that specific implementation.
Scenario 2 - MCP Guardrails
Let's jump back to the demo where we're going to now add in a guardrail on the MCP endpoint.
Andreea Niculcea: Scenario 2, our services are running. Let's take a look at the guardrail that we're introducing. We are going to introduce an active MCP symbol restriction. What this has, it points us to our requirement. What we're looking at is the actual configuration. We are saying that on the MCP server, we will add a list of denied-symbols. What this will do, it should stop us when we go back to Claude and say, get me my Vodafone trades again. The way this works is we have a snippet of our architecture in here, our trades and API architecture. We can see the MCP server. We can see a few details, like the image that's running, the port, as well as the controls. This is how we are connecting our defined controls, the requirement, the configuration, and where exactly they are enforced in our architecture. We've run CALM template again.
We get a new version of our infrastructure. We have our generated denied-symbols-configmap. This one is coming from the configuration that we've actually linked in our architecture. What actually happens here is that we have architecture. We have our control. CALM template is going to generate us these kube resources. We can see the generated ConfigMap. Then on the MCP server itself, we have Java code written that looks for this particular ConfigMap runtime. In the first scenario, there was nothing there. There were no symbols to restrict. This time, we're applying it. Our MCP server should hopefully be restarting soon. We can see in here that the MCP server is configured. Then it should start looking for this list. If we go back to Claude shortly, we are going to see that we're no longer allowed. Deployment successfully restarted. Probably going to need to do port forwarding again because we have got our new port starting up.
Now if we go back to Claude and let's say, get me my latest Vodafone trades again, please. We have to almost trick it to say latest or again, because if you don't, it'll use the history and it'll say you already have your trades in here. GetTrades is being called. Interesting, our Trades Connector is now returning an error that your symbol is restricted and cannot be queried. You'll want to contact your compliance team to find out why this has happened. This is just a very simple example of how we introduce guardrails, services. It's useful, but it's not sufficient. The complexity comes when we think about enforcing consistency as our systems move throughout our different environments, from development to staging to production.
Gating Deployments
This is where deployment gates come in. Let's talk about our platform a little bit because our platform is not just tooling and it's not just infrastructure either. The real value of a platform comes when it can remove decisions that our developers have to make repeatedly and independently. These are things like what API standards to enforce, what security controls are applicable. How do I implement observability? What are the audit requirements? All of these things, when allowed to be exercised by our teams, usually result in two things. They increase cognitive load for our developers because now they have to think about the infrastructure, not just deliver the business logic, but also the architectural consistency across our organization will degrade over time. This is where deployment gates and self-service come in. What we want is to provide our developers with a paved road. We want to encode institutional knowledge into our platform, and we want the teams to rediscover it, not redesign it every single time they want to use it.
Self-service requires guardrails. It has to tell us what can be deployed under what circumstances and to which environments. What we're proposing here today is that we achieve that by adopting Architecture as Code in CALM to encode our controls and enforce them once. Regardless of how many deployments are going through our platforms, what we're going to get at the other side of it is repeatable and auditable deployments.
Scenario 3 - Deployment Gates
In scenario 3, then, what we're going to introduce is this idea of an architecture deployer, and this is where we're going to enforce a few gates. We're also coming back to CALM Hub, and Jim mentioned this one earlier. Let's start by looking at CALM Hub, because while we started to adopt Architecture as Code in our team and started to have hundreds and hundreds of deployments, what we figured out is that only Jim likes to look at JSON schema and everyone else needs something visual. This is where we can see I've got a couple of namespaces. This is just some logical group of artifacts. What we have is architecture as patterns, business flows throughout our architectures. We can link decision records to the different patterns. What we're looking here is at a version of our Trades MCP architecture. We have Claude as our agent. We have the Trades MCP Server with the controls, the MCP guardrail that I just showed in scenario 2.
We have our Trades API, our Kubernetes cluster, a lot of the details. We also have control domains. Actually, before I move on, let's take a look at those. We have our MCP controls. This is the one that we saw in the JSON schema earlier with our deny-symbols and example configuration. For this particular demo, we are going to introduce two new controls. We have network micro-segmentation. We want to make sure that default-deny and micro-segmentation is set up in our minikube cluster. We have an example of whether ingress and egress is allowed. We also have a security control which is the permitted connection. Now that default-deny and micro-segmentation are on, we have to explicitly allow connectivity between the elements. One more thing is that we are working on a deployment feature for CALM Hub which is currently not present here, and this is because we have our patterns.
We are authors, like Jim published them and all of our teams can discover it, validate against them, use them from a single source of truth. We have our architectures, they are versioned, and these are valid architectures that are proposed for deployment. What we're missing is the deployments to actually know what is deployed across each environment and to track the whole life cycle.
Let's go to our demo. In this demo, I'm going to show almost two different paths. We're going to do a non-compliant check and see one of the gates failing, and then we're going to fix it and move to the second one. The first thing we want to do is generate our deployer from our QCon architecture. Now I've introduced a rogue architecture in this particular demo in here. We can see our deployer and we can see some of the configuration. Let's see if we're following an approved pattern of our platform. We are not. We can see in here that we've got our schema and it's referencing the QCon pattern. We're not working with the QCon architectures today. We're using a trades-api-and-mcp architecture. Gate is rejected. This is the kind of feedback that we'd usually get in a PR pipeline. We want to give this feedback early to our developers.
Let's take a look at an approved pattern now. We have our conforming architecture and it's referencing our trades-api-and-mcp. What happens in the background here, we're doing a call to our local CALM Hub, pattern exists, we can move on. Second gate is architecture validation. This is where we are reusing our CALM CLI tooling. If we are to look at the non-conforming architecture, we can see in here that we have a couple of nodes. On our Trades API, I'm missing value. I forgot to fill in this placeholder about which version of the image we're running. This is the detailed architecture that I was mentioning earlier with respect to our composite patterns, how it's referencing. We have our cluster and we're also missing a few controls on our relationships. We haven't allowed connectivity from our MCP client to our MCP server. If we do calm-validate, you can see in here that things are failing.
We have two types of outputs. We have a JSON schema validation output. This is telling us you are missing controls on one of our relationships. We also have a spectral-based validation that we've baked in our tooling, which says, you have not filled in all the placeholders. Placeholders are really details that we require our teams to provide to us. This is how they want things configured. How do they want their rate limits set? Which version of the API you're running. This is the kind of information that we'd usually expose as placeholders. Our gate is failing. Let's look at the conforming architecture this time. We can see we have our MCP client. Our Trades API is referencing our image this time. We have our relationships. We have controls on these relationships. We run calm-validate and our gate has passed.
What I've shown so far is a very simple example of almost like build time validation. We want this kind of feedback to come quickly for our teams. Before we go to non-prod, we might also enforce gates like smoke tests and API standards or entitlement checks. Then we want to push our service to production. There is increased scrutiny. We might want to change controls. We might want to actually run some check against our infrastructure. We want to revalidate our architecture because it's got different details in there and different placeholders. One thing that we have really pushed towards is security reviews. This is where in our team, by adopting CALM, we are collaborating with security departments to build things like drift detection, to build automatic security reviews. Now our service passes all of these gates and we want to push to production. We're happy and we might do something like traffic health checks with synthetic monitoring, for instance. What I've shown so far here is not technically challenging, but it's maybe like a cultural shift. We don't want governance to be around the deployment. We want to encode our controls. We want the platform to enforce them, regardless of whether it's the first deployment or the hundredth deployment. Governance is in the deployment process itself.
Scaling Deployments and Operational Change
We've seen how we deploy safely and how we govern at scale. The other theme of our talk today is MCP. MCP introduces an interesting shift in how our platforms have to behave from an operational perspective. Perhaps traditionally, when we think about our platforms, we imagine, client makes request, service process it, we get a response back. This is a transactional model. Our capacity planning, SLA modeling, everything is about request per seconds, average latency. Agents don't behave like that. If we think about a realistic example for Claude in an enterprise environment, let's say, we would probably ask, Claude, get me yesterday's trades and figure out which one of them violate my risk policies. In this example, Claude would need access to our Trades MCP tool, but it would also probably need portfolio exposure. It would need trading analysis. It would need different risk policies and different ways of doing compliance checks.
Suddenly, the simple question has gone from being a few tool calls to dozens, and chains of tools. Claude might not get things right the first time. It might have to do retries. It might have to call tools in different orders, partial failures. Really, when we think about scale, the complexity is not in the MCP protocol itself. What I'm trying to convey here is that if we think about a REST request in comparison, we have a lot more responsibility in an MCP tool call because we have to do things like JSON schema validation, paginations. We might have to do output shaping, depending on the results that we're getting from these agents. We have to manage workflow. What we want for our agentic systems to be operable is we want well-defined patterns and automatic controls built into our platforms that can be enforced.
There is another benefit to using Architecture as Code, which we haven't discussed yet, and that is platform evolution. We all know that platforms evolve. Our opinion changes. We want to do things quickly, react. Even more so that we're talking about agents and how our platforms can evolve. This is where in our team we've come up with something that we call the operational rollout. Let's say we have to react to a Log4Shell event or we have to do a security patch across all of our Envoy proxies. What we don't want to do is create hygiene programs and rely on our 100-plus teams to initiate their deployments because we have to change our platform opinion. This is why CALM Hub is really instrumental in this ecosystem, because CALM Hub gives us the golden source of versioned architectures. We know what is deployed on our platform at any given time.
Because we know that, we can then provide a new platform opinion in the form of a CALM template. As a platform, we can react to changes quickly and safely. What we get on the right-hand side then is our updated platform. In our team, we run hundreds of operational rollouts every month. Only since the beginning of this year, I think, I have been leading an exercise to perform zero downtime infrastructure upgrades across our 100-plus API platforms in production. The way we've achieved that is by providing a new platform opinion. We've provided a new platform opinion that removes a cluster as a routine configuration. We are not touching the integrity of our user configurations, so we can redeploy them. We'll go through all of the same controls again, as if a user initiated this check. We have our new platform opinion. We redeploy all of our architectures. On the right-hand side, we have client requests running on three clusters out of four. We've isolated a particular cluster. Now to bring you back in rotation, we've done the exact same exercise again, but we brought that cluster back.
Scenario 4 - Platform Upgrades
How does this look like for our scenario 4? What we're working towards here in our trades and MCP example is really CALM templates and CALM bundles. Let's go back to our demo. If we start, what we want to look at is pod limits. We just discussed why it is so important to have platform opinions, and especially when we talk about MCP. We can see in here they were doing a kube describe on our trades-mcp-server. We have no resource limits set up. How does the platform team react? The platform team provides a new bundle, and that's my bundle v2. We can check the diff in here, and you can see that we're adding some new platform opinion. If we are to apply that, we'll do another CALM template against our conforming architecture. We can see that when we do a diff now between our MCP deployment, our platform opinion has been applied.
In these examples that we've been using so far, the combination of CALM template and kubectl apply is almost like our self-service mechanism. What we are doing here is we're going to be starting our MCP server, and hopefully, we are going to see that the pod limits have been applied. This is a very brief example, but gives us an idea of how we can evolve our platform opinions without relying on our teams, without introducing too much bureaucracy, and taking control of how we evolve our platform. Our deployment has rolled out, and now if we're doing another describe, you can see that our limits have been set up successfully.
Rapid Platform Adoption
Jim Gough: That's really important, that ability for us to be able to update things all the time without our users having to get involved. If you think about controls for MCP, there's a new scare in the news every week about something that's been exploited, or a particular unexpected way of something working. While we continue to have that business value, we can continue to add controls in as our platform evolves. There will be cases where the user might have to make some changes, and that's where we have versions around patterns and architectures that would start to block something being deployed, for example, if it didn't meet that minimum requirement that it needed to meet. The other good thing about all that put together, like I think the first API we released took about two years to actually get that hello world, because there's a lot of different hoops and things you have to jump through, architecture bit to build out for the first time.
We eventually got that down to it being six months for every new API, which is still a long time. With the stuff that we've done more recently with CALM, we've managed to get that down with some of the automatic security approvals to more like one to two weeks. If you think about the time that that saves in terms of getting to market, that's absolutely critical. This leads us on to rapid platform adoption. I sometimes call this adaption. I'm still not sure if I've typed this wrong when I created the slide. There is now a real demand on us to change things very quickly. We also want to do that in a way that's safe and controlled.
In terms of artificial intelligence, it's really driving bleeding-edge adoption. If you think about Thoughtworks Radar, our typical traditional enterprise adoption is when something is already towards the end. It's in the adopt phase, and everybody's really comfortable with how it works. We know we can integrate. We know there'll be lots of vendor support and everything else around it. It might even be that there's somebody specific you can go, they've solved our problems three or four times before. Whereas I think what's happening now is that we are getting into this point where we're having to shift how we think about these things. We're having to go to try things that are in assess, or even trial and assess. That's now our standard for new stuff, which is also really risky. Some of it's going to work. Some of it's going to be outdated. Who knows if MCP will be around at the end of this year?
It'd be around, but is it going to be the key thing that everybody wants? Don't know. Our jobs is to try and keep up with that, to try and keep up with those capabilities and almost layer those things into what we already have. We need to have an ability to do short, quick feedback cycles in production or in non-production first, then production to verify that that is actually going to give the value that we expect. This is really driving a lot of change in traditional architecture models where you would sit in a room or try and get together with controls and governance groups. You're actually having to put these things in together on the fly. When it comes to things like network segmentation, which we hinted at in there and was a big subject of the talk last year, that is such a risk mitigation because you can start to constrain and bring things into a pod level isolation where you know that if something is compromised, it's fully isolated. Because there's not just us moving quickly, there's all the libraries that we're using, dependent open-source projects, supply chain that comes into this. Things like that become real key controls for you being able to adopt effectively.
This is a bit of my plug for APIs. APIs is the building blocks. They're still important. They're still that stable contract layer that sit or can sit behind all of these new tools. AI is going to be fast moving. There's new models, new prompts. There's new tool use and patterns of tool use shifting. There's things like skills being introduced. You just have to pick up anything. You see this happening, but you see it in the public domain as well. If you watch the Super Bowl from last year, and it's quite eye opening in terms of how much this has been pushed into general public and business domain. If you can anchor your experimentation, anchor your change behind APIs that are well robust, they follow all the best practices, they're a stable integration point, you've got a really good foundation to be able to adapt to things.
That's what we've been seeing with our platform. Our platform last year when I was talking about, ok, we need to do APIs in a way that scales, we need to do it in a way that's controlled and has our ability to roll out operational change, we weren't thinking that we needed to have something that would adapt to support this type of model. The reality is that it can. You want to keep your controls where they belong, APIs or API gateways. Interestingly, you will notice that we put the guardrail for the Vodafone filtering in the MCP server just to tease out this point. Like, should it really be there or should it be in the API? It's in the API. If you add something later, which we're going to do, you wouldn't need to add it again. If you're going to build this stuff and maybe throw it away quite quickly, that starts to become a point of, where do you place these features in your architecture? I think adapters will keep evolving. I call them adapters, but they still have a lot of complexity and a lot of work going into these. It's MCP today. It could be agent-to-agent tomorrow. APIs remaining as the key building blocks is a safe bet to have.
We're going to have a look at a demo of agent-to-agent, and we'll start talking a little bit about how that works. They are fundamentally different. Actually, this comes from building small examples. One thing I will say is Andreea is showing you all the tutorial that we have here. We are going to be putting that in the open source so you can run through it with a much more detailed description. You can play around with these things for real in real Kubernetes clusters. Having that available to us as a platform team is really important. We want to be able to experiment and try things out. Looking at that combination of MCP and A2A, we built an example, which I think if I was building it again this week, I'd probably design it architecturally a bit differently. That's because of this. MCPs are much more around making tool calls and having that standard way of interaction, whereas A2A is a little bit more about, you take away the idea of something calling a tool, and you've got more individual systems that exist that coordinate together.
Sort of go, is anything capable of doing this particular task? Discover, look around, find an agent that you can collaborate with. The model here is that there is that ability to do the discovery. There is that ability to have message exchanges that build on tasks and return messages. I was looking at this and thinking, is this like Akka, essentially, with a bunch of other things built on top? Maybe it is. There is certainly some parallels in the way that this looks. It's, again, more driving the change in what we're seeing, which is moving more away from tools to agents. You can see that within conversations that you'll have, especially at an architectural level.
Scenario 5 - Future Proofing
In terms of future proofing, then, what would we need to do to add an A2A adapter onto what we already have that's showing you MCP and showing you trades with deployments and gates and everything else? That's the basis for this next demo. If you remember back to the beginning, we had this really cool rebalancer script running that was showing you a portfolio. Effectively, what this is, is it is a mechanism that's often used within, you buy a bunch of stocks, and then you want to keep your portfolio balanced to meet some particular strategy. I would say that this strategy is not something that you would probably find anywhere, because it's just going, balance to this basic percentage. It works. What we can do is we can basically flood that with lots of information. However, before we do that, what I want to do is we've got essentially this agent-to-agent UI.
Now what this is doing is it's modeling an agent that you could create. This is sat on top of an agent, effectively. The interaction is, rather than it being an LLM, it's us. What you can see here is it's looking at the trades agent-to-agent configuration. UI speaks to A2A, and it goes, ok, what can you do? It documents a list of skills. Basically, it can book a trade, it can get a trade by ID, it can amend trades, basically do everything that the API can do. Possibly not the right way to structure the agent card. You can actually invoke this like you would with the MCP stuff, send requests over to it, get back your information, just using the same approach, but using a different protocol to achieve that.
When it comes to our rebalancer, what this is doing now, this is our Python script that's talking to the trades A2A, and it is looking at what's available inside, by calling GetTrades effectively. It's making a decision. It's looking at what's there. It's then observing any changes. Then it's acting on those particular things. In this model, it's not going to do anything, because we've not actually hit the trade button yet. Andreea, if you hit that there now, we're now going to put a load of new bookings into the system. What the agent's going to do, using its very simple decision tree, it's going to make its observation, and it's going to go, hang on a second, the portfolio's really changed, you've got now 5.2% of the NVDA stock that you didn't have before. Now we need to adapt to that, we need to start acting. It's going to book some trades against that.
Architecturally, what's interesting with agent-to-agent, we did this basically on the protocol and said, script, you make the decision when you need to rebalance, you do the stuff that you need to display here, and then book the trade itself. Whereas actually what would be more of an agentic way to do this is have, this is just your portfolio, and it says, this is what I've got, can you rebalance me? Do I need to rebalance? You start to make things around specific tasks rather than the specific protocol. Again, this is all added into, if you were to look at the Kubernetes cluster now, it's got the MCP server, it's got the trade server behind it, it's got the A2A server in front of it. It's all micro-segmented and secure. That's part of the way in which we can add new things. If you come up with something new tomorrow, and there are other things that you could potentially be using here as well, you can add those in and essentially trial, future-proof, figure out what is right for your specific use cases.
Summary
We've gone through a lot there, and we've deliberately not gone into CALM in too much detail in terms of the deep-down bits and pieces. We're the lead maintainers on there, so it's a part-time project that's turned into a real-time job. When I said that we have that trades example, we had nothing. The question I got asked by my boss was, are you sure you want to put an open-source project in the center of everything that we're doing, last year? The answer was, it seems like a good idea. We can fix it in the open source and make it work. I'm glad we did because of the advantages that we can get today, and that's specifically around platforms. Platforms provide us guardrails. They're not just about infrastructure. There's lots of documentation about folks who are spending way too much time working on infrastructure-related problems, trying to figure out how to build and deploy their environments in a way that just means that they spend 70% of their time on undifferentiated heavy lifting.
That's all they do, Terraform, and 30% of time on the business code. It's also to say that we want to be able to allow people to change the platform, though. We don't want it to be like a really constrained, nobody can do anything, there's no element of change that you can have. That's where CALM comes in as being the important vehicle in our patterns. We have our initial opinions, we have the things that developers can configure, and those can change over time. If there are new requirements, new things that we need to address, we can add those into the pattern. There are strategies around rolling that out to decentralized architectures. If folks are interested in that, we can talk about it as well. Those are all then captured and then run through that deployment every time. Every time we do something, it's going through security reviews, all the things that Andreea mentioned are real gates that we have on every deployment that we do.
We should be exposing business capabilities. The stuff that we end up deploying for our systems are what our users are trying to convey to external parties that we interact with as a company. AI changes things, though. It introduces a lot of new consumers. We don't just have a system calling an API anymore. We've got developers who are potentially interacting with our stuff. We've got apps. We've got agents. Developers are the worst because you know what we're like. We like to tinker around, try different things. Suddenly your volumes and spikes go up before you've even managed to answer anything. That's just the nature of the difference in the traffic styles, the difference in the retries, the difference in the API patterns all together. MCP and A2A, they really do support new interaction patterns, but the foundations of a platform really do still matter. I think if you're going and just doing those without thinking about the same foundations, you're going to end up in a potentially sticky situation down the road.
Security and policy and governance must be built into a platform, and that allows you to adapt to changes in policy, changes into security. Andreea mentioned patching for issues. We've done things now where we can roll across the entire estate and patch something within an hour across 400-plus gateways without having to have any downtime for our consumers. That starts to become a real vehicle of where this comes in to being a real, when we need to respond, we can respond. The goal is having a platform that helps teams move quickly, where they can bring their feedback. They can do that in a safe way and consistently. That's the purpose of the platform with CALM that we've shown today.
Questions and Answers
Participant 1: How has this changed the culture in your developers and architects and everything like that? Where I work, we architect solutions and then allow the developers just to develop it as they see fit. If we started to regulate that, which is what I see this is, not a bad thing. I'm blown away with this. I was just wondering, how have you managed that culture change, or was there one?
Jim Gough: I think there are two different ways you can do this. Our patterns and architectures tend to be more about configuring something that we fully manage. In terms of our Kubernetes clusters, our service mesh, our API managers, that's all rolled out and it's held as a central tool for everybody. Often, when we have developers come to us, they're using that almost more as how they want to configure their usage of the platform using their specific opinions. What we've also seen people doing with CALM is not just having something where you configure a platform, but also having it where you actually use CALM template and some of the associated tools to effectively bootstrap projects. In that situation where you're describing where you want to give people flexibility, but you want to deal with something specific, in that case you can have something. As an example, in the templates that Andreea showed, you can have a script that calls something like Spring Initializer or Quarkus Start, bring you something in, fill that in with the right baseline, and have something that works and can go straight to production.
If you have that then, the thing that always is great about that is you've got something working in production, now you can implement change. That's the other way that you can do it. We've seen that work quite well. We've seen some developers just almost think of it as configuration, and we've seen other developers and especially our other groups, like our control groups, start to look at it more about how they can get an insight into what's being deployed and running in a given environment. There's lots of different tradeoffs with it. You do lose some flexibility, but you also gain a lot of productivity as a result. Developers tend to like, if that's in prod and it's already working, now I can tweak it to meet what I need to do from a business perspective. That's pretty good.
See more presentations with transcripts
