BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Presentations How Starling Built Their Own Card Processor

How Starling Built Their Own Card Processor

Bookmarks
48:30

Summary

Rob Donovan and Ioana Creanga discuss what happens behind the scenes when one pays with a card, and how Starling built their own card processor, integrating traditional hardware with microservices.

Bio

Rob Donovan is tech lead for the card payments engineering team at Starling Bank. Ioana Creanga is Engineer Lead for the Payments Processing Group @StarlingBank.

About the conference

QCon Plus is a virtual conference for senior software engineers and architects that covers the trends, best practices, and solutions leveraged by the world's most innovative software organizations.

Transcript

Donovan: My name is Robert Donovan. I'm the tech lead for the card engineering team at Starling Bank. I've been at Starling, coming up to three years now.

Creanga: I'm Ioana Creanga. I'm an engineering lead for payments processing group at Starling. I've been with Starling for almost four years now. This is the presentation that we have on how we've built our card processor. It's going to mainly focus on the architecture of it, and we hope that you're going to see some design patterns in it, and decide to use some of them in your own architectures later on.

Outline

At this point, if you don't live in the UK or you don't work in the financial services domain, you might be having two questions. One is, what is Starling Bank? The second one is, what exactly is a card processor? For us to be able to give insight on how we built the card processor, we're going to have to discuss about the more global payment architecture and how does the card processor fit into that more global payments architecture. You're going to also find out what happens when you pay with your card at a terminal or at an online shop. We're going to reveal our architecture diagram of the card processor, because this is why we're all here. On the way, we're going to discuss really interesting concepts like HSMs, MIPs, how PIN verification works, and what cryptograms are. For now, let's just find out what is Starling Bank.

What is Starling Bank?

Donovan: Starling is a digital bank. You can see our lovely app here. We've won best British bank for four years in a row. We're going to be talking about our cards that you can see here. Just to give some perspective on the scale of the bank, we currently have 2.8 million accounts, with £8.4 billion worth of deposits in them. From a card volume perspective, we currently process just over 2000 authorizations per minute on average, which is around 33 per second. As you can imagine, our numbers are going up very quickly every day. We need to be able to scale upwards quickly.

What Is a Card Processor?

Creanga: This slide promises to answer this question, what is a card processor? For us to tell you that, we're going to go through a more global payment architecture and where does the card processor fit into that architecture. We're going to be discussing all of these parties that are involved when you pay with your card at a terminal. After we explain all of these systems, we're going to give a very simple example of a card authorization message. Let's start with merchant. Merchant is the company that you buy from, so this is Amazon or Pret a Manger. For these merchants to be able to accept card payments from a card association like MasterCard or Visa, they need to have an account with an acquiring bank. The acquiring bank is simply the bank that holds the money for the merchant and doesn't know anything about the customer that is trying to make the purchase. To get that answered, you will have to be able to put this transaction message into the payments network. We have there MasterCard, just because Starling supports MasterCard card networks, but it can be Visa or American Express. The payments networks role is to look inside a card transaction message and see what the primary account number for it is, which is what we call a PAN, which is simply just the number that is printed on the card. It needs to look inside, see this PAN and know where to route this transaction, so which card processor it needs to go to, based on that information.

Another thing that the payments network needs to be able to facilitate is the movement of funds between the issuing bank and the acquiring bank. When you pay with your card, obviously, money needs to leave your account and end up in the merchant's account. This movement of funds needs to be facilitated by the payments network. Now we've reached the card processor. This is what we're talking about. This is what this presentation is about. We're going to spend a bit more time here. You notice here that the card processor and the issuing bank, they're just two separate entities on this slide. The reason for that is that some banks don't have their own card processor. They have this outsourced to another company, a third party company. Starling used to be the same. Seven years ago, when Starling Bank launched, we didn't want to embark on this long journey of creating the card processor. We decided to just go for a third party company that does that for us. Now Starling Bank has two card processors, actually, because we're still using the old card processor with this third party company, and we also have the new card processor that half of our traffic goes actually through this card processor that we've built that we are talking about now, and half of it is just through the old card processor. That explains why these two entities are separated on this slide. We're just going to be talking about the card processor that we've built.

What is the main role of a card processor? It needs to be able to have access to the payments network. As you'll see later on, this is not a trivial task. It also needs to be able to decode these messages that are coming from the payments network and verify cryptographic transaction data. The simplest examples of these cryptographic operations would be the PIN verification and the cryptogram validation, which is just a way of figuring out if the transaction has been tampered with on the way. To be able to perform all these operations, it needs to hold some secrets and manage some secrets of the card. Things like the PIN block, it needs to be able to store that. Things like CVV and expiry date. Another thing that it can do, although it's not always the case, it can hold the account balance. For example, for the old card processor that we have, that is going through the third party company, they don't hold the account balances for us, they just make a call to us to understand if the customer has enough money into their account or not. Of course, for the new card processor, we do hold our own account balances. These are the main things that the card processor needs to be able to do. At the issuing bank level, business rules can be applied on a card transaction, things like, does this customer have enough money, or a lot of other fraud checks will be performed at the issuing bank level.

Now that we went through all of these parties involved in a card authorization message, let's just go through a very simple example of a card authorization. Rob here is not only an excellent coder, but he is also a great musician. His favorite merchant is Denmark Street. He goes to Denmark Street and he wants to buy himself yet another very expensive guitar. Because it's a very expensive guitar, he can't just tap into the terminal, he will have to put his card in and enter the PIN. This is what we call a chip and PIN transaction. Chip because the card has a chip, and PIN because he has to enter his PIN. What happens with the transaction afterwards is get packed by the terminal and sent over to the acquiring bank system, so we have there Barclays. We don't know if Barclays is actually the acquiring bank for Denmark Street. What Barclays will need to do, is it doesn't know anything about Rob because Rob doesn't have an account with them. It won't actually put any money into Denmark Street's account, it will send it over to the payments network. It will mark this card transaction being in flight for Denmark Street, and then put us into the network.

The network in this case is MasterCard, because Rob has a MasterCard card. What MasterCard will do, it will look inside the transaction, it will see what is the card number that is printed on the card. It will figure out that this transaction needs to be routed to the Starling card processor that we've built. It reaches the card processor, and here we're going to do all of the verifications that we need to do. We're going to verify that Rob remembered correctly his PIN. We're going to also figure out if the transaction has been tampered with on the way. We're going to also verify, are there other things here? Also, business rules like, does Rob have enough money into his account? Or maybe in the past, Rob had decided that to remove any temptation from buying more guitars from Denmark Street, he decided to just block this merchant. We're going to verify that here as well. We're also going to do some fraud checks. Is this transaction likely to be fraudulent or not? This is what happens with a card transaction message. This is how it flows through all of these systems that are involved. The question is, if it's such a complex job to build a card processor so that entire companies build their business on that, why did we even bother to go for this long journey of building our own card processor?

Why Build Our Own Card Processor?

Donovan: There are a few reasons why we decided to embark on this project. The first one of these is innovation. Starling has been innovating in the banking space since it started. Previously, there were certain restrictions on us being able to add new features that may be part of the card processing workflow. We really wanted to be able to bring this functionality in-house so we could create new products, where we're in control of them from the moment they leave the MasterCard network. Alongside that is time to market. By bringing this into the Starling ecosystem, it means that we can release our card processor as quickly as we can our other services, so we do many releases a day. Whereas if you use a third party, they obviously have to prioritize work depending on the needs of all of their various customers. We can build new features very quickly and release them based on our own priorities. If bugs come up, then we can also fix those very quickly. That brings us on to resiliency. When issues do come up again, with a third party, they have to manage that incident across all of their customers. We can very quickly identify the problem and release fixes, or even mitigations on our side. Also, there's this fact that MasterCard only give the issuing bank 7 seconds to respond to a card authorization. If you take longer than that, then MasterCard will make a decision on our behalf depending on the rules that we have set up with them. By bringing this in-house, that gives us the full 7 seconds to respond. Whereas using a third party, you generally get a much more restricted amount of time to execute our business rules. Finally, there's the obvious one, which is a long term cost saving. By having this in-house, it means we don't have to pay contract costs. It means that the interchange fees that MasterCard give us for each transaction, we can keep all of those, we don't have to give a percentage to a third party.

What Are The Challenges?

What are the technical challenges on this? The first one is that we're going to need some boxes sitting in a data center for this, firstly to connect to the MasterCard network, and also for our security purposes. We have these physical boxes, but we need them to be able to talk to our microservices architecture that's running in the cloud, which is where we have all of our very fast release cycles and resiliency, and all of that stuff. Next to security, so we're now going to start storing sensitive data from our customers that we didn't previously need to store. Previously, that was the third party's challenge to store that data securely. There's a huge amount of complexity in building a card processor. This is because of the sheer number of permutations that you can get in terms of types of messages that you have, how you have to handle those messages depending on certain jurisdictions and regulations in different parts of the world. You have mobile wallets, online transactions with 3D Secure, as well as terminal based transactions and all of these sorts of things. Finally, we want to make it so that if customers do encounter issues with card payments, we could very quickly identify where those issues are occurring, and understand them quickly, so that we can fix them if necessary, or at least communicate to the customer what's happened.

Starling Tech Stack

What's the Starling technology stack? We have things running in the cloud. We currently run everything on AWS. We were using containers and EC2 instances, but we're moving stuff over to Kubernetes now. We use infrastructure as code in the form of Terraform, for example. Our infrastructure team has done a brilliant job of this and really put the power into the hands of developers for releasing infrastructure changes. We can now make infrastructure changes as quickly as we can, and even standard service changes, which is fantastic. Our middle tier is pretty much all Java, backed by Postgres databases wherever that's needed. Communication is generally RESTful, but we have some stuff that's also just protobuf with Netty usually for performance reasons. We have a lot of this in our card processor.

Finally, the big mantra at Starling for our technology stack is keeping things simple. This is not a complicated, long list of hundreds of different technologies. We're very careful about introducing new pieces of technology into the ecosystem. Whenever you do that, you have to spread the knowledge throughout the engineering team, such that if you do have issues, then anyone is able to solve those problems. Also, it often just introduces new points of failure, new things that can go wrong. These stacks worked perfectly well for us for years. Resiliency is often achieved simply through ensuring that these RESTful calls, or RPC calls are idempotent. That means that if a service becomes unavailable, the upstream service can just say synchronously retry for as many times as it wants, and not worry about any nefarious effects of a call being made twice.

The Card Processor Architecture

Creanga: This is the architecture diagram of the card processor. We're going to divide it into three main parts, so we're going to talk about it individually. What we have on the left is the Starling data center. This is where we keep all of the hardware equipment that we absolutely needed to have to be able to build the card processor. Right to the Starling data centers in this diagram, there are two VPCs in the cloud. This is what we have in the cloud. The one in the middle is called zero trust VPC. This is where we perform all of our cryptographic operations. Things like the PIN verification or the cryptogram validation, this all happens here. The reason why we call it zero trust and it might be a different concept for other people, we call it zero trust because there's zero trust between the microservices that live in here. For example, to be able to communicate between each other, two microservices, we need to be very explicit and write infrastructure code to say that this communication is allowed.

As a tiny implementation detail on how we've achieved that, we use an AWS feature called NACLs, so Network Access Control Lists. For example, if we were to pick on two microservices in here, like MasterCard service and tokenization. MasterCard service is able to initiate and get responses from the tokenization service, but the tokenization service cannot initiate requests to the MasterCard service, it can only respond to MasterCard service. Every microservice in here has its own subnet, and we use NACLs to control the in and out traffic. The zero trust VPC here is not responsible for making any decisions about what should happen with the card transaction, if it should be approved or not. Its only role is to gather all of this data to learn more about this card transaction that is currently in flight, and send it over to core VPC where we have the card processor to be able to make that decision. This is the role of the core VPC, the card processor that exists in the core VPC is to perform all of these business rules that we've talked about. Does Rob have enough money for another expensive guitar? Is this transaction likely to be fraudulent? All of these business rules will be applied here in the core VPC.

Now let's get back to what we have in the data centers. We have two different devices in the data centers, and one is called a MIP, MasterCard Interface Processor. Where you see the MasterCard logo, what that's supposed to represent is a device called MIP. This is what we have from MasterCard, to be able to have access to their payments network. What normally these devices have are two Ethernet ports. One is to give access, to get access to the payments network. The issuing bank normally does not have access to that. We do not have access to that one. Another Ethernet port, we just plug in into the card processor itself. These are the devices that we get all of these real-time card authorization messages. This is where these are flying from.

Another device type that we have in the data centers are the HSMs, so Hardware Security Modules. People that work in financial services are already very familiar with these. We use them in many different ways at Starling, but in the context of the card processor, what we use them for is to get help with cryptographic operations like the PIN block verification, or the cryptogram validation. These HSMs that we have, they can be very generic, but the ones that we have for the card processor, they're very specific. They're so specific that you can literally send the command to the HSMs to verify the PIN block with the right arguments, the HSM will just verify it without you having to decrypt the PIN block, and look at the PIN in clear. This all happens inside the device itself. Another important feature of the HSMs is that they safely store the encryption keys. They have all of these different sensors on them so that if they detect that they're being moved from the rack, they will self-destruct and wipe the encryption keys. That way you can just be certain that your encryption keys are just very safely stored.

This is about Starling data centers. We have obviously multiple data centers across UK, and multiple MIP devices and multiple HSMs for redundancy and resiliency. Because we've mentioned these commands that we can just send to the HSMs, let's get a little bit into more detail about these PIN verification and cryptogram validation commands that we've talked about.

PIN Blocks

Donovan: When we talk about PINs, you normally think about just the four numbers that you put in, or sometimes more if you're in other parts of the world. When we send PIN blocks around, when they're in the clear, they're actually in this PIN block format. The reason for the PIN block is to cryptographically tie the PIN itself to the customer's account number that's on the front of the card. The reason for doing that is that using a symmetric key for encrypting the PIN, you don't want two different customers having the same PIN. You encrypt them, and then you could see that they're the same. To avoid that, we have PIN blocks. There are a few different formats of PIN block. The one that I'm going to talk about is the one that we use, which is ISO-0 format. It's a very simple algorithm to do this. We start off with this here, so this is just the length, PIN, followed by the PIN itself, and then some padding of 15. This is all interpreted as hexadecimal. Then we take the customer's account number and the 12 right-most digits on that, and left pad that with zeros. The last digit at the PAN actually, which is a 1 here, is known as a Luhn check digit. This is actually based on an algorithm to validate that the PAN has been entered correctly. It's not really for security reasons, historically, it was just to make sure that the customer hadn't made a mistake entering their card number. This is an interesting little fact there. To create the PIN block, we just take these two elements, and we XOR them to get the PIN block. When we talk about the encrypted PIN and passing the encrypted PIN around, we're actually talking about this PIN block.

The Chip and the Terminal

How does the chip actually talk to the terminal? When your card comes in contact with the terminal, or within a certain distance, if it's contactless, the terminal induces a current on the chip. The chip has a very small operating system that runs on it. That operating system supports advanced bunch of applications which are effectively functions, just take some bits as argument and spits them out. The terminal can select one of these applications to run and provide information based on what the customer has entered, for example. It also needs to agree a verification method with the chip. They will effectively compare what verification methods they support. There's an order of verification methods that we've programmed onto our chip in order of preference. An example for verifying the card holder would be for them to enter a PIN. Then the terminal is going to request a decision and usually send that authorization through to the acquiring bank, and eventually on to us through MasterCard.

Offline PIN Validation

There are actually two different versions of your PIN. There's the one that's stored on your card itself. There's also a secure element on your card. Similar to very tiny little HSM on your card. There's the online PIN. We'll start off with the online PIN verification. This actually happens in quite a lot of cases, because the terminal will only go online based on certain rules that we've set on the cards. That might be because it's above a certain value, or we've deemed it to be high risk because it's cash related, for example, withdrawing cash at an ATM. A lot of the time it will just validate the PIN offline. That means that it's going to start by getting a public key from the card. It then takes the PIN that the customer has entered into the terminal and creates a PIN block from it using the account number. It encrypts that using the public key that it got from the card, and then it issues a verify command, which is where the card is going to compare that with the PIN block encrypted on the private key, that's stored on the secure elements.

Online PIN Validation

Online PIN validation is when we actually receive the PIN block ourselves and we validate it with the PIN block we've got stored for that customer. Really, this process just involves passing around the PIN block through different parties encrypted with different keys. We start off with the terminal that encrypts the PIN block under a key that is owned by the acquiring bank. The acquiring bank receives that and then it's going to translate it under a different key. That's a key that the acquiring bank will have exchanged with MasterCard. MasterCard received the PIN block encrypted on the acquirer's key and we've also done an exchange with MasterCard with our own issuer key. MasterCard then translates the PIN block to be encrypted under that key, which is how we receive it. We also have the customer's PIN block stored under a separate key. We can take these two things, we can take the PIN block that we've got stored encrypted under key B, and the one that we got from MasterCard, send them into the HSM and ask the HSM to tell us whether they match or not. The HSM does its decryption and comparison all inside its secure housing, so we never have to see it in the clear anywhere.

Application Cryptogram

The application cryptogram is an element of chip transactions, which is to protect against tampering. The way this works is that the terminal will ask the card to generate an application cryptogram. It'll pass it certain information that's on the transaction, such as the transaction amount, for example. The card is going to take that information and encrypt it using another key that's stored on its secure element. The terminal then packages all of that up and sends it through to us. We receive the transaction details. We also receive this encrypted block that we call the cryptogram, which contains the same details. We can send that to the HSM and it will compare the values. We can decline if any of those values don't match, which indicates that it's been tampered with. There's also a transaction counter in this block that we can use to prevent replay attacks. That's some of the main cryptographic operations that we perform as a card processor. Let's head back to the architecture.

Zero Trust VPC Architecture

Creanga: Now that you've delved in the fascinating world of HSMs, we're going to go back to the zero trust VPC and what we have in there. We're going to talk about what's happening in each one of the microservices that live in here. The first two edge services that we have, they are MasterCard service and HSM service. MasterCard service, its main role is to accept messages from the MIPs and be able to decode them. Card payments globally have to comply to a specific format called ISO 8583. All of these card payments are coming into this format, and card associations have different implementations of this. Visa will have its own implementation, and MasterCard will have a different implementation. For MasterCard specification, what we had to do here in this MasterCard service is to decode all of these messages according to that manual. It was really hard work because this manual has over 1000 pages that we had to read, understand, and then implement. A lot of hard work went into this MasterCard service.

How does the MasterCard service even get the data from the MIP? What we have here, there's this TCP/IP connection. It's just streams of bytes just flowing through here. This is not all going over the internet. What we have here, there are VPN tunnels over a private network. We use AWS Direct Connect. That's why we don't have to send the data over the internet, we just use a private network. These are all encrypted stream of bytes going into the MasterCard service, which will then decode all these card authorization messages according to the manual that I've just mentioned, that's over 1000 pages long. The decoding will just transform all of these bytes into a very simple Java object that will represent a card transaction that will just be passed over inside this VPC. This is MasterCard service. HSM service just acts as a proxy client to send commands to the HSMs. Now you know all about these commands that we're sending to them. It's the PIN verification and the cryptogram validation. It's the same principle as the MasterCard service. There's also TCP/IP connection between them. We use Netty I/O framework to get help with managing all of these network messages that are coming from the MIPs or from the HSMs.

Another thing about the HSM service is to be able to send all these commands to the HSMs, it will need specific data like the encrypted PIN block, or the track data and so on. It doesn't have a database, so it will have to go through some other services to get the secrets from. You'll see that this is done only if HSM service holds some access tickets that it got from other services. If only those tickets are valid, it will be able to get this data from the safe services and be able to perform its commands. This is all about the edge services that we have. Now let's look what happens next, after we've decoded a card transaction message coming from the MIP? What happens next to the card transaction?

Donovan: The next step in the process is to gather the results of the cryptographic validation. Once the MasterCard service has decoded the message, the first thing that it does is it goes to the tokenization service to tokenize the customer account number into a unique ID. It does this using a hash function. The reason is just so that we are passing around something other than the account number, between the other services just as an extra layer of security. The MasterCard service takes that information plus what it got from the MasterCard network, sends it to this validator service. This validator service is going to create the access tickets that Ioana mentioned, to send on to the HSM service to be able to validate the cryptographic data on the transaction. The validator is going to get those results back, and it's going to package them up and send them on to the next step to execute the business rules. It very deliberately does not make any decisions on what to do with the transactions at this stage. It's just a dumb data and validation results collector. The next thing that needs to happen is the HSM service is going to need to get the data that it requires to perform that crypto validation so that it can send the results back.

Creanga: These are the microservices that hold the card secrets, like the PAN, or the PIN block, or Track safe. Track safe is just CVV and expiry date. These three microservices, they all store these secrets and they just hold simple pairs of a zero trust UID and whatever secrets it needs to store. A zero trust UID is simply just the hash function of the PAN. For example, for the PAN safe, it will have the zero trust UID and the PAN, for the PIN safe, it will have the zero trust UID and the encrypted PIN block. The way this works, what these microservice needs to do besides storing this, it also needs to be able to return these secrets to HSM service. They can only do that if HSM service is passing a valid ticket. They will all go before returning any data to the HSM service, they will go to the tokenization service who actually created these tickets to validate them. Only if these tickets are valid, it will return the data into the HSM service to be able to then send the commands to the HSM service. Let's look in a little bit more detail on how this access ticketing system works.

Safe Access Tickets

Donovan: This ticketing system really works hand in hand with what we're calling the zero trust model, provides. Each of these services has its own subnet. We have to define ingress and egress rules on both sides of the conversation in order for them to be able to talk to each other. The validator is going to need to create these access tickets for the HSM proxy to be able to get the data out. It creates those tickets and sends them on, but the HSM proxy itself does not have these ingress and egress rules, so it cannot reach the tokenization at all. The tokenization service has a number of different servlets that it exposes on different ports. One of those is for ticket creation and a separate one for validation. We can allow the crypto validator just to reach the creation port and the HSM proxy is blocked from reaching either. If Ioana here, for example, if she was upset with me about something, maybe because I'm spending too much time playing guitar and not enough time working, she tries to steal my account details. If she were to somehow manage to get on to the HSM proxy box, she could actually kill some of the endpoints on the safes. She wouldn't be able to create a ticket to get access to that information. If she were to breach one of the safe boxes, again, she could call those endpoints. The safe boxes can only validate tickets, they can't create them. They are blocked, there's no rules for them to be able to do so. What will happen is the validator can create the ticket, send that onto the HSM proxy, all that can do is pass it on to the next step to get the data out. The safes will call tokenization to validate that ticket and return the data back if it is valid. These tickets also have a very limited lifespan. They will only be validated successfully. If they're validated, it's very short time after creation.

Architecture

Creanga: We're now done with the zero trust VPC. We said that the main role of this VPC is to perform cryptographic operations and gather more information about this card transaction that is in flight. It will never make any decisions about if this card transaction will be approved or not. Crypto validator here, gather all the data and will pass it on to the core VPC where you see their card processor to make that decision. This is where we apply all of those business rules like, does Rob really have enough money in his account for this new guitar? Also, has he blocked maybe transaction to Denmark Street? All of these maybe other fraud checks that we have to perform, they're performed here in the card processor. Also, a tiny implementation detail here, what we've built in-house is just the business rules engine. We didn't use anything that was already there. It's just a very lightweight business rules engine that we built here to help us with that. I think that's about it. That's all you need to know about the architecture.

Technical Highlights

Donovan: It goes back to the challenges that we mentioned earlier. As we said, this bridging between a data center and a nice modern cloud stack was achieved using AWS Direct Connect, in tandem with a VPN tunnel, so that we can have this nice, secure private network to communicate between the two, but continue releasing stuff very rapidly in our cloud environment. Then, using the security features in AWS to achieve our zero trust model has been really interesting, especially in combination with this ticketing system that we've developed. We think that's a really nice robust way of protecting our data assets. Finally, this separation of the data gathering aspect of things from the business logic has worked really well. This means that when we do have issues, or we have customers asking us why a particular transaction was declined, for example, we know that there's just one place we need to go and look. That's on the business logic side on the core VPC. We can see the steps that it took through our workflow engine. The zero trust side of things just sits there and does its crypto stuff and just works. We don't really have to touch that at all, which is great. We don't have to look in loads of different places when you're trying to debug these things.

Questions and Answers

Ignatowicz: I'd like to explore more because you mentioned a lot of microservices that you run in production, which tools or techniques do you use to do observability of those microservices running in production?

Creanga: I can tell you about the stack that we're using, we use Prometheus for our metrics, and then we use Grafana for dashboards. Prometheus metrics is what we use for alerting, and so on. Obviously, we use some AWS features as well to alert on if some of the metrics that we have in there can alert us of anything going wrong?

Donovan: We use correlation tracking quite a lot as well, which is really helpful, obviously, when you're following the logs. Also, we use Instana as well for tracing. We've found that really helpful with this architecture for finding where bottlenecks are, essentially. You can track all of those service calls through all the different layers and where the time is being spent.

Ignatowicz: You talked about a time that you do migration that you are running your solution and also the third party vendor solution. I assumed that from the terminal point of view, the payment terminal, it would be a different latency for both services. Do you have any strategy to route part of the traffic for one or for another, depending especially on the latency that I cannot control in a third party provider.

Creanga: The routing is done by MasterCard itself. When you set up a card processor with MasterCard, you will get that. Initially, when we had the third party provider that was one setup that we had with MasterCard, and that is based on the card number. The card number and the BIN range will tell you exactly what card will go through. There's nothing that we can do on the fly to route them to the new card processor. For example, if you were to move everything, if you wanted to move the traffic, all the traffic to the new card processor, what you have to do is reissue all of the cards into the BIN that you have set up with MasterCard so that the traffic will be routed by MasterCard itself to the new card processor. It's a high cost to do it, to the company, to the bank.

 

See more presentations with transcripts

 

Recorded at:

Dec 29, 2022

BT