Transcript
Chad Schuster: I don't think anyone will disagree with the first statement on this slide, maybe the second one is a little bit more subjective, "Python is popular. Python is slow." Really this slide summarizes the motivation for this presentation. It really tracks a project that we had and continue to work on, and work on internally to try to figure out ways to utilize Python while making the parts of it that are really slow that we cannot have and trying to speed them up, basically. Can we utilize Python resources to build and maintain models that need high performance? It's interesting, this ties back to Lily's presentation. She was talking about taking Python functions and translating them or substituting in Rust. This is conceptually in many ways very similar. In particular, the example that she used that was very numerical would have really actually worked quite well with the technology I'm going to talk about.
Why Do I Care?
Just a little bit of background as to where I come from to give you an idea as to the, why do I care? I work in the U.S. life insurance industry, specifically more on the retirement space. Those would be annuities. Premiums are estimated to be on the order of like $1.3 trillion-ish this year. Just to give you an idea for the size. There are a lot of metrics that insurers have to measure that are very computationally intense, in particular for annuities. There's a lot of computational overhead. You can think of it this way. A lot of the regulations are basically centered around making sure that if you buy this long-term product, that when you get to the point where you want to utilize it, that there will be money there, that they can fulfill their obligations. What that usually involves is taking whatever the product is and modeling it out and tracking all of the cash flows, and then running that under various different kinds of scenarios.
It's very numerically, computationally intense. The initial reason for looking at this was trying to think about how we could help ourselves and our clients utilize resources that they have and that we have to build an ecosystem with Python but then make the parts fast that we need to make fast. Internally and externally, there are a lot of vendor and open-source solutions in the insurance space. I'm going to be focused, obviously, with Python, more on open-source solutions today. We internally, historically, have had C++ as our model. We still use that in most of our work now. We also have a vendor solution. The C++ solution is specific to these annuity products that are very computationally intense. One of the reasons for this focus on Python and trying to make it faster also is that a lot of insurance companies historically have had on-premises grids. They have a bunch of computers, 5,000 to 10,000 node grids.
It's a sunk cost, so people throw stuff at it. They don't really think too much about how long it takes to run, necessarily. As these companies move to the cloud now, how much each run costs, how much time it takes, has always been a concern. Now cost is also something that they're very focused on. That has also shifted a bit of their focus. Again, tying it all together, can we utilize Python resources that a lot of these companies have as subject matter expertise to use Python where we can, and then make it fast enough to be able to be utilized in these models? Just one last quote, which I think summarizes my work life. You could substitute I for hardware engineers and users for coders and consumers. No matter how fast we make our models internally or for our clients, they always want to go faster. I think that is for twofold. Faster, obviously, can be cheaper. More often than not, it actually isn't cheaper because they just run more stuff. At least it gives them more ability to do data analytics and get the information that they need from the models to do their jobs in a more efficient and better manner.
Personal Profile
I work for Milliman. It's an actuarial consulting firm. I've been there for about 22 years. My background is in math and physics and financial engineering. I would venture to say I probably have the least computer science background of anyone at this conference. I've been working on our own internal systems and in charge of them for the last 15 years. Then in that time also have been helping our clients develop these kinds of models, maintain them. In some case, we're building something that we then hand off to the client and they want to take over. That in particular is an instance where Python has been very important to those kinds of projects. Like people want to have bench strength. To have bench strength, at least from their perspective, having Python as an option as at least the main part of the language is really important to them.
Python Language
How we're going to talk through this somewhat tracks a little bit of the initial project that we did to decide to try to use Python and then the steps along the way where we quickly realized that Python wasn't fast enough. Then how we gradually found Numba. Then work through both the advantages and limitations in Numba. We'll talk a lot about those things today. My hope is that at the end of this, everyone has an appreciation for the tool that Numba is, where you might be able to utilize it, how simple it can be to utilize. Then also just a sense for the tradeoffs. I think a lot of the tradeoffs are really very dependent on the amount of code that you have to squeeze into Numba. If you have a few numerical functions that take a lot of time, I would say Numba is almost a no-brainer, if you really want to stay with Python.
You might still have to change a few things, but probably you're going to see a really good increase in throughput and not have to do a lot. The other flip side of that is what we had to do, which is put a lot of logic into Numba. With that, I will say there were some limitations that we'll talk about and some complications that we'll talk through as well.
Why Python? It's very popular. It allows for fast development. There's a massive number of libraries and an ecosystem around it. You can develop something very quickly with Python. I think at this point it's basically the de facto programming language for entry-level computer science classes in colleges. There are a lot of people coming out of school, even if they have an actuarial background, if they have any coding experience, it's Python. Our clients have that, we have that. There is, again, this overwhelming desire to try to utilize those resources as best we can. Conventional Python, or CPython as I'll refer to it, is implemented as an interpreter. That means that as it runs, it analyzes the source code through the program flow of control and looks at each instruction and executes a series of internal operations to achieve the correct interpreter state. A compiled program like C++, on the other hand, is creating the machine code before it ever starts to run, so it is inherently much faster than Python.
Python is a very object-oriented language, and I'll get back to that multiple times as we talk through Numba. It utilizes things like polymorphism, encapsulation, inheritance, and it makes it easier to read and understand, less prone to disruption when changes are made, and can facilitate reuse of code. All things that obviously are very attractive in terms of maintenance, and something that we, as we went through this project, really wanted to try to keep. CPython divides the interpretation process into two parts. The first part is going from source code to bytecode. Bytecode is just a more streamlined representation of the logic. It gets rid of things like comments, whitespace, what have you. It still has to be interpreted, so bytecode still needs to be interpreted. That process, again, is taking the bytecode and translating it into machine-level operations. The key there is that it's not actually machine-level code. It's more or less just calling predefined operations to move the interpreter into different states, which, again, is overall not the fastest way to do things.
Numba - Background
As we went through this project, we basically started with, we really want to use Python. We did a proof of concept where we created a C model, we created a Python model, and they were so different from a performance perspective that it immediately became obvious that Python was not going to work. We were pushed to try to find a way to get back to Python. The way that we found to do that was to introduce a library called Numba. It's an open-source library developed or initiated in 2012 by Travis Oliphant. It attempts to apply the process of compilation, similar to C++, to eligible parts of an otherwise interpreted Python program. The compilation process is performed as the program is running, so it's called just-in-time compilation. Just-in-time compilation is widely employed. There are other languages, which I'm sure you're all familiar with, Julia, Lua, all use JIT compilation.
Numba also uses LLVM for compilation and optimization. LLVM is used in a lot of different places in technology, so AMD, Apple, IBM, NVIDIA use it for their compilers. JIT compilation does happen in lots of different programming languages other than just Python. Julia, Lua, Swift, Mono, they all use LLVM as well. LLVM is an initialism. Originally it stood for low-level virtual machine. They've moved away from that. It's now just described as LLVM. The last thing that Numba gets you that ultimately for us became really important and very attractive is GPU. You can write something that works on Numba CPU, and the effort to go from CPU to GPU is a lot less than it is, say, in C CUDA. That became for us and for our clients something that they really wanted to try to hold on to.
Numba - Architecture
JIT is on a function-by-function basis, sometimes on class-by-class depending on the implementation. We'll talk through that a little bit more as we go along. The basis is based on Python decorators. A Python decorator is a shorthand way to tell one body of Python code that it has an opportunity to access and potentially modify another. By applying Numba decorators, a Python programmer can signify that an individual part of the program should be handled by Numba rather than the default Python interpreter. This makes it relatively straightforward, again, depending on what you're trying to Numbafy, relatively straightforward to apply Numba to something. Very quickly, the decorators, and I'll go into them just a little bit more. They're not too terribly complicated. There's JIT and then there's njit. njit is just a short way of JIT, and I'll show you what I mean by that. The use of LLVM and Numba is accomplished through a Numba subproject called llvmlite.
It exposes a subset of the LLVM API implemented in C++ to Python. Still LLVM, but a specific version to Numba that is part of the Numba package when you install it and download it. This is an example of a very simple Conventional Python program. We'll come back to this later. There's a problem with this, but we'll get back to that a little bit later. This is an example of applying Numba to these two functions. You can see at the top we're importing, and then the JIT decorators there. We use JIT, and you can see that it has this argument, nopython = True. njit has that implicitly applied. You don't have to put that argument in there. We've always used JIT because of some things that will become obvious later, but it allows you to turn Numba off very easily for either all of your functions or parts of your functions without having to go and do things individually.
You can make those arguments variables and have the ability to turn things off. This is what the Conventional Python pipeline looked like. Now we're looking at when we add in the Numba decorator. After the bytecode, instead of being interpreted, the bytecode then is translated into Numba intermediate representation, so that's the first green box. Numba IR is modified and analyzed to infer all of the actual data types for every function, parameter, and variable, since machine-level compiled code must commit to specific data types in advance. If type inference succeeds, the corresponding Numba IR is further modified and optimized and automatic parallelizations are included as well. To invoke the Numba compiler, so to get to the machine code, Numba must first translate or lower the type Numba IR into LLVM intermediate representation. When I talk about lowering the rest of the presentation, that's the arrow between Numba IR and LLVM IR. LLVM will then apply a host of optimizations to boost performance and increase efficiency. Once LLVM has compiled for machine code, then that resulting implementation is substitute anywhere where that original Python function existed.
To drill into Numba IR a little bit more, to talk about type inference, because it is an important part of the process and the pipeline. Again, all variable types, arguments, and return values have to be inferable at compile time, because this is going down to machine code. Ideally, subsequent invocations of the same Python function would shortcut a lot of the expense of the compilation process, so instead of going bytecode and then the two IR steps and compiling, you could skip over things. Unfortunately, Python functions are flexible with respect to parameter data types, whereas machine-level code is bound to specific data types. As a result, Numba has to account for the possibility that it's going to get different types passed into the same function. Numba effectively prepends logic to each JIT function that performs a runtime analysis of the types used in the current call. If the analysis is compatible with versions that have already been created, then it does skip the rest of the process. If not, it has to go through the entire pipeline that we just talked through. Numba's support for compilation and reuse of different type-specific implementations of the same Python function is called polymorphic dispatch. We'll talk just a little bit more about that later when we get back to object-oriented types of characteristics.
Numba - Advantages
What are the advantages? By turning on JIT, you can see one to two orders of magnitude of speedup for the functions that are JITed. In our original POC that I was talking about, we saw that Numba made the program about 75 times faster than interpreted Python, and so that was why we started down this path in the first place. We had a C implementation, and in order to try to get something close, and it did end up being actually quite close, we used Numba to effectively shrink the gap in performance differences. There are other things that you get, some of which I already referred to a little bit from using Numba. Threading is a bit easier. There is some automatic threading that happens. It's easy to parallelize NumPy functions or loops. You can also utilize GPU, as I said before. Because LLVM and Numba both have support for CUDA, so that's NVIDIA's way of coding to a GPU, so you are able to get GPU, and that became really important to us.
I'll tell you at the end of this slide why. I think it will be very obvious then. If you have something well-suited for a GPU, then you can get another one to two orders of magnitude of speedup above what you were already getting from moving to Numba on the CPU. The degree to which Numba can deliver these reductions in time and cost is definitely very dependent on how much Python code is jittable. We'll talk about what that means in a little bit. How much time is spent on I/O, and then how efficient was the original Python code. Then, finally, how well LLVM is able to optimize and accelerate the generated code. It's a very large example from one of these models that run in practice. This model, once I accidentally turned off JITing and I gave up on letting it run because it was taking so long, I thought I had hung the system.
To talk about CPU versus GPU, so this is again a model mostly in Numba. What this model is is everything around it, all of the data processing is in Python. Everything that goes into the model and comes out of the model goes directly into interpreted Python. Everything that does the numerically intense calculations is in Numba. For this particular model, we were able to see, going from the original implementation to Numba, it was about twice as fast. Then we were able to again fairly straightforwardly move to use a GPU. That got us another 750 times faster. We effectively saw one GPU could replace 750 of the cores that we were using on that particular run, and this ultimately translates into a cost reduction of about a tenth. For a lot of companies in the insurance space, their computing budgets for the years are in the multi-million range, so that takes them from multi-millions to hundreds of thousands, which is a fairly significant amount of reduction.
I don't know that they'll actually see that reduction, because I know that they will just run more stuff, but they will be able to run a lot of things that I think in the past they were not able to do. We've talked about advantages, and a lot of this sets the stage for some of the things that I'll talk about later in the presentation. What was really important to us out of what we've talked about so far is the speedup, being able to use GPU. Then one of the things that we'll see is lacking a bit is the ability to be object-oriented, which is something that we really wanted to try to get back to. For some of these models, we're building them and then passing them off to clients. We always have some concern about the maintenance long-term, allowing them to be independent and not need to call us every time that they need to make a change. It is important to us to try to make it as maintainable as possible.
Numba - Limitations and Challenges
As we say in financial services, and you've probably heard, there's no free lunch. There are definitely limitations. Not all Python code is jittable. Oftentimes, you have to refactor for compatibility. Numba just does not support all of the features of Python, so some of the examples, dictionaries with flexible key and value types are experimental. Exceptions, context managers, closures, list comprehensions are limited. Many common Python and NumPy functions like print, range, sorted, getattr are limited. We had to work around this. Again, a lot of this comes back to, what do you need to apply Numba to to see an increase in performance that makes it worthwhile? For us it was a lot of code. We had to put a lot of code into Numba. A lot of the things that I'm going to talk about, even the object-oriented preference on our part is really probably more related to the amount of code that we are trying to maintain and put into Numba.
If you have something that's more simple, like I think in many cases Numba is probably almost a no-brainer, like something that you should at least try. In Numba proper, there are limited OOP features. We talked about polymorphic dispatch, that's a type of polymorphism. It's probably closer to templates in C++ than actual polymorphism. It's a gray area. Whereas in C or C++, there's a very distinct difference. Because Numba is compiled just in time anyway it's always going to be dependent upon what data types are coming into the function. It is a type of polymorphism. There are some ways to preserve class-like behavior, so there are some experimental features. Both versions allow encapsulation of code and data to be put into a single object. They are experimental though, so jitclasses and structrefs are experimental features that get you some object-oriented type of characteristics. Some of the drawbacks, so first of all they're experimental so there's no going from version to version in Numba.
As more versions are released, obviously there's no guarantee that they're not going to change how it operates. That's one reason we try to avoid them. Probably the biggest one is that these two experimental ways of doing classes do not support GPU. After our initial experience with the cost reductions from GPU, that became a non-starter for us. We were still continuing to have in our minds that we wanted to try to get back to or at least think about object-oriented if we could.
The general implementation that we used is very function programming related. We're using things like NumPy arrays, tuples, very simple data structures because as I mentioned, there are some restrictions on what you can use in Numba. If you are able to go from your Python code data structures to something simple, then, again, we're seeing or have seen very significant performance improvements. I think it really depends on what you're trying to put into Numba, the extent to which you might have to rewrite or change your data structures. Just want to talk about type inference errors and lowering errors. These are two types of errors that can occur during the compilation process. Most of the time, Numba's error reporting is helpful and pretty clear-cut. There are instances where it becomes a little bit difficult to figure out what the error means, and I'll show you an example in just a moment.
The first one that we're going to talk about is type inference. This is during the compilation process when Numba's trying to figure out what all of the types are that it needs to deal with. Numba must resolve every single Python variable, function, argument, and return value to a concrete data type, whereas interpreted Python does not require a programmer to do that. That process of type inference can be quite involved because it literally has to track hundreds or thousands of variables and how they make their way through the program in order to actually ensure that it's getting the right data type. If Numba encounters a case where type information is either unresolved or inconsistent, then compilation fails and an error is emitted. Sometimes these errors are very cryptic and sometimes it's actually hard to follow where the error actually originated from. I'll show you what I mean in just a moment.
Another kind of error is from when Numba translates from Numba IR down to LLVM IR. In theory, any problems with the source code or type inference should have already been handled through the Numba IR process, but in practice for code that's very complicated or for things that you may have done that subvert the guardrails, I guess to say, if you're using some of the lower-level functionality that we'll talk about a little bit later, there are times where some of those things either slip through, or it's something that the user has done to go outside of the normal process, and you can get a lowering error. The lowering errors are particularly difficult to track. Sometimes it will associate with Python code. Oftentimes, it's hard to trace back from whatever error it's given you to the actual issue. Oftentimes, we would end up going through internet searches and trying to figure out exactly how to get past those particular errors.
We're going to talk through a type inference error. Here I've used njit. Again, that's just a shorthand way of JIT with nopython. The return value from foo is either 0.0 or 1, the first of which is a floating-point constant, the second which is an integer. The actual return value is determined by the input argument n, and whether or not it's even or odd. Bar uses the return value to select between two elements of a tuple, and then it uses that to look up a value from a tuple, and then the print statement prints out the value. If we put in 7, interpreted Python runs with no problem. It goes through and outputs with no issues at all. With the Numba decorator applied to both foo and bar, however, an attempt to run the code generates an error. This is a type inference error. You can see that because at compile time Numba basically says we have an integer and a float, I'm going to promote that integer up to a float because that will handle both cases.
Then when it actually tries to run, it has a problem because it tries to use a float for looking up a tuple index. You can see that this error is difficult to track. It actually traces the error back to bar. If you can see at the bottom the blown-up part of the function instead of foo, which is where the actual type issue actually occurs. Now in this simple example, there's only two function levels. Although it's not pointing to the right one, it's not hard to trace back. In practice, in our application where we have many layers of functions because of all of the code that we had to put into Numba, sometimes these type inference errors could be very difficult to trace. It will point you generally to a function, but the actual layer at which that issue happens may be pretty far removed from the function that it outlines.
The other thing to notice in the above, there's all sorts of other references to built-ins, get items, signatures, UniTuples, candidates, and intrinsic calls that are really very low-level things that have really no real bearing on the actual problem, and can be really confusing, especially when you first start working with Numba. It should be noted that when you run Python with the exception when it actually hits it, it will just generate an error, but a quite helpful one, which is tuple indices must be integers or slices, not floats. This points to one of the ways that you can get around and figure out some of the issues that you have in Numba, and that's basically turning off Numba all together for some debugging and error correcting. That again is why we would use JIT with the argument instead of njit for the ability to turn off Numba wholesale to deal with some of these issues.
The next thing I wanted to go through is just debugging. The Python debugger, because it's an interpreted language, it's basically tracking everything as you run. It makes debugging very easy. If you're used to that, you're expecting to get all of the information that you need to debug the program at a very low level and very detailed. When you move to something like Numba, it does output information that you can put through the new project debugger, but using the new project debugger requires a lot more expertise than using the Python debugger. Similar to the error that we just talked about, what we would do to get around that, one way is just to fall back to the Python interpreted code, turn that on for debugging, turn it off when you want to go back to Numba and do something fast. A lot of our ways around some of the difficulties with Numba come back to having that decorator and using Python when we need to do debugging, and using Numba when we really want to run something fast.
Sometimes some of those errors, like the lowering errors, they're not going to be caught by the interpreter. That can lead to some iterative processes to try to figure out exactly what's going on, which can be frustrating. Again, I think the degree to which you're utilizing Numba will also be the degree to which you have to worry about those kinds of things. Another thing to be aware of is compilation time. The overhead of JIT compilation happens basically every time you run the code. That's the time to translate from the source code into the compiled function. That happens on the first call to every single function. Generally speaking, the drag of that, and I'll show you a pretty simple example later, still isn't enough to make up that difference between interpreted Python and Numba. There are times where it can become a significant part of the process.
I talked a little bit before about different data types and input arguments. Generally speaking, even if you can circumvent part of the compilation process, so even if the compiler recognizes that it's the same types, it still has to go through type inference, which is still a bit of a drag, so to try to figure out what the types are and if it already has a version. Any time you're calling from interpreted Python to Numba, you're going to have to at least go through type inference. If you're calling a JITed function from another JITed function, you can somewhat short-circuit that process, even the type inference, not fully, but it will be faster. Anytime it's from interpreted Python, you at least have to go through type inference. In our particular example, as we put more and more logic into JITed code and then try to find ways to ultimately make the runtime faster, what we ended up doing was making it run faster but blowing up the compile time.
One of the ways that we did that was inlining. I won't go too far into that. What we found is that forcing the model to inline, forcing Numba to inline would actually make it run faster ultimately, but it also meant that we had longer compile time. One way around that is to do ahead-of-time compilation. Numba does allow for that, similar to the way C or C++ would compile. You can precompile and then use a Python artifact and send that out, and basically you can fully subvert and get around the compilation time that you would otherwise have to spend. It does mean that the compilation may not be as tailored to the system that you're running on. When you run just-in-time, it can look at the actual machine that it's running on and tailor the compilation to that machine. If you ahead-of-time compile, then the performance of that version may be a little bit less than the performance of the just-in-time version.
Extending Numba
We've talked a little bit about some of the issues that we encountered. Next, I'm going to talk about some of the ways that you can extend Numba. To the credit of the open-source team developing Numba, a lot of the things that are exposed are things that they also use in implementing higher-level Numba features. It does require a pretty advanced knowledge of Numba internals to do some of these things, not all of them. Not every challenge that we've talked about or that we've encountered can be addressed by customizations, but I think it is something that's very important to have in mind and at least understand. The first one, the event API, this is more for compile-time profiling. It's a read-only thing. I'll show you a quick example of that in just a moment. There are also extension APIs, so these use Python code to generate LLVM IR directly and introduce entirely new object types accessible to the JITed code, so you can extend both the typing and lowering processes.
You can do things like function overloading, new and custom types, attributes, and create intrinsics if you want to. This is replacing some of those parts of the pipeline that I talked about before with your own code. It is very flexible in that regard. You can also customize the compilation process all together. Numba empowers developers to override or suppress default compiler behavior or inject entirely new compiler logic. There is a Python compiler class, and the pipeline represents the default behavior. You can basically change that if you want to and if you have the expertise and a good reason to do it. You can introduce custom passes, alternative pipelines for advanced optimizations. You can literally change the process for compiling and inject other ways of compiling.
I talked just briefly about the event API. This is one example of something that is exposed. Again, this is a read-only way to look at compile time information. At the top, we're implementing a listener from the event API, and then at the bottom we're registering that with Numba. This again allowed us to do some things with profiling the compilation process to try to figure out where it was spending all of its time. In the process of this project, we implemented this code to be able to look at how the compile time was split out. The other thing that we did, and this involved more of those low-level features, was looking at execution time profiling, so to figure out which functions it was spending a lot of time in. This involves more of the low-level things that I just talked about, so the extension APIs and compiler customization. This included Numba intrinsic functions. We went pretty low-level to be able to do this. I will say happily, this is not necessary now. Since the time that we went through this project, the Numba team has introduced the ability to get at this information without having to go through the steps that we did.
Review - OOP in Numba
Coming full circle back to object-oriented, we really wanted to have a means of doing object-oriented programming. Just to review quickly, jitclasses and structrefs, both of them are facilities to organize code and data together, which gives you class-like constructs. Neither of them work on GPU. Both of them cannot participate in subclassing, so there's no inheritance, which inhibits reuse of code and, of course, doesn't really allow for traditional polymorphism. One difference between the two is jitclasses are actually passed by value, so every time a jitclass is passed to a function, it's passing a copy of the whole thing. structrefs are passed by reference, so I'll talk a little bit more about that. Probably everyone's expectation, and what we also expected, is that structrefs would generally be faster because of the pass by reference. We didn't entirely see that bear out, and I think part of it is because LLVM is just so good at optimizing.
That was a theme throughout a lot of the things that we did with Numba. There were things that we found that seemed like ways of implementing things, like, for example, jitclasses passing by value that you may not generally want to do. Then, again, the performance that you see after LLVM is applied, we wouldn't actually see that substantial of a difference in the overall performance from doing something that seems, on the surface, naive. The implementation that we ultimately ended up with, the more tuple-based approach, simple data structures, it does the same thing. All of those data structures are passed by copy, and we thought that that would be inherently slow. LLVM just optimizes away a lot of those issues. Along that same vein, although jitclasses are passed by value, LLVM and the optimizer actually pass back the information, so it still looks like it's pass by reference.
Two other things, and more on structrefs than jitclasses, both of them have quite a bit of boilerplate code associated with utilizing them, which is also something that we were overall trying to avoid. All of this leads into a proof of concept that we did to try to implement and create a class-like implementation that would have less boilerplate and also be pass by reference. That was one thing that we were focusing on. This proof of concept had some limitations, so we didn't have full support for NumPy. We took some shortcuts in order to be able to get it to work and to get it to work well and pass by reference. We used things like custom types, intrinsics, custom compiler passes, very low-level things to create our own version of object-oriented programming in Numba. It did have very high performance. We'll see in a moment it was actually the most performant of the class-like implementations. It did not have inheritance, so we did not get to the point where we were able to try to implement that. It had limited polymorphism. We did not have enough time to get it to work for GPU yet, but we do actually think that we could. It's something that we're still looking at and exploring.
Performance Benchmarking: BEL Model
Just a very simple example to compare some of these different implementations. This is not the model that has the extent of runtime that we talked about before when I was talking about applying the GPU, but this is a European standard Solvency II. We're running 100,000 policies, 50 years over monthly projections, so 600 timesteps for every policy, and basically projecting out all of their cash flows over all of these contracts. In practice, a lot of our clients, that number of policies would be in the millions range, so you can see how that starts to scale up and become something quite meaningful. This particular product is a bit simpler than the annuities and the other examples that I talked about at the beginning. This shows the compile time as well as the runtime for some of these different implementations. You can see all of them are quite a bit faster than interpreted Python.
Again, this is a simpler type of model, and we actually weren't seeing the 75 times that we saw for some of our other implementations on a larger-scale model. We did see that the proof-of-concept model was faster. We think part of that is probably maybe some of the shortcuts and the optimizations that we made when we were implementing our way of accessing data in our version. I don't know if that would hold up as we start to expand. It was encouraging to see that we were able to get back to something a little bit more object-oriented, and we feel like we're on the path to being able to expand on that.
Algorithm Design
An algorithm design, and this is more or less just to get people thinking about it. Especially in my world, a lot of times there are lots of different ways to do things. In terms of modeling these products, it's really important to understand not only the computer science, the programming part of it, but what the underlying algorithm is. One example of this is put-call parity and valuing options. There are annuities where you have to value through Black-Scholes puts and calls basically every timestep. If you understand put-call parity, you understand that you can get the value of a put and a call with one valuation, and the valuation of Black-Scholes involves calling to the cumulative normal distribution, which is actually very slow. It's one way for that particular function, just by understanding that characteristic of the problem you're trying to solve where you can really reduce your runtime.
Conclusions and Key Takeaways
Finally, Numba can really have dramatic performance improvements compared to numerical Python. There are challenges. I've touched on quite a few of them. There are ways around many of these challenges. I know I've said this multiple times, I think a lot of the challenges are probably very specific to the extent to which you're trying to utilize Numba. If you have pretty isolated code that is numerical, it's a few simple functions, then I think Numba is probably very obvious that you want to at least try it out. If you have a lot of logic that's going to have to go into Numba, you may encounter some of these other things that I talked about. Overall, I think it's a very important tool to have in your toolbox. Then, finally, just again, I know I went through algorithms pretty quickly, but there can be very subtle things, and just understanding the problem at times can allow for an algorithm that's very efficient. We've seen that time and again as we've gone through client models and have rewritten them to different languages. Oftentimes, part of the speed improvement isn't just the new language. It's actually redoing things in a way that isn't necessarily purely programming-based, but also based on utilizing foundational knowledge to the problem that they're trying to solve.
References
I'll leave you with a few references here. This is to Numba documentation, LLVM. Then, finally, we had written a white paper that goes into a lot more depth on basically this entire presentation. If you're interested, you're welcome to go and find that and learn more. Again, there's a lot more detail in there.
Questions and Answers
Participant 1: Is there any ergonomic considerations when adopting Numba? As a developer, if I were to adopt it into my existing ecosystem, just from a developer ergonomic perspective, does it seamlessly work out of the box, or should I be aware of some build tooling complexity that comes with it?
Chad Schuster: It's pretty seamless. Once you download Numba through pip, it just works. Other than importing it into your files, it's really just accessing those decorators. It gets more complicated if you want to then use CUDA and GPU. For the most part, if your main interest is getting Numba to work on a CPU, it's actually fairly straightforward in terms of the tooling. For the biggest implementation of Numba that we've used, there have been points in time where upgrading to a new version of Numba took some amount of effort because of the way things were implemented or changes that they had made. Overall, to just get started, it's fairly straightforward.
Participant 2: We use pretty much Python for the financial computation, running the risk models. It's like plenty of data. When we run with Python, and there also we call some Java libraries as well. When we use Numba, do we get any conflict between the Python Numba versus Java?
Chad Schuster: As long as your usage of Numba and the decorators is isolated from Java, I think you should be fine. I don't know if Numba and Java could talk to each other. Numba gets compiled down to what effectively, I think, is C and then machine instructions. I don't think there's a lot of consideration for interoperability with other languages. If you can isolate your usage of Numba to specific functions that are currently just in pure Python, then I think you shouldn't have any problems.
See more presentations with transcripts
