Transcript
Ward: Welcome to the functional evolution of object oriented programming. I'm James Ward, a developer advocate on Google Cloud.
Suereth: I'm Josh Suereth. I'm a staff engineer at Google Cloud.
Cycle of Learnings
Ward: Let's talk about the cycle of learnings. When we are using some paradigm, let's say, object oriented programming or whatever programming paradigm you want to pick, oftentimes we'll find some limitations in that paradigm. For example, I used to work with Java EE, and found some limitations with doing things in the Java EE EJB way.
What's the limitation that you found with Java EE?
Suereth: It was actually quite hard to test. The objects were pretty heavyweight and had a lot of stuff associated with them. It was really hard to get a good isolated test on anything.
Ward: Then what we did to deal with those kinds of limitations was to develop some patterns. For example, in this space, we started using XML dependency injection and just POJOs, instead of the things like EJBs, where we had to extend interfaces and that sort of thing. That was great that we had some patterns to help us deal with this. What is happening in programming, what continues to happen is that when we develop these patterns, oftentimes, these patterns will percolate into the underlying programming languages and frameworks. For example, following on with XML dependency injection and that pattern, was we developed a new pattern for doing this with annotations. That allowed us to evolve our programming language and our frameworks to deal with some of those original limitations.
Example - Calling a Function
Let's dive into some of these ideas a little bit further. Let's start first with an example where I'm going to call a function. We're going to play a little game. I've got a function here called addOne. You'll see I parse an integer here of 2, and get an integer back. Let's run this thing and let's see what happens here when I run this. I'm going to have you guess what actually happens. I've got a function. It takes a parameter of 2. What do you think the output is going to be here? What do you think that's going to be?
Suereth: It's should be 2 plus 1, which is 3.
Ward: Let's try it. Three, you are correct. If we call that same function, same inputs, again, what do you think is going to happen this time?
Suereth: It should be the same thing.
Ward: Wrong, it is 4. What about now? What if I call it a third time? What do you think it's going to be?
Suereth: It should be three. In this case, I'm going to guess it's 4.
Ward: Wrong, it is not even an integer. It is null. What do you think it's going to be this time?
Suereth: It's going to be null plus 1, which I think is 1.
Ward: Actually, it is an IllegalStateException, which is not even a value, not even a return value on that one. Let's go look at addOne, to see what was actually happening. We have a function that's taking an integer, but you'll see I've got this global mutable state here, which this function is mutating. That's how we were getting different behavior on each call to that thing. We're going to talk about some ways to do this differently.
Let's look at another example. Nothing hidden underneath the covers here. I've got my addOne function and calling it. Let's play our game one more time. No funny business down there. No imports. I'm parsing 4 to that addOne, what's it going to return?
Suereth: It should return 4 plus 1, which is 5. Although [inaudible 00:04:10].
Ward: Yes. We try again. What's it going to return?
Suereth: Five.
Referential Transparency in Pure Functions
Ward: It works. That's our example of referential transparency in pure functions. Tell us a little bit more about those?
Suereth: The idea here is you want the ability as a programmer to look at a method call, and be able to mentally replace it with its value, everywhere you see it. The exact same method call with the exact same value, because it really helps you understand what is going on in your code. It really avoids some of those magical confusion moments of, I didn't realize that I was bytecode rewriting some information for logging over here, and I threw an exception, and that totally crashed my program. I don't know, when I look at this method, to look over there and see that. Or, when I'm using global mutable state, I have no idea what the state of the global mutable state is when I look through the method, so debugging just becomes that much harder. Referential transparency as much as you can preserve it, can really make it easy as 4 plus 1, in terms of debugging what happens.
Ward: Pure functions like this addOne function here, what they do is they will always map a given input to an output. What that means is that every time I call this function, if I call it with 4, it's going to produce the same output every single time for that given input. This makes it so we can understand our programs much more easily. We don't get into weird states, where it's like, how did I get into this state? It makes it a lot easier to deal with our programs in this way.
This is an important concept in functional programming. In that cycle of learnings, it is one that is propagating much deeper into our object oriented programming, and so that we can have programs that are easier to grok, easier to debug, easier to understand, and don't get into weird states that we don't understand how. This is our first set of concepts: immutability, pure functions, and referential transparency. What else to add on that?
Suereth: This is really the evolution of the, prefer immutable objects from effective Java. When was that written?
Ward: 2001, almost 20 years ago.
How to Construct an Object
Suereth: I was thinking, we might start talking about the hardest problem in object oriented programming, how to construct an object. Here, we start with a vehicle class that we want to build. Our vehicle class has a lot of parameters here.
Ward: Apparently, most vehicles have four wheels, because that's our default.
Suereth: That's our default. Vehicles can have less than four wheels or more than four wheels. What's interesting here is we have different constructors to instantiate the vehicle with different set of parameters. Each time, we have to instantiate all these different fields. A vehicle can have a name, and then it uses the defaults, or you can actually parse a value instead of a default. Then we can use these optional values. If you wanted to have a six cylinder vehicle instead of an eight cylinder vehicle, or a no cylinder electric vehicle, you can do that.
Ward: One of the great things in this example is that we are using immutability. Our static class, vehicle, is immutable. That does mean we need to have all these different constructors to assemble the different possible ways to set those immutable values.
Suereth: Now we have a lot of duplicate code. It takes a little bit more to maintain this. Every time we want to add a new optional parameter, we add a new entire constructor.
Ward: That's the typical way that we do POJOs in Java. You can see it's cumbersome. What did we do? What was the pattern that we developed to help us with this?
The Builder Pattern
Suereth: I believe it's called the builder pattern. We can actually make another object that makes it easy to construct this object.
Ward: Let's go down to our VehicleBuilder. Here it is. What's happening in our builder?
Suereth: The VehicleBuilder stores all of the possible state that you'd want to parse into the constructor, effectively. It allows you to call methods to apply pieces of the state over time, including optional ones. Then it has a build method that finally calls a single constructor with all the parameters at once. You end up with one single point of construction for the class. The builder controls the flow through optional parameters and different ways to construct that class.
Challenges with Builders
Ward: This worked for a while. It has helped improve us from where we were with all the constructors. I've had a lot of challenges with builders. For instance, builders usually aren't very good at conveying some of the semantics about what is actually needed by the underlying POJO data structure. For instance, if two fields are related, like I set one field and then I need to set another field. That's usually something that is not conveyed through a builder. What can happen is you call build on the thing, and then it throws an exception because you didn't do something right. That's one of the challenges that I've had with builders, in particular, is that it just usually doesn't convey the actual semantics of what the underlying thing needs.
What other challenges do we have with builders?
Suereth: You actually have to still maintain both sections of code. You have to maintain the POJO and you have to maintain the builder. If I make changes to the POJO, I have to make changes to the builder at the same time. Actually, the cost of ownership is doubled.
Ward: That's true. Then also, with builders, things can get pretty crazy when we start having hierarchical things that we're building. Let's say we have our vehicles, but now we have owners and owners have vehicles. Our syntax starts to get pretty gruesome here for how we'll build a hierarchy of objects. You can see just a lot of code here. We oftentimes get to the place where we have to be able to go from our immutable POJO back to a builder, so that we can make modifications and then rebuild it. Things get pretty nasty, pretty quick, with hierarchical objects. What else on that?
Suereth: This reminds me of trying to remove a member of a repeated field of a repeated field of a protocol buffer.
Case Classes
Ward: What we are seeing with the functional evolution of our object around languages are some different paradigms for how we construct objects. In Scala, we have something called Case Classes. In Java, what is coming in JDK 15, or something like this, is something similar to Case Classes, which are I think, record types. Kotlin has data classes. A lot of the object oriented languages now are adding much easier ways, much better syntax for constructing objects.
Walk us through what's going on here, in this Case Class.
Suereth: Primarily, there's a philosophical change in terms of how you design and craft your objects. There's a few components here I want to call out. One is, the parameters can have default values, and then not be specified. You don't need a builder to be able to have default values anymore, because you can actually account for that in your constructor. The second thing is you can actually have named parameters when you construct. If we look here, with this vehicle, you can use the name of the parameter, so you can provide only the optional ones at the far right of your constructor, so that helps a bit. The other thing is, all of the arguments to the constructor actually become members of the class. You don't have to make getters and setters. If you look at CaseClass Owner there, it's quite simple. It's just I have a name, and age, and vehicle. Those are the pieces of data I need to know about. Those are the pieces of data I construct. Then the language builds in convenience mechanisms for you to build these effectively, and to modify them effectively. If you look at that copy method, before we'd have to call .tobuilder, and then modify the parameters that we want. In this case, the copy method is generated, and gives us the same parameters as our constructor. We can actually construct a new copy that has all the same data, but with a few things different.
Ward: This is, I think, definitely indicating that cycle where we made some pattern changes with builders, and now we're getting great language level support for constructing objects and making it much easier.
Putting Together Functions
Suereth: I think we're on to the next topic, which is, we've created these nice ways to construct objects and things. What if I want to take a bunch of functions and put them together, what do I do?
Ward: Combining objects, we can do through a variety of ways, which we saw through Case Classes. What if we just have functionality that we want to combine together? We can actually use some of the things that we learned about in object creation to do this. For instance, let's say I have a function, my function is called balanced. I'm still working with my vehicles. What it does is it basically just says, is this thing even? That's certainly my function. This is now a predicate of integer. Then we have another one called beefy, like, does it have more than eight cylinders? Then it's beefy. We've got two predicates for integers. We can apply those. On our number of cylinders, which is our option, we have a way to filter on that option with that function. We can filter on our option, number of cylinders, with our beefy function. That's all great.
What we can do is we can actually take these two predicates and combine them together. The way that we do that is that predicate actually has functionality on it called combinator, has a number of combinators. One of the combinators is OR. Because predicate has a function on it called OR that can combine with it another function, now we could take these two functions, combine them together into a new predicate. Then we can use that new predicate in our filter function like before. This is an immutable way to construct a new object. That new object is both of those predicates combined with an OR.
Suereth: Is this like a builder from smaller components?
Ward: I have had arguments about that with people. I think it looks and feels a little bit like a builder, but you don't call that build on it. You're not doing this mutable state, where you're mutating and then calling build. I wouldn't really say that this is the builder pattern, I would say that this is just a better built-in way through combinators to create new combined things.
Suereth: You're still achieving the same goal of small classes with clear, coherent responsibilities, and then clear operations to join those together to make a complex behavior.
Ward: Doing it all immutably.
Algebraic Data Types (ADTs)
One of the important evolutions that's happening in object oriented programming is algebraic data types, ADTs. ADTs allow us a better way to represent data structures. There's primarily two different ADTs that we can work with. There is a product type. A product type allows us to combine together values. For example, let's say that we want to combine together two values into a pair. We can create a pair object to do that, or our language may have something like this built in. If we want to combine three different values together, we can combine them into a triple. There is language support in many languages for doing this without having to give them names. These are called tuples. This is just combining a bunch of values together.
Suereth: It's like those data objects we were looking at before.
Ward: That's right. Exactly. This is called a product type. You can call it an AND because it's ANDing together values, or you could also call it an intersection type. Different names for this. There's another one which is an OR, so using those algebraic ideas of AND and OR, into our type system and how we combine values together, so now on to the OR, we can do this. This represents what's often called an Either, so something that can either be an A or a B, for example. We don't just have to deal with two values, we can have things that have three values. Let's say it's a Trither. A Trither can have an A, B, or C. We combine these together. This is called a sum type, or you could call it an OR, or a union. These data structures, this way of representing these types of values can be really useful for being able to have our type system convey the semantics that are actually allowed. Oftentimes, what we would do in the past was maybe just leave one of our values as null and set another one. That doesn't actually encode into our types, the actual legal values that are possible. With ADTs, we can much better encode them. One of the ways to represent the OR is with this Either type. There are languages that are adding direct support for sum types into the languages with special syntax and making it easier to construct and deal with these sum types.
Suereth: Most of the languages already have support for some sum types. Can't we just use inheritance?
Ward: We certainly can. That's exactly what this example is doing, because I'm in Kotlin, and Kotlin does not have actual direct support for sum types. What I've done is I've done this through a sealed class. Because I've made it sealed, you can't extend this anywhere else. Only right here can we actually extend this Either type with our actual implementations, left and right. You'll see that left has a value left, right has a value right. Both of those extend Either. Then you could see the example for Trither there too, pretty similar.
Pattern Matching
What has happened in a lot of modern languages that are using and supporting ADTs, is pattern matching. Pattern matching that is able to look at the actual classes, and be able to do smart things with it. In this example, we're back to our sum type, our Either. You'll see I'm using a left and a right sum type here. Now I'm going to do my pattern matching. I'm just matching right now on the r1. When r is in Either.left, then I can do something with it. Kotlin has some nice syntax here where it does this smart cast. It's like, I know that r is in Either.left, and so I'm only going to allow you to access that left property. I can't access the right property. Left didn't even exist on right, so right has right, and left has left. Kotlin, when we do that pattern match in our function that we're going to handle this with, it does that smart cast for us.
There's another nice thing with sum types and pattern matching is that if we don't handle all of the cases here, so if I comment out this one, then Kotlin gives us a compiler warning. It says, "You're not exhaustive. You are pattern matching on a sealed class." It knows that in that sealed class, there are two things, there's a left and a right. It's now warning us, it's not exhaustive. When I'm writing Kotlin, I actually turn this warning into an error. Scala has a similar feature. If you're not actually explicitly handling all the possible cases here in your pattern match, then it should be a compiler. That's what I do. Pattern matching allows us to replace that visitor pattern with something that is much more concise and easier to deal with.
What else to add on that?
Suereth: Aren't they discussing bringing that to Java after record types are out?
Ward: You are absolutely right. I don't know if that's JDK 16, or something else. Yes. Pattern matching in this way. Not direct ADT support. We can do the ADT part in other ways in languages today. Maybe at some point, Java will get actual language support for ADTs. They are adding the pattern matching, which would be really nice.
Extension Functions
One of the great language features that we've seen started to be added to a lot of languages is extension functions. Extension functions are a different way to do polymorphism, where we can add functionality to classes, just by creating functions that add that functionality. In Kotlin, the way that we do that is we create a function, Iterable. We can even use generic, so an Iterable of char. We're going to add a new function available on that thing called sum. When we want to sum up chars for some reason, we can do that with its extension function. If we've got our list, which is an Iterable of chars, we can call that sum on that. That'll just work.
If we continue with this example, we'll see, now we can add an Iterable of string, so that we can take strings as inputs in our Iterable list and do that. We've done this all with concrete types in that Iterable of whatever. What if we wanted to make this generic? What we can do is we can have an abstract class to char of t. Then have this invoke operation on it, which is going to return char. Then we can do a string to char, so now we create a new function that is going to be able to take a string, turn it to a char, and get the first one. Then charToChar, so if we've got an Iterable of char. Now we can do an extension method that is on our Iterable of t, but it needs to actually take this toChar converter that knows how to take that t and convert it into something that then we can do our fold on.
We've got this new method, sumT that we can call. We can say, list of chars, sumT, but then we need to give it our charToChar. That's great that we can do these extension methods with generics. Things start to get a little bit weird here because we're having to parse into sumT the actual converter that we need. Let's say we wanted to be able to do an Iterable toChar, so we've got an Iterable of t. Then we need to define that one, which we can do. It's a toChar of Iterable t. Then, that one needs to know how to convert the t to a char. Let's say we've got a list of a list of chars, and we want to be able to sumT those, now we have to provide the whole chain of things that are needed to do all the conversions. We need to parse that all in explicitly. The extension methods are great, but using them in this way to do these conversions requires us to be explicit about the ways that we're going to do those conversions, and we have to provide them.
Rust
Let's look at Rust, which has a better way to handle this particular case.
Suereth: Effectively, in Rust, it does what's called derivation, where it will attempt its best to construct these generic types for you from the specific type that you have. In Rust, we define our toChar as a trait. It's equivalent of like in a Java interface. It has a single function, toChar, that takes in the existing instance of a type, and then returns a character or char. Then we define implementation. For a character, just returns itself. For a string, we just grab the first character off and return that. Then there's this nice implementation where we can take in an abstract type and define it against any vector of any type. Say, if I wanted to find the implementation for a vector of any type, just give me the implementation for the type of the vector, the T. Then I will continue to unwind these things.
When we define sum, we just have to define one toChar interface, for any possible vector of T that we have. We can sum over this and it will infer through things, what we need. Here, we only provided toChar for generic vectors, for characters and for strings. In this println, you see if we parse in a vector of characters in the first example, that compiles and works. If we parse in strings in the second example, that compiles and works. If we parse in a vector of vector of strings, that will work, because it will recursively look through the vectors and infer that, the toChar for a vector of vector of strings is the vector of strings parsed in. It does all that magic for you. Same if we do a vector of vector of chars.
Example
You can run this to show an example. Then I will show you what failures look like.
Ward: We can run this one, and it'll work. The important thing to highlight is that this is like the Kotlin extension methods that we saw, except for we don't have to do that explicit parsing of the types that are needed, like we did in Kotlin. That one works.
Suereth: We know integers don't work. Let's call it an integer.
Ward: Let's go back, and let's put an integer in there. Then let's try to run this compiler again. What happened?
Suereth: You're getting two errors. One is it says that you can't make a vector of vector of integers if you parse characters through one of the integers. You're right. Thanks Rust. Also, when it tries to find the trait toChar, it tells you it doesn't have one for integer. It doesn't tell you it doesn't have one for a vector of vector of integer because it knows it can follow vector of vector. It just says, I need one for integer. Go write one. That's it. It's wonderful. The compiler is doing all this work for us.
Ward: This is a great evolution for OO. The name commonly used for this is type classes, but I think Rust actually calls them something different from type classes. What does Rust call this?
Suereth: I believe they're called traits, and type traits. C++ calls them type traits.
Ward: Different names, but this idea for the compiler to implicitly find what it needs to be able to fill in function arguments, essentially, because we've got our sum method, and it needs a toChar that can operate on that vector of T. Ad hoc polymorphism is also another term for this.
Summary
Let's pull together a lot of these concepts that we've walked through, into something that will make sense of why we need these things and how they can be useful. In this example, we've got our vehicle. You saw our vehicle earlier. It's our Case Class. It's immutable. It has those default parameters. We don't have any builders. Let's say that we want to do JSON. We want to go from JSON to a vehicle. We want to have some validation built into this as well. We're going to use combinators to build up some validators to validate our JSON. We're using something here called verify, and we're going to verify of string. We're going to get the first letter in that string, and we're going to verify that it is uppercase. If it's not, we're going to return an error. This is using some functions. This orElse is using a combinator to do our verification that a string starts with an uppercase letter.
Then we're going to also do a different validator, which is, we're going to validate that the length is at least 2. We're going to combine that using a combinator with our, startsWithUpper function. We're using combinators and we are building up a validator. Let's actually create a way to read our JSON object into a vehicle. We're using combinators. This and is a combinator that's going to take something that reads a property name, name, and uses the name validator on that thing. We're going to do a read[Int], but you'll see that for read[Int], I didn't actually tell it how to read an Int. This is where we actually see those type classes, that ad hoc polymorphism come in, is that read actually takes a read of t. It's parameterized like we saw before. This is in Scala, but you saw something very similar in Rust, where there is something that knows how to read an Int and handle that. We can also read a nullable int, which is going to be an option, which is our ADT or sum type for something that can have a value or not. We're seeing a lot of these concepts that we saw, ADTs, combinators, type classes all coming together so that we can create something that knows how to read JSON, knows how to validate that JSON, and turn that into our Case Class of a vehicle.
Conclusion
What do you want to clarify?
Suereth: Then I think the most important thing is if you look at this JSON that we take in, if we take a name and num_wheels, we're not taking its cylinders, we can actually parse that JSON, run our validation logic, get our errors out. Here, the response is an ADT itself that we can pattern match on, and say, if we're in the error state, we do some errors. If we're in the success state, we print a value. You can really start to see these things come together, this notion of composing small bits of logic that are easy to test, easy to reason through. Making sure that my combinations of that logic is easy to understand and reason through. Then build a very complex program from simpler components. That's really the focus of what we're going after here with functional program. That's really where we see huge benefits for Java developers, small simple components compose together.
Ward: We haven't even talked about the benefits of testability of this stuff. That's certainly a huge benefit of all of this, is being able to test these small pieces independently, and validate that they are correct. That is our example that brings it all together, and shows you how you can use immutability, how you can use Case Classes, ADTs, how you can use ad hoc polymorphism for a serialization example, which is just one way that this all comes together well. There are many others to explore as well.
Hopefully, that was useful for you to see how programming is changing and evolving, and how object oriented programming is, in some ways, becoming more functional to allow us to test more easily, understand our code, all that stuff. If you want to find the source code that we've been walking through, that is all up on my GitHub, github.com, jamesward, and then oop-evolution.
Suereth: I just want to encourage you to take a look at some of these languages coming down the pike and seeing what's coming in Java, because it'll make it there.
Ward: That's right. Evolution happens.
See more presentations with transcripts