Rúnar Bjarnason on Scalaz, Lenses, Functional Programming and Scala

Recorded at:

1. We’re here at Scala Days 2012 in London. I’m sitting here with Rúnar Bjarnason. Rúnar, who are you?

Well, I am a software engineer. I work with Scala so I write a lot of Scala in my day job; in fact, I write almost exclusively Scala and we’ve had a lot of success with it. I’m a contributor of scalaz library which is sort of a kind of extension to the standard library or it’s a third party extension, has all kinds of useful facilities that the standard library doesn’t have; and I’m also an author, I’m writing a book about functional programming in Scala.

2. Let’s start off with scalaz; so you’re saying scalaz brings features that the Scala standard library doesn’t have; but I thought the Scala library was perfect. What is missing?

Well, for instance, the Scala lets you do monad comprehensions, so if you have data types that are monads, Scala has a special syntax that lets you take advantage of the fact that these things are monads.

So all you have to do is implement a certain interface but the standard library doesn’t provide, for instance, an actual type for monad; it doesn’t give you the ability to abstract over all monads for example; but you know, you can write that yourself or you can use the monads in scalaz and that’s no problem.

3. So what other features are in there?

Runar: So many.

Werner: One of the most...as far as I know scalaz is a bit controversial, why is it controversial?

Runar: Is it? I don’t know that it’s controversial. I guess a lot of people think that there’s a lot in there and a lot of people think that some of the things that are in there are too complicated, too high level, too abstract but I don’t know, not everything is useful for everybody; so some things are super easy like, for instance, validation. Validation is something that anybody can get into. It’s basically a data type, you know, it can be one of two things: it can be either an error or it can be a value, that’s not an error ; and it provides facilities to accumulate the errors so that if you’re validating say, a web form, you can just kind of compose validations for the individual elements in your form and then it will accumulate the errors for you if there are any; and that’s something that’s very useful.

Another thing that people find useful and can easily get into is monoids and monoids allow you to sort of abstract over things that you can add together; for instance, you can add numbers together; you can add lists together by concatenation; but it’s really compositional so that if you have maps of lists of tuples of functions of things, then you can add those together as well to an arbitrary nesting.

And it all just sort of works automatically; there’s nothing extra that you have to do except import scalaz and then smash things together.

4. So a monoid comes from Algebra or Category theory, it's basically just addition?

Yeah, a monoid is an addition with a zero or a multiplication with a unit, so that’s it.

5. So it has to comply with certain laws like the associativity and things like that?

Yeah, monoid has to be associative and the unit has to be a unit for the operation.

6. So they’re defined for all these different types like lists; or other things?

Yeah, there’s a large amount of them and we provide a lot of them in scalaz; but you know, you can make your own.

7. What are some of the features that you worked on in scalaz?

Runar: So many.

Werner: What’s your favorite one?

Runar: I don’t know. I like the I/O monad so I implemented the I/O monad; and I’m surprised how many people are using it because I just thought it might be handy to have and so I implemented it and stuck it in the library and now people are using it all over the place and it’s kind of interesting.

8. So maybe for our audience, the I/O monad is the one from Haskell or is it a different idea?

It’s based on the one from Haskell; it’s essentially a state transformation so that, you know, you have these I/O actions that sort of in theory, you pass in the real world and it gives you a value and the next state of the world but we don’t actually reify the real world; the real world is basically just null; so it’s sort of at the outermost layer of your program, you run these I/O actions by passing them.

9. These are sequentializing operations in the monad basically?

Yeah, instead of running your side effects directly in your program, your program constructs a program an I/O program that you then run as your main; so your main, it basically does nothing except to run one of these I/O actions.

10. So the I/O monad is basically the interpreter for this I/O actions?

Runar: Yeah, exactly.

Werner: So the audience can see that monads are actually useful.

Runar: Yes, they are I think, I mean, it’s a nice abstraction to be able to reach for.

11. So what are other features of scalaz? Is it mostly about comprehensions, monads? What features from Haskell do you miss in Scala?

Well, you know, having an abstraction for monads and monoids is an obvious one; and you know, that’s just a library issue; it’s not really a difference in the languages; but I mean, what I miss from Haskell when I go to Scala is laziness, that’s like the big thing that I find, that I want in Scala. I mean, we have by-name parameters but they implemented as functions really and so it’s not quite the same; and so being non strict by default, being call- by-need by default is something that I miss when I move from Haskell to Scala.

12. So laziness means that nothing gets evaluated until you read it basically?

Yeah, it just means that you write your program but it’s not actually evaluated until you look at what it actually is; you pattern much on it or you pass it to something that actually requires it.

13. And you have to do that in Scala explicitly with the by-name convention?

Yeah, so Scala is strict by default but then you can annotate arguments to functions to be non-strict, but they're by-name so they’re evaluated every time you reference them but you know you can cache them in a val or something like that.

14. So your talk yesterday, you had two talks and one was a talk on lenses; for the functional program hipsters, is the lens the monad of 2013?

I don’t know about that. A lens is something very, very simple and it’s actually ... so it wasn’t really my presentation. I was presenting a paper by Tony Morris who is also my co-author on the book; and so the paper is about lenses and a lens is just like a purely functional reference, it’s a way for you to focus in on a part of a larger structure and either modify it or get at it, sort of focus your context, that’s why it’s called a lens.

15. Are they somehow related to zippers?

That’s a good question. I suppose they are, I mean, I suppose it’s a sort of similar kind of thing where you have a focus and they both involve co-monads; so there’s probably a connection there.

16. They’re both about mutability or introducing mutability?

Yeah, they’re both about having a purely functional pointer into a structure and being able to update that structure; so there’s probably a deep connection there that we need a little time to think about.

17. There are bi-directional lenses which map you can map things to one lens and then map backwards?

Yeah, so bi-directional lenses are something else; bi-directional lenses are where you can take some input to your program and then that gives you some output but then you should be able to edit the output and go back to the input that would have produced that output; so you can sort of make edits on both sides and sort of like a bi-directional edit; so you've probably seen an HTML editor where you know, you’re editing your HTML source and then you have a rendered page and you can edit them both and they both update each other, that’s the kind of thing you would do with bi-directional lenses.

But asymmetric lenses are what I was presenting and so it’s only in one direction.

18. So we’ll look out for lenses in the future?

Runar: For asymmetric lenses?

Werner: For all of them.

Runar: Yeah, the more the merrier.

19. You mentioned you are writing a book; what’s the name of the book?

The title of the book is Functional Programming in Scala and it’s to be published by Manning and we’re hoping for either end of this year or sometime beginning of next year, we’re not quite sure.

20. So what’s the focus? Is the focus on functional programming or Scala?

The focus is on functional programming. We really want to teach functional programming and use Scala as the vehicle to get there. So Scala is not the cargo of the book, it’s just how we get there.

21. So it’s basic concepts?

Yes, we start with basic concepts like what is functional programming; what do we really mean; it’s like referential transparency and purity and why you should care; and then we sort of move from this restriction that you put on your programming; this restriction that you can only write referentially transparent functions and from there, we have to revisit the entire act of programming. We have to build up our entire toolkit from scratch, things like handling errors, you can’t throw exceptions in purely functional programming and other things like looping, for instance; you don’t have variables so you can’t loop, you have to recurse instead.

And so we take it through all of these tools and these abstractions that sort of fall out of it and we tried to take you all the way to being productive with purely functional programming.

22. So you also, I assume, you also talk about how to structure bigger programs with functional programming?

Runar: Yeah.

Werner: What’s the approach there in Scala?

Runar: Well, the approach that we take is we start by introducing the concepts. The book is in several parts: and Part 1 is just sort of introducing the concepts and then Part 2 is we look at a whole bunch of libraries and the reader will sort of develop libraries in a purely functional way; and then in the following part, we abstract over libraries; so we create libraries that support other libraries and so on and then we will have a part that talks about programming against the real world, databases and enterprisy things like that.

Werner: In your work, you use Scala everyday?

Runar: Yes, everyday.

23. Are you working on greenfield projects or integrating other projects? Do you have any problems interacting with people who are not familiar with functional programming?

Yes, so I’m mostly doing, yes, greenfield projects. There’s one big project that I’m working on, that’s totally greenfield; but at the end of the day, it’s supposed to interact with a larger application.

And we find that you know, there are little libraries that we can write, even big libraries we can write like we have a like an analytics engine that’s written in Scala; and you know, we plug this in to a larger Java application and then we find that at the edges of this library, people start writing Scala to interface with it and then it sort of grows out like a zombie virus.

24. You're infecting outwards?

Yes, we infect outwards and in fact large parts of the system have been rewritten in Scala and we found revisiting how we do things and using functional programming concepts we can get like a factor of 10 line count reduction which is pretty good; it makes the code more compact and comprehensible at a glance.

25. Fewer places for bugs to hide?

I don’t know if it has fewer bugs, I can’t promise that but it’s definitely easier to look at.

26. So we’re here at Scala Days.It’s a great conference and we’re assuming that Scala is perfect but for a moment, what are your ..do you have any issues with Scala, anything that bugs you to see solved?

I don’t have any major issues with Scala, I mean, I’d like type inference to be better; I’d like to have kind variables; so I’d like to be able to talk about kinds in a way that is better than now and I’d like to be able to do a partial application of type constructors in a much more straight forward way but these aren’t major complaints, these are just little annoyances; so I have to do a little more typing than I’d like in both senses of the word.

27. So is there any particular feature that you might see added? So here at Scala Days, a lot of people are talking about macros, is that something interesting to you?

Yeah, that does interest me; there’s a lot of boiler plate that we have to write and I think that macros could definitely help with reducing that kind of boiler plate. In some cases, I’m even generating code. I’m using Scala to generate Scala code for things like Algebraic data types and I think having a macro to do that for me would be much easier, exciting stuff - the stuff that’s coming up.

28. So compiler macros would be interesting?

Runar: Oh yes definitely and type level macros too if we can get those.

Werner: What are type level macros?

Runar: Well so that you could not only it generates programs and that kind of things but also types so that we could do things like, if we had, if you wanted to have vectors of different sizes and you wanted to generate different types that are based on certain numbers; so I think there’s a lot of potential and it’s exciting to see that Scala is an evolving language, they’re not afraid to take it forward; whereas with Java, you see it’s just stagnating; I mean, Java hasn’t been updated in a decade but it’s exciting to see that Scala is moving forward. Werner: It’s a benefit for Scala.

Runar: Yeah, I think so too.

29. Our special ScalaDays question today is, Rúnar Bjarnason, describe yourself as a monad.

As a monad, well if I were a monad, I would be the reader-writer state monad transformer because I can take any other monad and turn it into a monad that has the capabilities to read an environment, to write, to some log or something and to modify some state and you know, that’s all you need really at the end of the day.

30. So you’d be the imperative monad basically?

Runar: Yes, basically and I’d be the ultimate program writing monad.

Werner: Is that heretical to be the imperative monad and a functional programmer?

Runar: No, I don’t think those two things are

Werner: ...they go together

Runar: They absolutely go together and in fact, I think that functional programming is the best way to do imperative programming; so if you’re purely functional and everything that you do is referentially transparent, then you can reason algebraically about imperative programs and I think that’s a very powerful thing.