I am Sadek Drobi, and I'm here at Erlang Factory with Francesco Cesarini and Simon Thompson. Can you please tell us a little bit about yourself and what you have been busy with lately?

Francesco (F): I am Francesco. I have been working with Erlang for about fifteen years as a software consultant, founder of Erlang Training and Consulting, with offices in London and just finished authoring a book "Erlang Programming" which is published by O'Reilly.

Simon (S): I am Francesco's co-author so we have both just finished authoring the book. I am Simon Thompson from the University of Kent. I am an academic teacher and researcher; I have been working in functional programming for about twenty five years. With Haskell previously, more recently both with Haskell and Erlang.

So you have just written a book about the Erlang programming. What are your target readers?

F: People who are new to Erlang people who want to learn Erlang and learn it in depth. From my side, when I started writing the book I was thinking of my students and lecturing in class, so the actual writing was based on a training material I just started lecturing and lecturing then typing away what you normally say in class, then you'd have a second round in complement with questions and focus on areas people thought were hard.

S: But the people you are teaching are people who are experienced programmers who are learning Erlang for the first time. And it was interesting for me, I have written books in the past for people beginning to program. So it's a different audience for me. So you can use to talk about people's experience and explain how Erlang is different than the languages that people are more familiar with.

F: That's were it worked really well writing the book with Simon as a co-author because we merged these two views and aspects which gave the result we have got today.

S: And I think that for me it's the first time. I have written a few books in the past on my own. And I think it has been very interesting writing with a co-author. Because you think "Ok, it's going to be half the work" and that's not the case. I think it's the same amount of work but I think what you get at the end is of better quality because there are two sets of eyes who have seen it, you argued about the order of material and so on.

F: And you get very different views as well: you get the academic view and you get the industrial view, which really provides in my opinion the best mix and the best results.

S: So the question is can you tell who wrote which chapter? I think that we hope you can't, we hope there's enough blend in that you can't immediately say "Oh Francesco obviously wrote that or Simon obviously wrote that".

Except for the Italian accent.

F: What was happening is I'd write a chapter and Simon would write a chapter and then we'd send them to each other and we'd go through them again. And then I'd add jokes and send them to Simon, and Simon would remove them and then resend it back. And so by the end, there was no single author to a a particular chapter, we just both worked with them together and contributed probably just as much.

S: And I think what is interesting as well in writing a book like this, you learn stuff. I mean Francesco is very experienced in Erlang but I suspect he learned a number of things. And I certainly learned. I knew about functional programming, I knew about the fundamentals of Erlang but I learnt a whole lot about how Erlang works with other languages, like Java and C, perhaps more that I have expected to, at the beginning of this enterprise. And also about Erlang and GUI programming, and so on. So you learn a lot doing something like this.

One of the sections of the book talks about design patterns, Erlang design patterns, which is more than simply using Erlang, right? Can you talk a bit about this section?

F: It deals more with working with concurrency in Erlang, and reusing patterns from one process to another. And we split it up into two areas because first we covered design patterns with pure Erlang and it should be seen an introduction to a middleware called OTP which has libraries which encompasses design patterns. And so by introducing these various patterns, it makes much easier and facilitates understanding of these libraries which are covered later on in the book. If you think of concurrency and Erlang the major design patterns you work with today, include client server, include supervisors, include event handlers and finite state machines which are the key ones we discuss.

S: I think it's interesting the concurrency model in Erlang is based on message passing between processes that don't share any memory. And that is a very nice model. But I think what makes Erlang really successful is not just the design of the language which has evolved and is very subtle and complicated thing, but it is this middleware layer that sits on top. So it is possible to build a client-server system, for instance, without having to worry about the patterns of communication. There is a generic part, a generic server, that comes with OTP, so you can simply take that application and you write yourself the particular bit of behavior. Suppose that it's something that will check if something is an anagram or if two words are an anagram of each other; you just fire the words off, and write the little bit of program that would tell you whether one word is an anagram of another. And you get built for you the whole client server infrastructure; it deals with failure and all these sorts of other things for free.

F: When we are building distributed massively concurrent systems with requirements and high availability, the programming language in itself is not alone. You need middleware, you need built-in libraries, you need a way to structure your code and structure your system, and that is what OTP brings to Erlang, the programming language itself.

S: So people who say they are using Erlang will be using Erlang and OTP. And I think all the practical large application you'll see will be Erlang plus this middleware layer.

Another section of your book talks about functional programming. What about that?

S: Well I guess that fits, it's a very nice basis on which to build a concurrent language. Functional languages think about the values how you calculate this result from these inputs, you are thinking at a higher declarative level, you think about relationships between values and not abut how to change one value into another through a series of assignments or whatever. So it's a nicely abstract model and that fits tremendously well with this message passing concurrency. Erlang is a concurrent language and you can easily learn the functional programming parts of it. It's not difficult functional programming but it enables you to think about what is difficult and that is the concurrent or distributed part.

F: I think I would like to quote Simon Peyton Jones here who said: "The concurrent languages of the future will be based on functional programming languages but we won't necessarily call them functional programming languages".

S: That is a nice insight, I like that. We do talk about the idioms that will be unfamiliar to people. So we talk about working in a language that doesn't have control structures, so instead of having loops that run forever which are inside a while construct, people learn about writing tail recursive programs so you have a function whose last statement in the body of the function is a call to that function again. Perhaps with some slightly different data which effectively represents a state. But you are handling that state in a purely functional way, without having to do assignments to process that state.

F: I think other interesting parts are programming in a side effect free environment where most of the functions are side effect free, and you try to minimize a number of side effects which in Erlang include IO, writing to databases, and message passing. And that is another hurdle for people trying to understand and learn Erlang and really embrace it and use it properly to get over.

S: But the message passing, if you are using OTPm if you are using the generic side of it, that is just handled in a functional way, you just say how this message should be processed, you don't have to worry about how it's received, how it's stored in the mailbox or whatever, that is abstracted away, you just use this generic model.

F: It's the pitfalls of concurrent programming: dead locks, race conditions, in a library so the programmer doesn't have to worry about it.

S: Cool stuff.

Interesting. Also you talk about type annotations inside the book. Can you tell us a bit about this section and type annotations?

F: This was a very recent addition, a new tool which originally was developed by Upsala University which came out just recently with R13 release.

S: What we are trying to do, Erlang in some senses is a bit like C, it is quite a small language it's a very liberal language, it doesn't have a strong type system like Haskell or ML, but what that means, it's a small language, it's relatively easy to write tools that work with it. So Erlang comes with a whole lot of tools: for tracing, debugging, tools for refactoring and so on. But there are particularly a very nice set of tools that allow you to do type checking effectively; you can state the type that you think a function has, you can type check that, using a tool like Typo, so it's this very liberal language but the tool environment is very powerful, very strong, and that really helps with Erlang programming.

Often you talked about software upgrade which is one very interesting feature of Erlang.

F: Software upgrade is something that you need to build into a language from the start it's not something you can add at a later date with some after effort and it was one of the very first features they looked at where how do you build non stop systems, you make sure you can upgrade the software during runtime. So using some very simple principles and constructs, you are able to change a pointer to the module where a specific process is executing maintaining the state, maintaining the values of the variables but just changing the actual code which is executing. And using these very simple principles you are able to do software upgrades which give you systems with five nines availabilities including upgrades, and that is just a few seconds per year of down time.

S: And just to stress, in other languages you would have to take the system down.

F: Or find complicated workarounds where you've got primary you've got stand-by, you switch off to primary and upgrade it and then switch everything back. You don't have to worry about this scenario like most of these cases are built in and handled with the capabilities and the tools that come with OTP. Not to say it's not simple, if you have backwards compatibility issues you need to upgrade the database schema and all backwards compatible protocols, it's still complex, but it's less complex than ordinary languages used for server side programming.

S: Just to make a general point about Erlang, one thing that is very nice about it, it's very focused, it was designed in the computer science research lab in Ericsson about twenty years ago and they wanted to find languages they could use to build things like telecom switches. And they did an evaluation of all the languages that were around and found that none of them really worked. So there were just three people doing this work so it means that it's not like a language designed by committee. There is a small number of features, each feature has to justify its existence, you can see where code swapping comes from, you can see where distribution comes from, where passing comes from. And the language has been remarkably stable it's not proliferated there is no bloat that you see perhaps in Fortran or languages like that where they changed beyond all recognition.

F: It's a language designed by prototyping, they were prototyping telephony applications and testing ideas and if an idea worked it made the code more compact, and easy to understand the capital feature. If it didn't help, they removed it. And prototyping which took several years before they started using it in commercial products.

S: And I think one thing that's intriguing it was designed with, OTP stands for "open telecom platform", so there could be a worry that Erlang is just a language for doing telecom applications. But in fact what you need to do telecom applications is precisely what you need to do for a whole lot of other things. So you see Erlang being used to handle systems where there is inherent massive concurrency, you are getting SMS messages coming at thousands per seconds.

F: Instant messages, web services, any server side.

S: And the point about that is that you then have a process per message, each message that comes in spawns a new process and you quite naturally get this huge amount of concurrency, which is very hard to do in something like Java. I think the other intriguing thing for me about functional programming in general. These ideas have been around for a long time and Haskell is similar in age to Erlang. And it's just now that these ideas are really taking off. It was the same with OO, if you think similar.

F: It was probably too daring when it first came out and it has taken time to adapt, understand and embrace it.

S: So Simula was designed in 1967 but it was only in the late eighties, so it was about twenty years leap time, for OO to come in and perhaps we are in the position where functional languages are really going to take off now. One of the things that you can, and again I am old enough to remember what people said about functional languages in the eighties, that these were going to be the general purpose languages for the next generation machines which were all going to be parallel. The eighties people thought the sequential processor is dead, now of course it didn't happen but now we have multi core processors as standard. Any modern laptop will have more than one core. And I think it's another reason why functional languages will be becoming more and more popular, because in something like Erlang we don't talk about multicore but the current version of Erlang is one way you can quite naturally scale things on to multiple cores, eight, sixteen cores it's really pretty straight forward. Their challenge is going up to hundreds of cores but certainly the standard distribution you get will support multicore naturally. And many applications are just naturally multicore.

F: I mean by taking existing code basis from a single core machine to a quad core machine even with recompiling gave us an increase two hundred and eighty percent, these are simple benchmark we have done. Ericsson has done even more improvements once you go in a more fine-tune, in the concurrency model and it's a perfect fit for multicore, the main reason behind it is the fact that there is no shared memory, and in process where there is no shared memory that's where your biggest lock contention lies, without shared memory you can just schedule under various cores and they keep on running their little life and keep on doing their work in parallel. So true parallelism.

So apart from performance, Erlang has the after … which is also useful not only for performance but also for modeling things, right? Is it something easy to capture for new Erlangers?

F: No it is not easy, I think it takes time, and takes a bit of experience and practice but hopefully I think we have addressed this in the book. And based on the experience from teaching it, to outsiders new to it we've hopefully taken the necessary steps to facilitate understanding.

S: We hear people saying this about OO, and I think it's more problematic with OO this idea that each object represents something in the world. I think that the idea that a process represents something like an SMS message is a very natural idea.

F: It's natural but it's very different from comparative languages or object oriented languages which means you need a slight mind shift to start thinking in a different way, when you start programming with Erlang and just realize processes are cheap, use them for every truly concurrent event in the system, and I usually tend to use instant messaging and instant messaging is a truly concurrent event in the system, is not a session when a user is logged on, it's a user receiving instant messages or a buddy update and it's for instant messages or a buddy update where you spawn the process and not for the session itself. I think that is a mental shift people need to be able to do, to really maximize the whole throughput of their system.

Erlang started as a language for doing some work in Erickson, and suddenly it becomes very popular recently, we see old Haskellers becoming Erlangers.

S: That is right, yes. I think you can be both, I think there is room; I think the languages do different things: Erlang is designed with concurrency and fault tolerance right from the heart, whereas Haskell is designed, well people were very concerned with Haskell building type safety for instance right from the start and to build in controller side effects much stronger controller side effects. But I think there are tradeoffs. I think that there are people looking at concurrency in Haskell and how to incorporate it but it's harder to see how you could incorporate fundamental Erlang style concurrency in Haskell. I mean it supports other models for the moment, and in a similar way people are thinking about how you can incorporate types into Erlang. And I think the way that is going to work is through tools, it's not going to be through redesigning a strongly typed language. But I think they both share a lot in common it's deciding you focus on values and how they are processed, you get away from this idea that you have to understand a program by understanding these series of transformations the data need to go through the history of values and a variable, a really peculiar way of doing a computation. Makes much more sense to think about how outputs and inputs are related together directly from my point of view at least.

F: You said recently, but I always disagreed to that, I think it has been all along a very tight connection between the computer science lab and Erlang and Haskell and cooperation, the first major project was Phil Wadler and Simon Marlow back in 1995 when they tried to do a type system for Erlang. And they covered the majority of the cases and I came to realize that the remaining cases were very, very, hard if not impossible to. And I think there has always been a very open communication channel with Haskell.

S: And QuickCheck. I am sure you have talked to John Hughes about QuickCheck. QuickCheck started in Haskell and then migrated over to Erlang and that has been interesting in itself so I think the functional languages community is not one I mean obviously there are groups of people around different languages but I think people are not tremendously, there are not massive wars between the different approaches.

F: I think each functional programming language has its niche, and it's the right tool for the right job.

Some languages like Scala and F# are trying to get inspired from several languages and to mix things in like Scala. They have the actor model of Erlang and they have a very strongly type language. What do you think of such solution, will they get mainstream, are they of a great use?

S: There is always a question about whether you want to make it a complete break with the past and I think that one of the reasons that I think C++ was so successful was because you could carry on doing C and add some object oriented, I mean after all C++ was a library on top of C initially, it wasn't a separate language. So it was easy for people to make that slow migration. And I think perhaps in the past people who have been fundamentalist about functional languages will say what you got to do is throw away this old fashioned way of thinking and embrace this new way, throw away all your libraries and all your code and that is clearly unrealistic so one of the things we talk about in the book is about how you can interwork between Java and Erlang for instance. And a very nice way of working between the two is a model in Java and Erlang processes effectively. So you can from within Java talk to Erlang or from within Erlang talk to Java. So if you want to build a system where you do concurrent stuff in Erlang and you do the GUI in Java that sort of interworking is possible. I think you can do multi language programming and that's becoming a norm.

F: I think we also covered the integration between Erlang and Ruby, where a lot in the Ruby community have embraced Erlang and take it on board, to handle their performance issues that they have been seeing on the server side. And when talking about Scala, well someone with an object oriented background who doesn't really want to take the full step will go down the Scala route. Someone who wanted to program in .Net and to have access to the libraries and use Visual Studio, they would go down the F# route. So, there is no one size fits all.

S: And that is clearly being in F# I think the Haskell people did look at how they might incorporate within .net, and there is some sort of bridge between the two, but it's not tremendously easy, whereas what the F# people have done is take the ideas from OCaml, they have modified them and decided we are really going to go utterly to try and build a functional language within the .net paradigm. And that's great and particularly the tooling. I am not sure if you are talking to the F# people but it's interesting talking to people who have applied it in the financing industry; they would say the crucial thing they got people using is the fact that it was in Visual Studio and you saw those little read lines. And you could get type information and so on, it convinced them it was a first class language. People are doing similar things with Erlang you could now incorporate it within Eclipse so if you are coming from the Java development world and you are used to using Eclipse, then you can do similar sorts of project development in Erlang inside its plug-in. But other people are happy using Emacs.

F: There are various options which are out there. Different people are comfortable with different editors, different languages, different technologies.

What is interesting also about Erlang is the virtual machine which does a lot of abstraction. Do you think that we could implement something else than Erlang on this virtual machine?

F: The VM has a team at Ericsson that is managing and implementing and I think that's where a good part of the effort is going today. The VM we've got a highly scalable concurrency model, it's thanks to the VM scalability and multicore, is almost linear. Just to give you an example between R12 and R13 all you needed to do is recompile your Erlang code to get a thirty percent increase in speed, and thirty percent reduction of memory usage. This is just a general benchmark and just to show the effort which is being put in. On the VM they built there is a something called core Erlang on which they have built several other languages one of them is Lisp flavored Erlang, one of the mentors of Erlang has a little pet project allowing people to use the Lisp syntax, but embrace the power of Erlang. Another is RAIA which does a similar thing, but much more influenced by Ruby. So it is happening and it is going on and also I think a lot of people are looking at Erlang VM and looking at how the garbage collection works and how the process management works, how scheduling works, trying to get ideas and implement them in their own programming languages, in their own VMs. We are actually seeing the influence of the language only penetrating.

S: And I suppose it's in places like that having concurrency at the heart of Erlang right from the beginning makes a difference because for example there is garbage collection on a per process basis so it's a very fine grain and it's fine grained in the way that it goes with the grain of writing concurrent programs to try and retro fit that to another language. I think that in Haskell people would really like to have that sort of garbage collection. But it's very difficult if you are incorporating it into a quite different sort of machine architecture.

F: We were branded as insane back in the 90s when we were telling people that we were implementing soft real-time systems or a language running on a VM, it was not possible. I mean real time Java, which came around a few years later, the garbage collector would kick in and stop the whole VM for up to a minute. I mean these were the hardest I've hear, for up to a minute when trying to garbage collect. Whereas in Erlang they put concurrency from the beginning and as Thomas was saying we've a per process garbage collection, you reduced the impacts on it and you were able to produce very efficient soft real-time systems, where you had impact on the performance but it was minimal compared to other languages but from there you gained all the advantages such as software upgrades and so on.