Phil Trelford on Functional Architectures, F#

Recorded at:

1. We’re here at GOTOCon Copenhagen 2012; and I’m with Phil Trelford; so Phil, who are you?

Hi, I’m a U.K.-based software developer. I’ve been developing since I was 13, that’s nearly 30 years ago which, I guess, makes me about 27; and I started off in video games and I’ve kind of criss-crossed between line of business applications and video games but I’ve got obviously a lifetime of programming and real passion for it even now.

2. I see you’re doing a talk here at GOTOCon on functional architecture, what is that? I thought all architectures were object oriented?

Well, they’re not; I wanted to highlight the idea that your choice of language and platform does affect your architecture in the way you think; how picking a functional language or architecture can really benefit the system.

If I just take a simple example say we write a retail application, if you think just about objects and mutation, then you would think when you’re constructing a basket of items, as you add an item you'd increase the quantity and if somebody returns an item or cancels an item, you'd decrease the quantity, you’d be doing it wrong.

With retail system, you need to track what’s actually been done so you can detect fraud, it’s one of the key elements of the system; so you actually add new values each this time, you never mutate.

So just from one of the most basic systems that we interact with everyday, functional immutable style is actually close to the architecture you want.

3. It’s basically what bookkeepers have been telling us for centuries: "never change your books, always append. "

Exactly, and as Rich Hickey did in his keynote earlier, you can think of functional system perhaps more like source control, a functional database where you keep versions, you don't throw the old information away, this is how we construct functional programs; this is kind of sensible, mutating and throwing information away, now that space constraints aren't there, it doesn’t really fit; so and if you want to do any evaluation for fraud or just for seeing what’s happening in the system, it’s not a good idea.

4. That’s sort of a basic idea of building functional programs or functional architectures on a large scale, can you say that?

I guess you could; I guess what I’m trying to say is if you start off with the defaults in functional programming, immutability and transformation, and having that as part of your toolbox on top of thinking about objects is a very powerful concept.

5. Have you looked into CQRS?

Yes, CQRS, there's an implementation for CQRS in something like 50 lines of F#, so very it’s not perhaps entirely functional but it’s very close to functional concepts.

6. Basically append-only...

Yes you're appending messages, then you can replay those messages; this is a great concept. If I can just slip into an analogy, so I was working on a system for House of Fraser about ten years ago and they're a large retailer in the U.K. and all of the actions on the till are written out and XML was very trendy in those days, JSON not so much but it was 2001-2002; so we'd send those out as messages to the central server and that could be logged out.

The first day, everything went wrong; one of the stored procedures was incorrect but it wasn’t a problem. We changed the stored procedure, rolled back the day’s trade, ran the trade again because it was all messages and we managed to get into a consistent state.

So you can see the beauty of a message based approach there. And we were doing that within supposedly object oriented language C# but applying that functional concepts.

7. ...and losing the destructive component of imperative programming. Which I guess is one of the key points that Rich Hickey makes, never destroy just add.

Or at least consider that; thinking of that as your default is probably the right place to be and then when you really need to take that performance, you need to do the mutation because you’re in that bottom part of that tree, that inner loop, then by all means, go ahead, have an array and mutate it and actually one of the more useful data structures is a dictionary, a mutable dictionary is gonna be way faster than most persistent data structures in functional programming.

8. That’s an interesting point; there's always the obvious objection to these append-only proposals is performance or even today, I guess, space problems because at some point, you will run out of space, even Gigabytes will fill up; so are there any approaches to fixing this or things like snapshotting. Do they fit into that?

I think that’s kind of.. it can be a sort of one dimensional view ; so in a single threaded environment, clearly the mutation in place may have some better performance characteristics but let’s take the example where we have multiple consumers of a data source; if we need to lock and copy, then we've blocked all of those other consumers; now with a Reference based new instance which may be referring back to the old instance, those consumers are no longer blocked; and so in fact for a bigger world, large world problems, your performance is way better and again, it’s a better way of thinking about it.

You’re thinking about the system rather than the low level programming details, so that’s kind of important.

9. So it actually removes contention in many ways.

Yeah, it can do, but if we go for another example, let’s say we have a word processor which is actually a single consumer; when we have a small text file, Notepad's working fine and so are other text editors; but when we have a huge Word Document and say, we’ve written a book, we’ve got War and Peace there and we want to add a line in the front, if we use a sort of classical mutable array data structure, all we have to do to insert that text is we have to copy millions of bytes down and then insert that value.

If we do it in the middle, again, we have to push everything out; so if we think of that from a functional data structure point of view, we could say, we’ll create a Reference to the original document which is one pointer, four bytes on a 32 bit system, and we’ll say that in addition to that, we have an appendage of these four bytes or four characters starting at point zero then anybody reading that document goes to this data structure at which point we’ll unroll that.

And so we haven’t had to move the bytes about but that kind of data structure is called a rope; but it’s thinking differently about the problem ; so again, scalability even in a single threaded environment comes from having that immutability by default can help.

10. So I guess a good way of thinking is, as you just said, default to immutability and just break out if you really know, if you've profiled and you know it’s a problem.

Yeah, I think that default to immutability and default to readability, these are good choices, a functional approach is typically a more declarative approach which should be more readable if you’re doing it right; yeah, the default is good; and defaulting to some sort of micro optimization trying to unroll loops and things right from the beginning, premature optimization, micro optimization, again, you’re doing it wrong probably.

11. So what’s your weapon of choice in functional programming ?

My language of choice is F# at the moment; it’s a very pragmatic language, it’s multi-paradigm; so you’ve got, actually in some ways better OO support than, say, C# but at the same time you’ve got all of those great functional defaults, not having null by default in F# is just huge; you feel the pain when you go back; having tuples, first class functions is hugely rich. The thing I find practical and pragmatic about F# is it’s an easy language to learn and use and very powerful and we can leverage well of that pre-existing code in .NET and that interoperability is hugely valuable.

I also have an interest in Erlang, but again, you would not essentially write a UI in Erlang, you're in a specific set of problems there.

Beyond that, I consider scripting language like JavaScript and Lua because they have first class functions to be kind of the set of functional programming languages, but certainly not OO.

12. Is F#, can you say it’s object oriented; is it similar to Scala in the way Scala combines OOP and functional programming?

There’s a lot of similarities between Scala and F#; I think Scala looks more like Java or C# because it keeps the curly brace syntax but in both languages you can construct objects that can be consumed in Java or C# easily but fundamentally, these F# and Scala are multi-paradigm languages; so you can do functional, OO, imperative, procedural, whatever kind of coding that suits the problem; and I think that seems to be the way languages are going, C# is adopting a lot of functional concepts, lambda functions and anonymous functions, the LINQ statements.

In the next version, you’re getting the Async keyword which is, like what we’ve been using in F# since 2007 with Async workflows, so, a lot of the languages are becoming much more multi-paradigms so you can pick the right tool for the job; but I think the fundamental difference you can write anything in any language but what are the defaults; do the defaults suit you; with F# the defaults are immutability, the default is not have null; in C#, the default is mutability, the defaults are null.

So you’re building in some pain by default, the defaults are also very verbose but the default is to something that is the familiar, curly brace or mustache based syntax.

13. So that brings us to our old friends, the monads, as you brought up LINQ and async workflows; how do you take your monads in F#? Do you have monads in F#?

We try not to call them monads because it’s a quite intimidating, hard to grasp term; monad is a monoid in the class of endofunctors or some other silly statement; we have the async workflow, computation workflow in F# that’s heavily used. You can create your own and it’s very easy to create your own; reality is, most people don’t because if it is something that’s easily reusable, it will have been written already; so the cases for you doing it are much lower; it’s something that’s much more heavily used in pure functional programming languages like Haskell where, perhaps it’s a little harder to do I/O operations.

14. Is there language support for monads or I think computational expressions?

Yes, that’s the F# flavor; it’s built-in language support but it’s not something that you would default to doing. You can do all sorts of things but it’s not the first point of call.

15. It's not the first thing you do?

Whereas in Haskell your first default would be to create five monads and some type classes that’s just not what you'd do in F#, they're different languages.

16. As you mentioned, they're less required, you don’t need monads as much in F# because you have ways out of the purity in a way, can you say that? Or is that wrong?

I think Haskell is where most people use monads and it’s just a completely different language; it has shared ancestry in terms of syntax; but I think that’s probably where it should stop.

17. So where do you see F# take hold? You're from Cambridge so you’re slightly biased; but does it take hold in other places? Like the City [of London], for instance?

So the information I have, I have co-organized the London F# user group which started about two years ago; it’s hosted by SkillsMatter in East London. We now have over 400 members; to put this in context, the .NET user group in London has approximately 450 members; so it's a relatively large size and that’s and older user group.

I believe the Scala user group is over 600, so Scala is also very popular; if you put those together functional programming is more popular than .NET, but those figures are just for the user groups and those are self selecting people who are clearly interested in their profession enough to take their spare time to go and listen to talks and maybe give talks.

From the people we meet at the F# meetups, there's quite heavy use in financial institutions, trading institutions, so that would be people like investment banks, large utilities, hedge funds, proprietary shops. Besides the financial and trading institutions, we see insurance companies, pension funds, actuaries, and also in sports betting and social media, it’s quite obvious. I think anywhere where you’ve got computation or concurrency; so we are doing some processing, if you’re just filling in forms or doing a CRUD application, you could probably use any tool and it’s gonna work out for you.

18. So to wrap up, talking about tools, I think you have some open source projects? Let’s talk about some of them.

Actually the F# community is pretty rich on the open source area; as you’d expect from people who are doing professional software; there is a lot in the testing space; there’s a good number of fluent DSLs for unit testing; there’s fsunit, very pretty, 42 lines of code - the kind of libraries we have in the F# community are files you can copy into a project; hopefully you’ll find that a refreshing change.

NaturalSpec is a fluent API again, it also has mocking built in and that’s kind of interesting, tying the two together comes up with beautiful sentences. F# allows whitespace so your code looks more like English.

I have a behavior-driven development library; it can also be used as a framework called TickSpec, which is quite popular in this space and actually will parse over a text file and let you hook English or acceptance tests up against your code and you can actually breakpoint the textfiles in VisualStudio.

There’s a project called FSharpX which aims to bring a cacophony of useful utilities in one drop for both C# and F# users and that contains one of the small libraries called MiniRX which is a version of RX, reactive extensions written so it’s small but it does the 80-20 rule, that’s 80% of what we need; so I’m just giving you a flavor of what’s in it.