Rachel Reese on The Good and Bad of Microservices (with F#)

1. [...]Would you mind briefly introducing yourself to our audience?

Manuel's full question: Hello. I’m Manuel Pais and I'm here at QCon London 2016 which Rachel Reese, senior software engineer at Jet.com and functional programming geek, if I might say so. Thanks for accepting our invitation. Would you mind briefly introducing yourself to our audience?

Sure. Thank you for having me. This is awesome. So like you said I am definitely a functional programming geek. I have been working with and learning F# since 2010, probably more. I doubled down in 2012, 2013 and I have decided I just really love the language and wanted to spread the word. Go out and show everyone how amazing it is.

2. [...]Do you want to tell us a bit why you decided like that?

Manuel's full question: Your talk here at QCon is about microservices at Jet.com and you told me you have a particular case. You started directly with microservices and kind of the general wisdom says it's better to start with a monolith to understand your domain first and then figure out how to break it into the different services you need. Do you want to tell us a bit why you decided like that?

Jet was actually somewhat of an accident to come into microservices. They chose F# and sort of doubled down on that and were building a lot of the technology behind Jet in F#. And F# being a functional language one of the main sort of features and ideas is that you should consider malfunctions. You know, what data goes into your function? What data comes out of it? What are the inputs? What are the outputs? And stuff like that. Microservices end up mapping just very, very well to F# scripts.

Another thing we considered, or sort of did along the way, we didn't name things “have a skew service”. We just said this is a script that imports skews so we named it “Import Skew” and a lot of these things were just sort of more abstracted functions. As we went further and further along in our technology, we just sort of woke up one day and realized we had microservices, that we had been building them all along, just naturally with F#.

3. That's great. Did you sometimes realize you had to rethink how you had the F# functions organized and did you ever have to split them up? If you did, how costly was it to make those changes?

In general, the way we group things is mostly by bounded context. So we'll have a group of say five or six. Within that, it's somewhat common to maybe move a microservice to another section.

It's also fairly common to split one microservice into two or three as we realized that this actually does a whole bunch of things. One of the things I talk about – actually not in my QCon talk but another talk – is if there are side effects. If you are sending an email for example, then you definitely want to split that into two: one section is writing to the database, another section is sending the email. You really want to have that separated into two. And so at that point you would do that. It's almost trivial.

The coding part of microservices becomes very, very easy. It's a matter of refactoring a few functions. We move this part over here and then add a fair amount of testing.

4. In that case, you think F# helps, as in the language itself helps to break down the services?

Definitely the language itself but I would say even just the mere fact of having microservices ends up being very easy to split them up.

5. [...]Did that ever happen or no?

Manuel's full question: Kind of the reverse question, did you ever realize like “ok, this microservice and that microservice, they actually end up changing at the same time because in fact they have the same life cycle. We better combine them into one”. Did that ever happen or no?

Almost never. I couldn't think of an example and I hit up several of the other engineers and we sat there and were like – no, we definitely have split the microservices up and had two or three come out of one but I would say never [the reverse].

6. A more generic question: in which ways have microservices made your life easier? And in which ways have they made it harder (comparing to past experiences without microservices)?

Definitely one way they have made it easier is being able to refactor the code and being able to release things. I mentioned the refactoring but just having this one small tiny piece of code, it makes it very easy to release. It makes it very easy to scale everything. It's a lot easier to work with the code and the microservice itself.

The much more difficult thing that I think many folks don't think about when going in is how much more difficult the infrastructure is. We ended up rolling our own but you do need some sort of infrastructure to handle availability, to handle all of these things that come into effect when you start talking about microservices. It's a much bigger deal.

7. Your talk here at QCon is focused on microservices chaos testing. Is that in the sense of Netflix's chaos monkey or is it something different?

It's very much Netflix chaos monkey. We are taking that same idea at Jet and we have implemented a proper chaos program. Again, we wrote our own for a few reasons. We're using Azure – Jet, as an e-commerce company, is very much in competition with Amazon, so we are not using AWS. But we also wanted to write something in F# that would take advantage of a lot of the things that we had already done and be able to work in our own infrastructure.

8. [...]How do you cope with that at Jet.com?

Manuel's full question: To benefit from microservices – as you mentioned – you need to be able to deliver them independently. That gives a lot of flexibility and scale and so on but there are also some challenges. You mentioned infrastructure but for example there are also issues in terms of dependency management between services and testing before deploying. How do you cope with that at Jet.com?

For dependency, I mentioned that we organize services into small groups and generally have a bounded context. We'll often have one sort of shared library for one of those groups so any shared code is handled there. That's one way to handle some of these things.

For testing, two things actually were interesting about testing. We have found that in general because of all of the features F# has, it's somewhat less important to test. I know this is very unpopular. But we do have the integration testing. We have unit testing. I think at least a couple of the teams are using FsCheck which is random testing. It's fascinating.

And we still have most of the same testing infrastructure I suppose. Not that it's not as thorough but there is so much less code with F#. There is a really great blog post that a guy named Simon Cousins wrote a while ago on a very standard business application that he had taken in C# and a few of the devs on his team and him had rewritten into F#. And the entire lines of code for F# was less than the number of blank lines in C#.

Manuel: That's a good one.

Yes.

9. [...]But it sounds like, from what you are saying – that with F# you have a smaller code base – that you probably have a tighter team and don't really need those contracts because you already know what's being tested by the other services?

Manuel's full question: Some teams or some organizations use contracts between the microservices, microservice A defines what its dependence from microservice B is and gives that team the tests to run. But it sounds like, from what you are saying – that with F# you have a smaller code base – that you probably have a tighter team and don't really need those contracts because you already know what's being tested by the other services?

Yes and no. We didn't have them to start with and it started to become an issue. I know that for a while I had been saying we have 350, maybe 400 microservices. Again I asked our engineers yesterday, I was like “so I'm sure it's way more than that by now but are we like 450 maybe”? And a couple of the teams started listing and we have 12, 13 teams and they are like “We have 80. We have 90.” So we are in the 800 to 1200 range maybe. Nobody has any idea. And at that level, yes, we needed to have some sort of contracts in place to make sure everyone was really clear on what's happening.

10. I'm also curious if you've used different technologies stacks, other than F#? Did you in some cases think “oh, for this service we better use a more imperative language” or whatever or a different database?

Across languages less so but technologies yes, it's very much a right tool for the right job. I would still say that's true although we have found that F# has been the right tool for most of what we do. But we have a few programmers that write Node. Especially in our front end, there is quite a bit of Node. But you know, we're sort of all over the place in back end. We have Kafka, we have Azure queues, we're heavy users of much of what Azure does.

11. In terms of shared data persistence, do you have any tips how to deal with that?

Actually a couple of different things. I mentioned that we use a lot of different technologies. So know what your actual need is for that service or for that piece of the puzzle. If it's log storage, if it's a key value storage, if you are caching something. Know what you actually need from that and know what sort of recoverability and consistency requirements you have around that and then from there just choose what fits best.

Another thing that has been important is one of our teams has completely outlawed two-phase commits.

That was really interesting. Their take was that the transactional logic adds a lot of complexity and many times that's accidental and that ends up being bad. But I have worked at several companies where everything you do has to be two-phase commits and there is like that other side of the story had never even occurred to me.

12. And the teams themselves have the autonomy to make those decisions in terms of their stack?

Very much, yes. If you want to incorporate some technology, then as long as you have time, you go ahead and try that out.

13. Last question, if you want to tell us a bit about your current interests in functional programming or – I know you are also organizer of meetups around this theme.

Yes. I love F# and obviously one of the reasons I work for Jet is because I loved F# – “Yes, there is an F# job, this is amazing!”.

I organized a conference in New York last year, F# Gotham. I try to organize as many meetups as I can. Any time somebody is coming through New York City, we try to have them speak at least at Jet or one of the other user groups in town. And generally conferences. Yes, I love F#.

Manuel: Great. Thank you very much.

Thank you tons.