Bio Square's CTO Bob Lee is a technical leader, engineering smartphone apps, DSP algorithms, embedded firmware, payments processing infrastructure, and more. Before Square, Bob led the development of Android's core libraries at Google. He created the Guice framework, co-authored the book "Bitter EJB.", leads the spec of JSR-330 "DI for Java," and he's a member of the JSR-335 and JSR-166 EGs.
Software is changing the world; QCon aims to empower software development by facilitating the spread of knowledge and innovation in the enterprise software development community; to achieve this, QCon is organized as a practitioner-driven conference designed for people influencing innovation in their teams: team leads, architects, project managers, engineering directors.
Hey, how is it going Michael, nice to meet you!
Michael's full question: Pleasure to meet you as well. So we’re here at QCon San Francisco 2012 and Bob is hosting a track, an entire track here, entitled Java Renaissance; so before we get into that, Bob can you just introduce yourself to the audience, tell us who you are?
I sure can. First of all, I guess nowadays I’m the CTO of Square we’re a company that enables you to accept card payments on iPhones and Android phones and we’ve also have another product called Square Wallet that enables you to make payments, so we basically support both sides of the counter. Prior to that, I was at Google for about 5 and a half years where I created Google Guice, as you said, and I was also the core library lead for Android.
Well, a lot of people, I guess, kind of left the Java community, they are very excited about newer technologies, they wanted to try something new, I guess the shiny thing which was Rails first and Node.js more recently but in the last couple of years, a lot of companies have kind of hit the limitations of these other technologies and realized how valuable Java still is and how far ahead Java and specifically the VM is, still ahead of all these other platforms and there is another number of companies like Facebook and Twitter and Square that have all readopted Java. In the case of Square, for example, we started out as a Rails shop on the server side, all our services were implemented in Rails, and we ran into a couple of challenges: one was scalability; that is you can only handle about 20-25 concurrent requests per machine, whereas with the equivalent Java service you could handle hundreds.
We tried JRuby early on but the reality is most of the libraries written for Ruby aren’t written with concurrency in mind, so they aren’t really thread-safe, so you have to run JRuby in thread-safe mode which defeats a lot of the scalability advantages you get of using Java in the first place. Also from a maintenance perspective, for code that is going to be around for a long time, like our payments stack, for example, we found that Java code tends to be more maintainable, static typing, for example, and in particular makes it easier to navigate your code and understand your code and to change it, whereas with a dynamically typed language, like Ruby, can be kind of challenging to understand and maintain a larger code base, you can’t just tell what’s going on from reading the code. That said, we do still use Ruby pretty extensively, within Square, just not for our mission critical services, like payment processing, but we do use it for front-end websites, for example, and lots of our internal services.
Performance issues actually aren’t the biggest challenge, though. I mean they certainly do tend to be slower than Java, though JRuby is pretty fast. The reason you can’t have so many concurrent requests on a Ruby server or use MRI Ruby or a JRuby in thread-safe mode tends to be more RAM based. You are going out of memory because each process, each worker is running in its own process. Really what the track was about today was a couple of years ago with Square, we started making this migration and readopting Java and I had been at Google for 5 and a half years before this and Google tends to have some proprietary technologies so I had to relearn what a modern Java stack looked like and which open source technologies to use, and I found that really challenging. So partly what I tried to do with this track was to sum all that up and save other people the time it took me and expose them to the latest and greatest technologies, for example Dropwizard which is an integration framework that integrates Jersey and Hibernate and makes it easy to start up servers and have an SOA architecture. It really saves you a lot of this integration effort which turns people off of Java.
For example if you just try to do it yourself you'll go and get Hibernate and Jersey and a DI framework, it's really challenging to figure out how to integrate all of these whereas with a framework like Dropwizard it's easy. In addition to that, we also had Kevin Bourrillion to speak about Guava, which is Google's core library, a project I contributed to. People like to think of it as the missing libraries for Java. We had Jesse Wilson talking about Dagger, a successor to Guice that we're working on. It's not quite ready to replace Guice, but it is work in progress but that is definitely the intention. Then we had Eric Burke talking about Android, which is one of the most exciting things that is going on for Java nowadays, I don’t have exact stats, you know, but my guess is Android is attracting more new developers to the Java programming language than anything else at this point. And finally we had Dan Smith from Oracle, he is one of my fellow experts on the JSR 335 Lambda Expert Group where we are designing what you might think of as closures for Java.
5. [...]So this is basically an enhancement of Guice and you were telling me off camera that it’s a kind of a re-write based on things that you’ve learnt. Can you kind of explain what the motivation behind that was?
Michael's full question: That was a couple of my questions there; I was going to ask you a little bit about Dagger. So this is basically an enhancement of Guice and you were telling me off camera that it’s a kind of a re-write based on things that you’ve learnt. Can you kind of explain what the motivation behind that was?
Yes, definitely. So I created Guice, it was about 6 years ago, we’ve been through three versions. Jessie Wilson who is also at Square now, created or launched and led the second version of Guice, so we’ve been working on this staff for quite a while and a number of things led up to Dagger. One was, we wanted to take a step back and ask ourselves, knowing what we’ve learnt over the last six years, how we would do it now. And the reality is we’ve come up with a lot of simplifications, for example provider methods in Guice came in version two, if they had existed in version one, Guice would have been a lot simpler. For example in Dagger, we have provider methods only, we don’t have this binder API that is used for the configuration, and one of the things that that enables is if you only have provider methods that means that everything is statically checkable, I mean you don’t have to execute any Java code to figure out your configuration, which means that, because it’s simpler it’s all statically checkable. I mean that it’s going to be a lot more easier to tool and it should be very easy to have IDE support which has been one of the, I guess, major complaints about Guice.
It’s theoretically possible to have IDE support in Guice but the reality was it’s a little too non-orthodox the way you have to do it, that is like running your Guice modules and actually running Java code to produce the configuration. The second thing that drove it is, we were using Guice in our Android applications and Guice does a lot of upfront type checking, and it was while it was for contexts like this, reality is running it on an Android device it has to do a lot of reflections, which means I have to allocate a lot of method objects, a lot of field objects, a lot of constructor objects, and that just tends to take a long time and adds up to a few seconds to your app startup time and that is just really not acceptable. One of the innovative things that Dagger does is instead of using reflection at runtime, it uses code generation instead, so it actually generates the code that wires all your code together, which makes it a little easier to start up because loading classes tends to be faster and doesn’t require so much work in the Java space. So far it’s been going really well, I guess it’s a work-in-progress but so far as functionality goes let’s say it’s like 70-80 percent to the way there and we’re already using it in our Android apps.
It is open source, it’s out on Square’s GitHub account, so it’s at http://gitgub.com/Square/Dagger so you can track the progress even we haven’t even released 1.0 yet but we are using that actively. And actually Google has starting working at it also, a couple engineers from Google, so they are actively contributing to it too.
Michael's full question: It’s interesting because I’ve written an article, a while back on Guice 3.0, I think it was, and I got a lot of comments about dependency injection introduces a lot of complexity without a lot of benefit, and we have Spring and there are other ways, what would you say to those critics, that say dependency injection is overrated?
I think they fall into two buckets: the ones that are advocating Spring over Guice. If we are talking about the traditional Spring way that is using XML to explicitly wire everything, that requires an order of complexity more configuration than the Guice approach does. With Guice you just specify a binding one time, like anytime I see an executer I want this executer service to be injected, and you don’t have to do anymore configuration beyond that, everywhere else it just gets injected automatically based on the type as opposed to like the traditional Spring way where you had to have explicit XML for all those injections. But again, since then, Spring has adopted these annotations, I actually led JSR 330 which is dependency injection for Java and it basically specified the annotations and left the configuration to frameworks like Guice and Spring, so Spring actually supports this method of injection now.
There is also a lot of people out there who prefer plain Java over dependency injection, but really I think most of the time those people tend to just look at a simple example, that is like: “I’ve configured one thing to be injected and it’s only injected in one place”, it’s not a lot of win in that case. And then I would say that I probably wouldn’t use dependency injection in that case because it’s more abstract and makes it harder to understand the code. Where dependency injection is really a win is where you are using that dependency in lot of places, ten, dozens, hundreds of places, that is where it really starts to win and saves you from having to write and handwrite all these factory classes. I think one of the signs of a really good programmer is that they recognize and fix problems that other people don’t even see and I really think like factory code is one of those problems, people are just so used to writing factories in Java that they don’t realize how much boiler plate they're writing, they don’t actually have to be writing. And dependency injection definitely addresses that.
It’s nearing the end stages, it’s going to be in Java 8, which I think is coming out next year, everything seems to be on track, a lot of the effort right now is on the language changes, and the library changes that go along with it. I’m really excited about where the project ended up, I mean there were some different ways that we could approach this and there was certainly a lot of contention in the early days and Java is got a lot of flak for having Closures or what we more specifically call Lambdas later than the other languages, but the reality is we’ve been able to learn from the lessons of all these other languages like Scala and Ruby and apply those lessons and make it arrive at a really simple solution that doesn’t really impact Java that much.
For example we’ve got most of the benefit without having to add a whole another type system, it’s like a lot of languages have a completely separate functional type hierarchy that is separate from the object type hierarchy, we haven’t really had to do that in Java, we’ve come up with some clever ways to fit it into the object type hierarchy. So it’s going great, I’m really excited about it and it’s not just Lambdas either, those will save a lot of boiler plate. There are also some other language features that are coming along with it, for example extension methods. The main idea of the extension methods is that you can specify default implementations of methods and interfaces and this is huge, because it means we are going to be able to essentially add methods to interfaces which is really necessarily in this case because we want to retrofit Map and List and Set, for example in the collections framework to be able to take Lambdas and use Lambdas and we really need this to make Lambdas like a first class part of the platform. But also I’m extremely excited about, I’m almost more excited about Extension Methods than I am about Lambdas just based on the amount of boiler plate that is going to enable me to get rid of.
I guess one of my beefs about Java has always been even though is the right thing typically to use composition over inheritance, you have to write a lot of boiler plate code when you’re using composition and writing forwarding methods and that sort of stuff and having default methods or extension methods, it’s going to make that stuff a lot easier and make it a lot easier to compose your objects out of lots of distinct types.
Michael: So it’s analogous to Traits and Scala?
It’s very similar to that.
Extension methods aren’t silver bullets by any means, I mean we are limited by the fact that we have to add this to interfaces and to Java which is a 17 years old platform that was not really designed with this in mind and they are not quite as powerful as Traits, like for example you can’t have state in your interfaces and extension methods and that sort of stuff, but we will still able to do quite a bit of stuff. It’s like I think these examples of, you know we always have this predicate interface and you have to have a way to compose predicates. It’s like basically Boolean functions, and it’s like you typically have an And and Or method to do that, you’d typically had to use like an abstract class for your predicates, but with this we can just put And and Or methods just right on the interface and it would enable you to make any class of predicate, which is really nice rather than having to have only one super type.
10. [...]Java has its roots in OOP, so I would consider it an object-oriented language, as we start adding these kind of functional it’s becoming more multi-paradigm and is that being motivated by languages like Scala and Erlang and Haskell?
Michael's full question: This is more than a general question, there is a lot of multi-paradigm languages out there, Java has its roots in OOP, so I would consider it an object-oriented language, as we start adding these kind of functional it’s becoming more multi-paradigm and is that being motivated by languages like Scala and Erlang and Haskell?
Whether we are talking about Java or C, I’m definitely a fan of programming in a functional way when it makes sense; that is you don’t have a lot of shared state is the main thing, you don’t have a lot of variables that change, you don’t have immutable types and that sort of stuff. Java makes it very easy and in the Guava library has a lot of support for that and Lambdas are going to make it even more of a first class concept, but I’m not dogmatic about it, I like to use it when it makes sense, when it’s easy but a lot of time you have to jump through a lot of hoops to write things in that sort of ways from what I found. When it comes like domain models where you do some work with mutable state or especially in UI programming where you are linking a lot of UI objects together and everything I found is hard to do, like programming in there.
A more simple example just think like a FIFO queue and how you implement that in a functional language with persistent data structures versus if you are going to do shared state, it’s like you are trying to do it in a functional language, you’d have to do something crazy, like it’s a finger tree or something like that, whereas in a more straightforward language or when you're using shared memory you just do a circular buffer and it’s extremely efficient. So I definitely think there is places, and that’s just one example, where functional programming works and where it doesn’t, but I’m a fan and definitely recognize its maintenance advantages when you can use it.
Yes definitely, it’s a huge big part of my responsibility. I’m a big believer in intrinsic motivation. If you really want to excel in something, you really want to do your best work you have to be intrinsically motivated to do it, and there is an author named Daniel Pink who wrote the book “Drive” which is a New York Times bestseller and he said that: “Three things contribute to intrinsic motivation: it is autonomy, mastery and purpose”, so I really try to optimize for those things at Square with our engineering team, autonomy in particular. I think it’s very important for very talented engineers like we hire to have control over their designs and control over the decisions that are made in their implementations and rather than tell them how to design things or which technologies to use, I see it more as my role to make sure we have the right communication mechanisms in place, so they are communicating and making the best possible decisions, it definitely scales a lot better.
The second thing is mastery, I think we’ve been doing a great job of keeping the talent bar extremely high at Square, I think it’s one of our key advantages. I guess the level of talent in our engineering team is definitely evident in our products and level talent both engineering and design and one of the key ways that we’ve achieved that is that our interview process is very centered around programming at an actual computer. At Google prior to this, I conducted hundreds of interviews and then I’ve come to Square and conducted hundreds of interviews here. One of first things that I really recognized when I came to Square very early on was that keeping the talent bar high is critical: one because like I said it’s evident in your products, and your product will only be as good as your team; but two also it’s like really top engineers and top talent want to work with other top engineers, this really speaks to mastery. They don’t want to just take the skills that they already have and just keep applying them like a factory worker or something like that; we really want to keep improving ourselves and keep learning so I do really create that environment where we have those top engineers and people who you can collaborate with and learn with and learn from.
So we’ve definitely done a great job, I really think like hands-on programming is the best filter there. I’m not a fan of programming at a whiteboard for couple of reasons: one, I get really nervous during interviews even mock interviews, not this one so much, actual programming interviews. I really tried to design our process to minimize that and really just try to suss out like how somebody it’s actually going to work on the job and I found that like programming at an actual computer in the language of your choice using your familiar tools, really enables you to do that and it minimizes those false positives and negatives you get from programming at a whiteboard.
So as opposed to getting grilled at a whiteboard and you are kind of on your own and you don’t have a test to back you up, I really like it to make that so they were working together on a problem, I think it turns into a much better experience and you can go a lot deeper on a problem and learn a lot more about a person and their technical abilities and the culture of it.
Finally the third aspect of intrinsic motivation is purpose and I really think Square has that in spades more than any other company right now just a level of opportunity and I guess our ability to impact all our customers’ lives, both for merchants and for sellers and buyers. On the seller side of the equation prior to Square, I don’t think that they were trying to accept credit cards payments before but before Square you have to get something called a Merchant Account. It took like 6 weeks and you had to get a credit check and literally half of sellers got turned down, because they had no credit history or bad credit history or maybe they didn’t have 6 months operating expenses in their bank accounts.
What this they really came down is that the banks and the traditional merchant processors weren’t able to assess and manage the risk of these smaller sellers in a scalable way, and that was really one of the key innovations of Square. It was that we were able to do that, in fact like you can download our app and sign up and be accepting, you answer a few security questions and can start accepting credit card payments in just a couple of minutes. And that is really powerful to me, today especially the way that economy is right now, literally we have over three million merchants now, enabling all these people around the U.S.to start their own businesses, a lot of people who couldn’t even find jobs before. So we have that aspect where we are, I guess, empowering sellers and then also we are innovating on the buyer side of the equation too, which impacts me personally and directly, I mean we are making it easier and faster to actually pay for things and more social and more personable.
So with Square and Square Wall in particular, you can go into anyone of our sellers and say I’m going to go into a coffee shop, I just order my latte and instead of like taking my wallet out and swiping my card or tapping my phone on something, I don’t have to take anything out, I just tell them to put it on Bob, it’s kind of like the old days when we used to be able to run tabs at stores and that sort of stuff. The really cool thing about that is the technology really fades away into the background, it’s really an illustration of the fact that design is not just about pixels, sometimes it’s about technology getting out of the way and it’s about the entire experience. To me those aspects we’re in the U.S. and Canada right now and we are going to take this all over the world, I think it’s going to be very impactful and help people’s lives all over the world.
12. Especially with the payment system security is a big issue, InfoQ has written extensively about security bugs in Java, Java 7 in particular, Oracle's slow response to that in addition, what is your take on all that?
It’s really unfortunate for a couple of reasons, one it’s more of a PR problem than it is an actual problem, that is really depressing because Java is very secure especially on the server side. What problem we are seeing is with using Java in a browser as an applet, and when is the last time that you actually run an applet? These things don’t need to be enabled anymore, and the problem is specifically that browsers try to rely on the Java Sandbox for security and reality is the Sandbox doesn’t work very well, it never worked very well, it just can’t work very well as you’ve seen from all the security issues that keep coming up. Obviously this only affects applets, it doesn’t affect Java services or anything like that like we have at Square.
And then it also doesn’t affect platforms like Android which don’t rely on Java’s Security Sandbox, they actually take a different approach much like you are on a server, they rely on Linux and process separation and running apps as different users in order to enforce that security, and you don’t have to worry about circumventing that sandbox. So it’s really just about the Java Sandbox which only applets rely on, and probably shouldn’t rely on, it’s better that in the future browsers should just not rely on that and they should just sandbox applets at the process level and run them in separate process from the rest of your browser.
That is one of the things that I really love about Square, is that we have so many different things to work on, I get to work on all of them. We are one of the few companies that I know of that do hardware in house, so that’s been really exciting to me, specifically like our reader has like a little CPU in it with 256 bytes of RAM, so we do magnetic stripe decoding and encryption and transmit the data to the phone, so obviously you have to do is streaming fashion because it won’t all fit in RAM so that was really an exciting challenge to write code for something like that and then all the way on the opposite end of the stack we have some really hardcore Java infrastructure.
One of the things that came up in my talk is that I’m a big fan of understanding things very deeply and also rebuilding things when you know how to build them better, it’s not reinventing the wheel if you are making a better wheel. We definitely do it a lot at Square, one because we have the talent and two because we have the motivation to. I think scalability isn’t the biggest challenge in payments but availability very much is, we can’t afford to get down because it creates an embarrassing experience for our customers and they lose money, we lose money, it’s just not acceptable, so we take availability very seriously and we build a lot of custom infrastructure to make sure that we can have higher availability than traditional technologies allow. And we are constantly pushing the envelope whether it’s on the mobile clients, iOS and the Android, or whether it’s on the server like I said we build a lot of Java infrastructure, custom RPC frameworks, custom service containers and that sort of stuff.
On the client for example we are really pushing the boundaries and exploring the boundaries for how to reuse code across iOS and Android, we are pushing the boundaries on how to network between the clients and our services in particular like TCP doesn’t work very well over mobile networks because we use connections but it’s not really made to work over mobile networks, it’s really made to work on a reliable network which they’re anything but. So we’re exploring a lot of interesting possibilities there and the goal being to make sure our service is not only available but it’s also the fastest out there. Jack [Dorsey], our CEO, likes to say that: “Money has been around for five thousand years and we are focused on building the next five thousand years”, so when it comes to building infrastructure we really try to do stuff right and do it with a long term in mind.
Michael: Well Bob thank you for taking the time to visit with us today!
Thank you very much for having me, it’s great to see you!