Clojure in Action, Second Edition, Review and Authors Q&A
Manning’s Clojure in Action, second edition, written by Amit Rahore and Francis Avila, is an essential, thorough, and well organized introduction to Clojure 1.6 that explores the core parts of the language while introducing the reader to Clojure’s pragmatic and idiomatic nature. InfoQ has spoken with Francis Avila, one of the book’s co-authors.
The second edition brings a much needed update to Clojure 1.6, whereas the first edition covered Clojure 1.2. In the process, the chapters about a few external libraries, which have been meanwhile replaced by others or have become obsolete, have been removed and replaced through a broader and renewed focus on Java-Clojure interoperability, testament to the attention that Clojure has been since receiving from a broader programmers community, which includes many non-Java programmers.
Clojure in Action aims to provide a complete and detailed coverage of Clojure, without forgetting the JVM ecosystem it lives in, and it does so in a somewhat agile package where, most notably, no unneeded duplication of information is to be found. The book takes an interesting approach to sample code, which in a number of cases comes directly from Clojure standard library, e.g. when discussion Clojure macro system, so they are as close as possible to Clojure real code; in other cases, examples try to build abstractions that complement existing abstractions in Clojure, such as when discussing polymorphism and protocols.
The book contains eleven chapters, which cover the language from the ground up, starting with its building blocks, such as data structures, functions, conditionals, namespaces, etc., then gradually moving to more advanced topics such as multimethods, macros, protocols, etc. Two final chapters are devoted to two specific, highly-relevant topics: how to use Clojure in a test-driven development setting, which dwells upon unit testing, stubbing, and mocking; how to use write advanced macros and domain specific languages (DSLs).
InfoQ has spoken with Francis Avila, one of the book’s co-authors to learn more about his book, Clojure’s advantages, and its future.
InfoQ: Could you explain the motivation to write this book? Why was it necessary?
The immediate concern of the second edition is to update the book to reflect changes in Clojure 1.6. (The first edition was written to target Clojure 1.2.) But also in the intervening years the programming landscape has changed: immutability and functional programming are no longer completely strange ideas (especially to practitioners of dynamically-typed languages) and more programmers outside the Java world are starting to become curious about Clojure. The necessity of this book now is introducing a wider audience of programmers to Clojure. Ironically, this means becoming something of an apologist for Java and the JVM, which is unfamiliar to (and regarded with some skepticism by) programmers from dynamic language backgrounds. I know because I was one of those programmers!
InfoQ: Could you shortly describe your background and your experience with Clojure? What led you to Clojure?
Francis: My background is not in computer science at all but in the humanities: philosophy, theology, and ancient Greek and Latin. I was, however, a geek from an early age and taught myself Linux and Python in college. After grad school I became a web developer and wrote mostly PHP with a smattering of many other things. I discovered Clojure reading the book Seven Languages in Seven Weeks by Bruce Tate. (It’s one of the languages.) I was very excited by Rich Hickey’s ideas about immutability, state, and complexity, and the more I studied Clojure the more I felt it was confirming the poorly-articulated intuitions I had gained about managing complexity in large software projects. (The thing about PHP projects is if you aren’t keenly managing complexity, it will happily manage you!)
I eventually discovered a company in my area was hiring Clojure developers, so I applied and began working at Breeze in mid 2012. Breeze is a small Electronic Health Record company based in Louisiana which uses Clojure and ClojureScript almost exclusively.
InfoQ: Your book targets developers with no experience in Clojure or any other functional language. What advice would you give them when reading this book? What should they expect? What should they not expect?
Francis: The most important advice I can give is to keep a Clojure REPL running as you read and try things out as you think of them or have questions. Poke at the edge cases and see what happens. The code in the book is available for download (neatly arranged into namespaces by chapter) if you want to save on some typing. This process is much smoother if you can use an editor or IDE with an integrated REPL, such as Cursive (IntelliJ IDEA), LightTable, or Proto REPL (Atom). With an editor-integrated REPL you can edit into a file or buffer and automatically re-evaluate, re-define, and test (i.e. poke at) your code as you go instead of copy-pasting into a command line.
If you follow this advice, by the time you have finished you will have a firm grasp of the mechanics of writing Clojure, how to approach problems from Clojure’s perspective, and what parts of Clojure’s abstraction toolkit to use for what and when. (e.g., when and how to use first class functions, multimethods, protocols and records, macros, etc).
However, this book is not going to familiarize you with the large ecosystem of Clojure libraries and system-level patterns that are out there today. The first edition made a more earnest attempt to do this, but this is the kind of information that ages very quickly. Many of those libraries have either changed a lot since then or have been supplanted by better techniques. Rather than update those sections we have scaled them back and concentrated more on developing a core competency in Clojure and keeping library dependencies to a minimum.
InfoQ: Could you describe some of the features that Clojure provides that can help programmers be more productive in your experience?
Francis: The biggest productivity multiplier in Clojure is the rich set of immutable data structures by default. Never having to worry “what if some other code changes this thing?” or “What code changed this?” by itself cuts out a huge number of bugs and and uselessly-wasted brain cycles. It becomes much easier to program “in the small” more of the time, so you concentrate your attention on smaller pieces of functionality and compose the whole later.
Making immutability the default assumption in the entire Clojure ecosystem has enormous multiplier effects on other Clojure features which by themselves would be merely OK. For example, the core functional programming constructs provided by Clojure (map, reduce, etc) make it easy to build simple, data-centric transformation pipelines, but lots of languages have them now. (Even Java!) But this approach really sings when combined with immutable data structures because you can freely and confidently re-arrange and inspect the data in your pipelines without worrying about propagating accidental mutations.
InfoQ: Your book gives plentiful coverage to macro programming and to DSLs. This makes for a somewhat more advanced material. Could you explain why you decided to include such more advanced material?
Francis: Macros are a key part of the value proposition of Clojure (and of lisps in general), and using them effectively is a key skill. That said, DSLs are much simpler to create using macros than using the techniques common in other languages. Macros are about controlling the evaluation of forms (i.e., code-as-data, before evaluation) embedded in what are already valid Clojure programs, so you are not starting from scratch the way you are if you have to write your own grammar, corresponding lexer or parser, AST, and runtime. Instead, the full power of Clojure’s runtime is available and you are mostly concerned with adding your own syntax and expanding it in-place into executable Clojure.
In addition, Macros have more general uses besides writing DSLs. Whenever you want to emit the result of a computation instead of the code to compute it (i.e., “precompile” values), replace a function call with inline code, or control whether a block of code is evaluated or not and in what order (e.g., in a control-flow construct), you will need to write a macro, so it’s important to know how they work.
InfoQ: You also mention how Clojure can help minimizing accidental complexity. Could you give some practical examples of this? What is accidental complexity and how can Clojure minimize it?
Francis: Accidental complexity is a name we give to those problems we programmers make for ourselves. “Complexity” is contrasted with “simplicity”: a simple idea is one that cannot have anything else removed from it while remaining what it is; a complex idea can still have something taken away. “Accidental” is contrasted with “essential”: the complexity is not a part of the problem to be solved (essential), but a part of the way it is solved (accidental).
Clojure cuts down on accidental complexity by providing simpler methods of abstraction: not “simpler” in the sense of “easier to use” but in the sense of “nothing further can be taken away”. This is easier to see with a pair of contrasting examples.
In most programming languages a collection (such as an array of references) is mutable: it has some members in it now, but it could have different members in it later. These collections are “complicated” because they combine value (i.e., what members do I have) and identity (what is my address in memory) and time (what do I have in me right now). If you want to share this collection someone needs to carefully copy it before they can change it (because its value and identity are inseparable). Nor can you easily acquire a snapshot of the collection without possibly copying the entire “universe” at a moment in program time.
In Clojure, these three needs are served by distinct abstractions. Collections (those immutable data structures I keep talking about) are only values because they can’t change and because their equality with other collections is not tied to whether they have the same address in memory or the same underlying implementation. Because they are immutable, they are also a-temporal: you can indeed “hold on” to a reference to the entire state of your program at a particular moment in time without any possibility that it will change. Values are “held on to” by what are essentially boxes into which you may put a value. There are four different kinds in Clojure, each concerned with one specific way of managing change over time, e.g. async vs sync, coordinated vs independent. These value-boxes only have identity and only manage the particular kind of change (i.e., time) you need.
After some experience with Clojure, you will find that most problems can be solved almost entirely with only values and pure functions which take in values and produce other values. There are only a few places where you need the complexity required by mutation (which necessarily combines time and identity), and in those places you use one of Clojure’s reference types (i.e., the value-boxes). However, most programming languages make you pay the cost of mutation everywhere whether you need it or not, and this becomes accidental complexity unrelated to the solution of your problem but which you nonetheless need to manage.
If you want a quick explanation of these core ideas (state, value, time, identity, and behavior) and what they mean in Clojure vs in OO programming, I recommend this blog post,“The unofficial guide to Rich Hickey’s Brain”.
InfoQ: What would be the areas of the language where a new programmer, say someone coming from an imperative background, should put more emphasis? What is likely to be the hardest skill to learn and how to do it best?
Francis: The hardest skill to learn is a “soft” one: thinking of your program in terms of data transformations instead of operations or actions. A way to practice this skill is to examine your functions frequently and refactor:
- If there’s a side effect, eliminate it.
- If there’s a mutable var/ref/atom (especially a local one) eliminate it.
- If it consumes a collection, try to rewrite it to consume only one item in the collection and use map or reduce (or other sequence functions) to generalize it.
- If there are intermediate results you need to bind (with let) and use in a particular order, see if you can rearrange your steps or function signatures to use a threading macro instead.
InfoQ: What could be a relative or competitive advantage that a company could expect from adopting Clojure? And what could be the major challenges at the present time?
Francis: From a corporate perspective, you can do the same work with less code and fewer developers, and a smaller codebase is easier to change. You will also get very good application performance for very little effort. (It’s not the absolute fastest language, but it will be pretty close to Java and an order of magnitude better than Python or Ruby.) In general, Clojure will shine best in problem domains that are data-centric (vs computation-centric) and where requirements change frequently. You will have a harder time finding Clojure developers than, say, Java developers, but you will not need as many of them.
However, there are many problem domains where code safety or performance are paramount and the requirements are fairly fixed. There are some structural typing (Prismatic’s Schema) or gradual algebraic typing (Typed Clojure) libraries, but these are optional or not fully mature. Clojure also doesn’t work very well in constrained environments such as a phone or tablet. (There are related alternatives for the small client-side such as ClojureScript or Skummet.)
InfoQ: Talking about Clojure’s increasing adoption for commercial projects, what are in your opinion the most relevant factors that are driving this trend?
Francis: A lot of applications are moving to web-based deployments and need to scale out rather than up, and immutability makes scaling out much easier to reason about, both at the micro level (thread-level concurrency) and the macro level (cluster-level coherency). So I think that’s one factor contributing to its success.
Another is that it is hosted on the JVM. On the one hand, there is a huge base of Java experience, expertise, and software, but frustration with the complexity of the Java enterprise ecosystem. Clojure allows you to make incremental steps towards simplicity without throwing all that away: it’s simpler and more expressive than Java but can still coexist with it. So for an organization that’s unhappy with its nest of Java but can’t afford the risk of trying something completely different, Clojure is a good alternative.
From the other side, people from a dynamic language background who would have avoided Java because of its static typing and complexity see Clojure as an approachable language–also dynamically typed, also with first-class functions–which runs much faster than their own language. The most popular dynamic language implementations today have performance or concurrency limitations (such as global interpreter locks, or no threads at all!) that the JVM does not have, and Clojure gives them a way to leverage the power of the JVM without touching too much Java.
InfoQ: Any idea about where Clojure (the language) is headed?
Francis: I think there are three kinds of pain that some Clojure developers are experiencing which will motivate future developments in Clojure.
First, the Clojure runtime is large and starts up slowly. In a server environment with a long-running application this is a non-issue, but increasingly people want to use Clojure for short-lived programs (like command-line tools) or on resource-constrained machines like an android phone. Clojure’s extreme dynamism is a liability here because it slows things down. Clojure 1.8 added an optional compilation setting, called “direct linking”, which removes var indirection when a var is never redefined. This allows Clojure to generate smaller Java classes and more static calls. This helps a bit but isn’t enough. Other ideas that will probably make their way to future Clojure releases are tree-shaking (i.e. dead-code elimination) for smaller code size and lazy loading of namespaces for faster startups.
Second, Clojure is starting to see more implementations on more hosts. ClojureCLR (Clojure on .NET) has been around almost as long as Clojure itself, and in the past year ClojureScript has exploded in popularity. Clojure has historically had a very host-centric philosophy, but increasingly developers want to write Clojure code that is portable at least between Clojure and ClojureScript. (In fact, many Clojure libraries can already run in either environment without modification.) Clojure 1.7 introduced a feature called “reader conditionals” which allow conditional evaluation of pieces of code in the same .cljc file depending on the Clojure implementation. I expect there to be more effort to ensure that platform differences are kept to a minimum among Clojure versions. For example, Clojure 1.8 introduced additional string functions to eliminate the need to call some Java methods and increase portability of code between Clojure and ClojureScript.
Finally, there is an increasing interest in algebraic typing, especially gradual typing. Programmers want the compiler to verify types and invariants so they don’t have to write as many tests; but they also don’t want to be forced to type their entire program explicitly. There is already work in this space in Clojure, as I mentioned. Prismatic Schema, Herbert, and many other runtime structural typing libraries exist in Clojure, and the Typed Clojure project supplies optional compile-time gradual typing. I expect these projects (especially Typed Clojure) to mature further and see wider adoption. I don’t expect any changes to Clojure core to address these needs, though.
The book is available both in electronic or printed format, and its style is well demonstrated in two chapters that are freely available for download: Introducing Clojure, and chapter 8, More on Functional Programming.
About the Book Author
Francis Avila has been developing for the web for the past eight years. For the past four years he been writing Clojure and ClojureScript professionally, on the front and back end, at Breeze, a small medical software company which uses Clojure exclusively. He lives with his wife and two-year-old daughter in Lafayette, Louisiana.