Clojure Roundup: Distribution with Crane, Mathematics with Incanter, Builds with Leiningen 1.0

by Werner Schuster on Dec 13, 2009 |

FlightCaster analyzes data with Clojure, using Hadoop for distributing the work. The company has now open sourced Crane, the tool that handles a lot of the mechanics of distribution at FlightCaster.

Crane currently works with EC2; it makes it simple to start EC2 instances, push Clojure code to them and make them accessible via ssh. The remote instances can be controlled from a master, eg. it's possible to remotely evaluate Clojure code on the instance (from the Readme):

(eval! socket (execute (workflow some-cascading-workflow)))

The socket holds a connection to the remote instance, which will execute the 2nd argument locally.

More information is available in Bradford Cross' post about the Crane release.

FlightCaster also recently contributed statistical-learning code to Incanter. Incanter is:

[..] a Clojure-based, R-like statistical computing and graphics platform for the JVM.

Incanter bundles mathematical Java libraries with the visualization libraries of Processing. The tool can be used interactively (by using the bin/clj executable in the Incanter distro) or as a library in Clojure programs.
As the announcement of FlightCaster's code contribution mentions, there's work under way to make Incanter work with Hadoop.

Finally, the build and dependency management tool Leiningen 1.0 has been released. To get Leiningen follow the install instructions:

1. Download the script: http://github.com/technomancy/leiningen/raw/stable/bin/lein
2. Place it on your path and chmod it to be executable.
3. Run: lein self-install

Once installed, lein new PROJECT_NAME creates a new Clojure project skeleton with the basic necessary files, including the Leiningen configuration file project.clj. (Note: if the command fails with an error, the system might be missing the right Clojure.jar).

Leiningen helps with building, but also with getting dependencies for a project; it works well with Clojars, a repository for Clojure libraries which builds upon Maven.

For an example of using Leiningen, Building Incanter applications with Leiningen and Clojars.org walks through an example using Incanter as dependency.