The content has been bookmarked!
There was an error bookmarking this content! Please retry.
Posted by Werner Schuster on Dec 13, 2009
FlightCaster analyzes data with Clojure, using Hadoop for distributing the work. The company has now open sourced Crane, the tool that handles a lot of the mechanics of distribution at FlightCaster.
Crane currently works with EC2; it makes it simple to start EC2 instances, push Clojure code to them and make them accessible via ssh. The remote instances can be controlled from a master, eg. it's possible to remotely evaluate Clojure code on the instance (from the Readme):
(eval! socket (execute (workflow some-cascading-workflow)))
The socket holds a connection to the remote instance, which will execute the 2nd argument locally.
More information is available in Bradford Cross' post about the Crane release.
FlightCaster also recently contributed statistical-learning code to Incanter. Incanter is:
[..] a Clojure-based, R-like statistical computing and graphics platform for the JVM.
Incanter bundles mathematical Java libraries with the visualization libraries of Processing. The tool can be used interactively (by using the bin/clj executable in the Incanter distro) or as a library in Clojure programs.
As the announcement of FlightCaster's code contribution mentions, there's work under way to make Incanter work with Hadoop.
Finally, the build and dependency management tool Leiningen 1.0 has been released. To get Leiningen follow the install instructions:
1. Download the script: http://github.com/technomancy/leiningen/raw/stable/bin/lein
2. Place it on your path and chmod it to be executable.
3. Run: lein self-install
Once installed, lein new PROJECT_NAME creates a new Clojure project skeleton with the basic necessary files, including the Leiningen configuration file project.clj. (Note: if the command fails with an error, the system might be missing the right Clojure.jar).
Leiningen helps with building, but also with getting dependencies for a project; it works well with Clojars, a repository for Clojure libraries which builds upon Maven.
For an example of using Leiningen, Building Incanter applications with Leiningen and Clojars.org walks through an example using Incanter as dependency.
18 agile and lean practices for effective software development governance
A Guide to Branching and Merging Patterns
Improve Java Garbage Collection, Runtime Execution, and JVM visibility with Zing
In today’s hyper-competitive world, later may be too late to adopt Agile development and this Roadmap for Success will help you get started. Download "Agile Development: A Manager's Roadmap for Success" now!
Attila Szegedi talks about performance tuning Java and Scala programs at Twitter: how to approach GC problems, the importance of asynchronous I/O, when to use MySQL/Cassandra/Redis, and much more.
One category of risk that project teams need to ensure they address is business value failure – delivering a product that fails to provide value for the business investor.
InfoQ spoke to the authors of Software Systems Architecture on a couple of new topics, the System Context viewpoint and Agile, which have been added to the second edition.
Alex Papadimoulis discusses ugly code, where it comes from, how to avoid it, and how to get rid of it.
John Davies examines Visa’s architecture and shows how enterprises have architected complex integrations incorporating Hadoop, memcached, Ruby on Rails, and others to deliver innovative solutions.
Sean Comerford unveils ESPN.com’s architecture, what components are used and why, and the current changes the website goes through.
Are there repeated patterns of failure on Enterprise Agile Enablement efforts? Sanjiv and Arlen discuss Seven Deadly Sins to avoid when adopting Agile in an enterprise.
Erik Dörnenburg answers: What is Enterprise and Evolutionary Architecture?, discussing 4 issues: Turning strategy into execution, Ensuring conformance, Where do the architects sit? Buying or building?
No comments
Watch Thread Reply