BT

New Early adopter or innovator? InfoQ has been working on some new features for you. Learn more

IBM, Databricks, GraphLab Present Notebooks as Unified Interfaces for Building Prediction Apps

| by Mikio Braun on Dec 02, 2014. Estimated reading time: 2 minutes |

At the StrataHadoop conference in Barcelona last week, Rod Smith, Vice President of the IBM Emerging Internet Technologies organization, presented work on an internal product they have been developing in their consulting work with clients that integrates data sources, and data analysis. Using an IPython-style web-based notebook interface, users can search for data sets, extract data, and build visualizations that are embedded in the document in real-time.

Smith says that business people only know what they want when they see it, so it's crucial to have a platform where they can quickly develop and prototype ideas together with data scientists. The tool, which is only used internally so far, also has a more technical view where one can directly interface to Python and Spark. Finally, the data analysis results can be easily exposed via a REST interface, for example, to build JavaScript-based visualizations.

IPython notebook is a feature of IPython, which presents an interface similar to a text console over a web browser. It has since been extended to also integrate plotting capabilities, turning it into a kind of operating system independent console with graphics output. Zeppelin is an open source project which also covers other languages like Scala, or SQL.

This approach emerged as a theme during the conference. Shawn Scully from GraphLab presented their new data analysis product, GraphLab Create, which follows a very similar approach. In a live demo, Scully put together a recommender in a web notebook, and then deployed the learned model from there. Their goal is to provide a simple tool which allows data scientists to quickly create what they call prediction apps. Scully said instead of prototyping a system in one programming language, and then reimplementing the pipeline in order to deploy it, you can easily deploy the pipeline without having to change toolsets.

Prediction APIs were also the focus of the PAPIs.io conference that took place on the two days before the StrataHadoop conference. According to Louis Dorard, who organized the conference, bridging the gap between analyzing a data set and putting the results into production is in fact still a big challenge, and he predicts there will be a lot of future potential for innovation in that area.

One of the first prediction apps was Google's prediction API, first released in 2010. Lately, offerings like Azure ML from Microsoft, or Databricks cloud provide a similar approach of a web-based unified interface to quickly prototype and deploy data analysis solutions.

Asked whether these different product will eventually be merged into one unified solution, Rod Smith said that different audiences still require different kinds of solutions. Tools like Databricks cloud, Azure ML, or GraphLab are more geared towards data scientists, who are also proficient in programming, while his work focuses more on presenting a clean interface where data scientists and business people can interact well. In short, "notebooks will be the new spreadsheets."

Rate this Article

Adoption Stage
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Discuss

Login to InfoQ to interact with what matters most to you.


Recover your password...

Follow

Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.

Like

More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.

Notifications

Stay up-to-date

Set up your notifications and dont miss out on content that matters to you

BT