Making Sense of Large Amounts of Data with Pivot
Microsoft’s Live Labs has published a new research project called Pivot. Pivot is intended to help people make sense of large amounts of information. Currently, the web is a huge collection of isolated pages that have links from one to another. One can browse through the large amount of information existing in these pages in a linear fashion, moving from page to page. It is very difficult to get a broad view on all the information one can see in a browsing session. Pivot tries to organize the information in such a way that one can easily navigate from top to bottom and back.
The basic concept used by Pivot is a Collection which is a group of of hundreds, thousands or millions of items that have some attributes in common. By selecting one or more of these attributes, Pivot will reduce the items displayed only to those having the selected attributes in common, helping the user discover the information they are actually looking for. Each item has an image associated to it, and practically Pivot manipulates images. When a particular image is chosen, the user can jump to the web page associated to it.
There are three types of collections:
- Simple – the most common type, easy to create, it can contain up to 3,000 items; data and visuals are static
- Linked – medium complexity, limited by storage; it is generally used for thousands of items which are loaded once when the collection is accessed; visuals are static
- Dynamic – difficult, unlimited interactively gathering information from all over the Internet, i.e. data is dynamically discovered and loaded; visuals are partially dynamic.
The Pivot application is a .NET application using the Seadragon rendering engine, also a project from Live Labs. The Seadragon technology enables one to smoothly browse graphics and images even if it requires large amounts of data or network bandwidth. The Silverlight implementation is known as Deep Zoom, but there is a Seadragon Ajax version that works without a plug-in. Some of the application’s features are:
- Speed of navigation independent of the size or number of objects.
- Performance that depends only on the ratio of bandwidth to pixels on the screen.
- Smooth, continuous, transitions.
- Rapid, near perfect scaling for screens of any resolution.
When the user has found the information they were looking for, they can simply select a link to jump to the web page behind the associated image, and Pivot will display the page using the embedded IE’s Trident engine:
One of the quickest ways to create collections is to use the Pivot Collection Tool for Excel. By filling the cells of a spreadsheet with item information and exporting it, one can obtain a collection that can be stored on any web server and browsed inside Pivot.
Other tools are Deep Zoom Composer or Deep Zoom Tools Command Line Utilities, both being useful to create Deep Zoom images associated to a Pivot collection. For production collections, it is recommended to create the images and CXML file programmatically using the Deep Zoom Tools Library (Deep ZoomTools.dll).