BT

IBM’s Software Architecture for Astronomically Big Data

by Michael Stal on Dec 01, 2011 |

IBM has recently prototyped a software architecture that can deal with large amount of data flows. IBM’s software is built for the SKA telescope (Square Kilometre  Array) and allows to automatically classify astronomical objects. Radio astronomer Melanie Johnston-Hollitt at Victoria University, Wellington , NZ, has collaborated with IBM for developing the system.

Main goal of the SKA project  is to perform unprecedented observation of radio sources using a network of dishes and aerials spread over Australia and New Zealand or through Southern Africa. A main design challenge is how to process one Exabyte of raw data per day. This is the data amount anticipated when the SKA system as the world’s largest and most sensitive radio telescope will be ready; it’s construction will start in 2016. IBM claims that this data amount exceeds the entire daily Internet traffic. The amount would suffice to fill over 15 million 64 GB iPods.

IBM announced on 30th November that it has prototyped

a new software architecture for automating data management, potentially making it easier for researchers to collect usable information from mega-scale data collection projects like the Square Kilometre Array (SKA) global telescope which aims to address unanswered questions about our universe.

With the support of Dr. Melanie Johnston-Hollitt the company created the Information Intensive Framework (IIF). According to IBM, the software uses the International Virtual Observatory Association Ontology to classify collected data into concepts understood by astronomers and then provides intelligent 'guided search' functionality. The ontology is technically based on  the Ontology Web Language (OWL). By automating classification, astronomers hope to increase productivity and creativity.

While originally developed for the SKA, the IIF could also be leveraged  in other domains. As Douglas Watt, Chief Technology Officer  of IBM New Zealand, explains:

While developed with SKA in mind, the results are also applicable to other organisations faced with a ‘data deluge’. We have identified several local scenarios which would benefit from automated analysis of performance data to uncover trends, identify anomalies and improve decisions. These range from individual manufacturing plants and telecommunications companies to whole transport networks and healthcare systems.

Further work on IIF will comprise, among other topics, the achievement of performance improvements by leveraging parallel processing.

Readers interested in the SKA project can view an image at Flickr that illustrates some of the impressing details of SKA.

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Daisy chain of press releases - where's the actual info? by Michael Nygard

All that this article or the linked press release says is that IBM did some work. Is there any actual detail available about the architecture itself, or is it just an announcement that they've done a project?

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

1 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT