Building a Smarter Planet. A Smarter Planet Blog.

John Kelly, Senior Vice President and Director, IBM Research

By John Kelly

A few weeks ago, I shared a dinner table in Johannesburg with Adrian Tiplady, one of the managers of Square Kilometre Array South Africa, which is managing the country’s involvement in the Square Kilometre Array astronomy project. The SKA is one of the most ambitious science efforts ever launched. The goal of the 10 countries involved is to decipher radio waves from deep space in order to solve the riddles of the universe and the nature of matter. Yet something Adrian told me totally blew my mind: he said the computing challenges posed by the SKA are just as great as those related to astronomy.

It’s gratifying when scientists from other domains come together to push computing and computer science forward. And it’s even more gratifying when organizations like Tiplady’s form partnerships with IBM to bring cutting-edge technologies to bear on the most demanding tasks ever dreamed up by humans. Today, SKA South Africa announced that it has joined IBM and ASTRON, the Netherlands Institute for Radio Astronomy, in a multi-year public-private partnership funded primarily by the Dutch government aimed at developing an information technology system for harvesting insights from the SKA’s data.

The SKA’s more than 500,000 antennas scattered across southern Africa and Australia will produce a torrent of data that’s equivalent to 10 times the daily global Internet traffic. Yet today’s state-of-the-art computer technologies simply aren’t up to the task of gathering, processing and making sense of so much information in real time. So I believe that SKA represents the ultimate big data challenge, and that the technology we develop for handing its data will provide a foundation for the next era of computing, which we at IBM call the era of cognitive systems.

IBM has placed multiple bets on the SKA. We’re working with ASTRON and South Africa on the DOME project, the effort we launched with ASTRON last year to develop technologies that we believe will be crucial to managing the SKA’s data. ASTRON is already using an IBM Blue Gene supercomputer to process the data produced by LOFAR, a precursor to the SKA radio telescope. Meanwhile, in Australia and New Zealand, IBM has provided supercomputers for use by university researchers who are analyzing data from the Murchison Widefield Array radio telescope in the Australian outback—another precursor to the SKA. Working with those scientists, a team of IBM software programmers has created a prototype system for collecting and searching the large amounts of data collected by the SKA.

Bradley Frank, Prof. Erwin de Blok and Moses Mogotsi doing SKA research at the University of Capetown. Photo credit: Katherine Traut

Far beyond the SKA, I believe that advances we and our partners make will be useful in a wide range of computing applications—everything from core scientific research to analysis of business data across a range of industries, including health care, banking and telecommunications.

Remember Watson, the IBM computer that beat two past-champions at the TV quiz show Jeopardy two years ago? The work we’re doing for the SKA will enable computer scientists to develop cognitive computing system that will help us penetrate the incredible complexity of data that is sensory, noisy and vast. Like Watson, it will learn and reason—helping human experts to see the world more clearly and make better decisions. The SKA work truly is one of the first major forays into a new era of computing.

——

Read the white paper, The Square Kilometre Array: The Ultimate Big Data Challenge.

On March 26 at 17:00 Central European Time, the IBM Diversity Recruiting team invites professors, scientists and university students to participate in a virtual recruiting event for several open positions on the DOME project.

Technorati Tags: ASTRON, DOME, IBM, Square Kilometer Array