How the Deep Web and Petaflop Power will put the smart in Smarter Planet
Smarter Planet means intelligent infrastructure for our energy grids, transportation systems, food supply chains and healthcare networks. Ok, got that part.
It is also about trillions of devices and objects connecting to the internet and changing the way billions of people live and work. Check.
But the real revolutionary and truly transformative frontier may be the one that is this week’s focus: a New Intelligence, the leap in knowledge and insight that will come from new ways to process the large and diverse sea of information that smart systems and the “Internet of things” will generate.
For individuals, these new smarts could be manifest in things like digitally enhanced memory, what some are calling a
“life log” that captures, stores, and organizes an electronic collection of everything you do or would want to save: conversations, images, transactions and interactions of every kind, all searchable and as accessible as your biological memory. Augmented memory made it into IBM’s latest Five in Five report, which focuses on five technologies that may change the way we live in the next five years.
This new intelligence also promises to serve your health and wellness. Not only could all of your lifelong healthcare-related data securely collect in a kind of online health data bank, but new computing and sensing built into your clothes and environment would continually monitor the state of your health. Together these resources, along with the growth of personalized genomic medicine, could keep you healthier by catching any early signs of illness so that doctors could then proscribed advanced preventative care.
For businesses, the New Intelligence will offer richer and more realtime ways to run a company, including better ways to forecast market trends, make smarter decisions and have a more precise awareness of every facet of a firm’s operations and performance.
For society, this ability to understand and probe the world at a fundamentally new and deeper level will enable scientists and researchers to hunt more effectively for solutions to our most pressing problems, such as clean energy, climate change and water scarcity.
This idea of an evolutionary jump in the IQ of the world may seem on the one hand fanciful, and on the other, a bit abstract. But several of the new Smarter Planet ads that launched this weekend bring the concept back down to Earth, like this one, called Smarter Petaflop.
The world may be becoming a more complex, and information-intensive, place, but at the heart of this vision for New Intelligence are two basic ingredients.
• Unprecedented new sources, and quantities, of information
• New processing power and programs to turn raw data into a more valuable commodity: insight and intelligence that humans can use.
For the sake of brevity, I’ll just touch on one example that illustrates the two sides of the new intelligence coin.
Data Mining the “Deep Web”
Almost anyone can sense that the amount of information in the age of the Internet grows relentlessly. In fact, the Web we see or come across in searches is literally the tip of the info-iceberg. This “surface Web” is about 167 terabytes of data, while the larger, total amount of information on the Internet, the so-called Deep Web or Deepnet, is estimated to contain about 91,000 terabytes. We are already in the era of Big Data.
Powering Up the Petabyte Age
A petabyte is 1,000 terabytes, or 1M gigabytes. To put that in perspective, Facebook stores about a petabyte of user photos, or roughly 10 billion pictures. Fortunately, the power to process this deluge is keeping pace.
That information arms race is being enabled by supercomputers like IBM’s Roadrunner, housed at the U.S. Department of Energy’s Los Alamos National Laboratory, which in 2008 broke the “petaflop” barrier, or more than a quadrillion calculations a second.
Indeed, the Web itself is becoming a vast platform for computing and data processing, through new techniques like grid computing, which harness the data crunching power of idle computers and devices around the world, or cloud computing which turns the fabric of the Web itself into a processing layer.
One of the ways in which all this digital horsepower can extract insight from massive sets of data is to build very sophisticated models that can simulate and predict the solutions to complex challenges like how protein molecules fold, understanding weather patterns and environmental conditions, or optimizing logistical nightmares like global airline routes.
Down the road, even more powerful models, based on emerging frontiers such as quantum computing or cognitive computing (which seeks to mimic the power of the human brain) will open up new ways to squeeze even more useful intelligence from our ever-growing sources of information, often in close to realtime when that new knowledge may be most valuable.
Finally, it may be less sexy than the long-term promise, but some of the most immediate uses for the new intelligence are going to be in the operational nuts and bolts of business, fueling the productivity and efficiency gains that will empower enterprises to recover from the current economic crisis and get back to growth and innovation.
Of course, all of us are smarter than any of us. So what does this New Intelligence model mean to you, and where do you see it going?
IBM Global Business Services, Strategic Programs