By Martin Fleming
In a recent New York Times article, reporter James Glanz asks: “Is Big Data an Economic Dud?” Mr. Glanz seems to answer his own question skeptically. The “data era,” he suggests, will not match the earlier revolutions in manufacturing, domestic life and transportation.
In addition, the Wall Street Journal posted a blog discussing that Big Data is at, or near the peak of the Gartner “hype cycle” and “big data technologies are now soon to be due for a fall into the ‘trough of disillusionment.’”
While it’s easy to be skeptical about an uncertain future, in fact, Big Data is providing business and consumer value today. Take for example the energy industry. The massive transformation of the US energy industry means that the US will soon become the world’s largest energy producer. This success is built on the use of hydraulic fracturing – or fracking – that has made vast amounts of formerly inaccessible hydrocarbons profitable to extract. Fracking technology is, in part, the result of the use of millions of sensors that generate massive volumes of data that drive and govern exploration and production.
In a related industry, electric power suppliers around the world are deploying smart grids to more effectively manage the transmission, distribution and use of power. The recent 10th anniversary of the blackout that struck the northeast US is yet another reminder that over the decade the industry has begun the process of strengthening the electric power infrastructure with increased use of information technology. Transmission and distribution systems now produce massive volumes of data that significantly reduce the probability of large scale blackouts.
In addition, smarter transmission and distribution grids have also increased the ability of the electric power infrastructure to make use of renewable energy sources, such as wind and solar. While the use of these energy sources is still in its infancy, the possibility of a significant reduction in fossil fuel use for power generation has been greatly enhanced.
There are, today, thousands of Big Data industry applications in early stages of deployment. The value these applications create is not in the data but in the use of the data. The ability of organizations to apply analytic skills to drive change and transformation is the route to value creation. The value is not in the data, per se, but in its use.
With billions of gigabytes of data coursing through the internet, it is not just the business sector benefiting from Big Data. Consumers are also benefiting.
For every successful industry application, there are thousands of consumers who are realizing substantial gains. In the energy industry, for example, the benefits from increasingly secure energy supplies and reduced prices flow, in part, to consumers.
More directly, the explosion of web-based capabilities has created an enormous increase in the variety, choice, convenience and quality that consumers now have available. The result has been a significant increase in consumer satisfaction. While businesses measure value by increased revenue and profitability, consumers measure value by the satisfaction received over and above the price paid. Everybody loves a bargain.
The vast expansion of available search tools has increased the ability of consumers to find products and services of choice with much less time and effort. Search providers – Google, Microsoft and others – use the massive volumes of data generated from search to improve and refine their offerings, algorithms and capabilities. Consumers benefit from the improved search at little to no cost to them.
Similarly, the step function increase in convenience and flexibility that online retailing has brought to consumers is also, in part, a function of Big Data applications. Successful online retailers, such as Amazon and ebay, are using Big Data applications to continuously refine available offerings, test the most responsive price points, and innovative new promotion ideas.
Recommendation engines deployed by online retailers, music sites and others that provide ideas about products that might be of interest are terrific examples of Big Data applications. These recommendation engines have been the subject of much-advanced research in machine learning and data mining. Clearly, the ability to make good recommendations can increase sales and customer satisfaction.
The deployment of Big Data remains in its infancy. With the Great Recession and financial crisis receding into the past and with the US and other economies well along the healing process, a period of stronger growth is ahead. While growth will likely remain below average for the immediate future, three or four future decades of much stronger growth is probable. The decades ahead will be one of massive global transformation with Big Data applications playing an increasing role. Ultimately, living standards will be increased for much of the world’s population.
At a similar point in economic history, John Maynard Keynes in 1931 wrote: “the standard of life in progressive countries 100 years hence will be between four and eight times as high as it is today.” After a period in which mass production technology was broadly deployed, the US standard of living 82 years later is six times higher. In the decades ahead, the use of Big Data, as part of the integration of the world’s digital and physical infrastructure, has already begun to deliver similar gains.