Instrumented Interconnecteds Intelligent

Dr. Hendrik Hamann. Physical Analytics manager, IBM Research

Dr. Hendrik Hamann. manager, Physical Analytics at IBM Research

By Hendrik Hamann

Five years ago, a few of my IBM Research colleagues and I played a hunch. Large-scale solar power was taking off, but we realized that for solar to fulfill its potential for helping to produce a more sustainable energy future, it would have to be integrated into electrical grids. For that to work, you would have to know ahead to time how much solar power would be generated when and where. That realization spawned our solar forecasting research project.

Today, we have shown that we can generate accurate forecasts of solar energy (from minutes ahead to many days ahead), which in turn can have a significant impact on the energy business – and on the future of sustainable energy. Our preliminary findings, including a test conducted at ISO-New England, the grid operator serving Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island and Vermont, show that our system can be 30 percent more accurate than other state-of-the-art approaches.

Through the US Department of Energy’s SunShot initiative, we’re now making our forecasts available to government agencies, utilities and grid operators in 48 states so they can evaluate how solar forecasting could affect supply and demand planning, as well as grid operations.

The forecast technology uses a combination of machine learning, big data analytics and mathematical modeling of complex weather systems. We developed it in collaboration with government, academic and industry partners. The system continuously monitors weather conditions including satellite observations, and analyzes that data to forecast how much solar energy will be available at different locations and times. One of the differentiating features of the IBM approach is that we incorporate a large number of weather and solar energy prediction models, which are then blended using historical data as a function of weather situation, forecast horizon and location to create a so called supermodel. Other solar forecasting systems instead take narrower location and timeframe views.

Our advances are important for the future of alternative energy. (We also use the same basic approach to produce wind and hydro forecasts.) But the implications are far broader. Our approach of combining information from domain models (in this case, weather models) with machine learning and big data analytics can be applied to many other domains – everything from structural health management of large infrastructure such as bridges to predict when maintenance will be needed, to understanding the workings of the human body to better administer medications.

Think of it as big data meets science. By bringing these elements together, people in a wide range of industries and professions will be able to understand better how the world works so they can make better decisions.

But let me take you back to the beginning…

Five years ago, my colleagues and I were eager to test some ideas about how we could provide better solar forecasts, but we needed some data. So, we developed a specialized sky camera system which projects the image of the sky in real-time on a video camera. This sky camera, which is good for short-term forecasting (5-10 minutes into the future), allows tracking clouds with respect to the position of the sun, as well as observing the formation and dissipation of clouds. The system had to be special because it requires unique optics (the original system used a projecting mirror, but we now use a fish eye lens) that can’t be saturated by sunlight – it’s like staring at the sun without squinting. In the photo below, the first generation camera is to the right, with the second generation to the far left, and the third generation in the middle.

solarcameras_3gen

Three generations of sky camera installed on the roof of the IBM Research Thomas J. Watson Research Center.

The camera designs have come a long way since we built the first one. The camera is almost 10x smaller now, and to avoid saturation it uses electronics rather than a mechanical shutter. Because of their compact size, the new cameras can be readily deployed at a solar power plant, where is can measure the height of the clouds using triangulation techniques. This in turn helps to improve the forecasts further.

With these cameras, we capture thousands of high resolution images per day, which we feed to the system so it learns to identify different kinds of clouds and cloud behavior. We also provide the system with a stream of satellite images of the earth and countless weather forecast models. Our “secret sauce” is the way we categorize different weather and climate situations so we can learn why various weather models fail under certain circumstances. That helps us fine-tune our weather models for particular locations.

Because solar and wind energy production is intermittent and it has been largely impractical to store large amounts of electricity, it’s vital for utilities and grid operators to be able to accurately predict the amount of solar and wind energy that will be available. Armed with that information, they can efficiently manage their entire portfolio of energy sources, including coal, gas, wind, and solar.

Until now, most experts agree that because of intermittency, solar energy won’t supply more than 20 to 30 percent of the United States’ energy. However, there is good reason to believe that with better forecasts, it might be possible to push solar’s energy contribution up to 50 percent. As we continue to refine our system in collaboration with the DOE, we hope to double the accuracy of the system in the next year. That could have a huge impact on the energy industry – and on local businesses, the economy and the natural environment.

Physical science has been a big part of my life. So it’s gratifying to bring together my curiosity for understanding the physical world with two of the most exciting fields in computer science – data analytics and machine learning.  I’m convinced that it’s at the intersection of scientific disciplines that the future will be invented.

 

Bookmark and Share

Previous post

Next post

31 Comments
 
May 25, 2016
8:14 am

One of the best post I have seen!


Posted by: Mobdro for pc
 
May 25, 2016
6:45 am

when the development environment is decided and the plan sorted, what about deploying the code as it takes shape, from their laptops onto testing boxes while ensuring the repositories are managed?


Posted by: shareit for pc
 
May 24, 2016
5:13 am

download shareit


Posted by: shareit offical
 
May 24, 2016
5:12 am

download cinemabox


Posted by: cinemabox hd
 
May 24, 2016
5:12 am

download vidmate apk


Posted by: vidmate download
 
May 24, 2016
5:11 am

check pan card details


Posted by: pancard status
 
May 23, 2016
8:26 am

Great post on solar energy forecasting applications.


Posted by: vidmateappdownload.net
 
May 20, 2016
6:36 am

good It can be the ideal software the goal of training video internet and this with your good friends, you can talk about them with Mobdro APK nice.


Posted by: Mobdro Download
 
May 16, 2016
11:01 am

Good post.You have noted some excellent points. I as well believe this may be a very superb website.


Posted by: see copa america live here
 
April 13, 2016
2:01 pm

you waiting get this latest working cinema box for ios http://www.playboxhdappdownload.com/download-cinemabox-hd-for-ios-ipad-iphone-ipod/ and watch latest tv shows and online movies


Posted by: cinema box apk
 
April 11, 2016
12:30 am

keren infonya, saya suka sekali


Posted by: celana hernia anak
 
April 7, 2016
8:32 am

Yes i agree that solar power is one such reason why energy is eveil but superman was revied by it in the BVS movie as well


Posted by: iTube For PC
 
April 7, 2016
8:31 am

wow such a nice post and i really like it very much indeed


Posted by: iTube For PC
 
March 30, 2016
6:11 am

what are you waiting get this latest working cinema box for ios http://cinemaboxappdownload.com/cinema-box-app-for-ios-ipad-iphone/ and watch latest tv shows and online movies


Posted by: jiya
 
March 24, 2016
12:07 am

I do consider all of the ideas you have introduced to your post.

They are very convincing and will certainly work. Still, the posts
are very quick for newbies. May just you please prolong them a bit from subsequent time?
Thanks for the post.


Posted by: lawyers studied
 
March 4, 2016
8:20 am

nice


Posted by: imo for pc
 
March 4, 2016
7:19 am

get the latest movie app http://playboxhdv3.com


Posted by: arda
 
March 3, 2016
12:10 pm

thanks
snapchat pc


Posted by: snapchatpc
 
March 3, 2016
12:05 pm

thanks
mobogenie for pc


Posted by: mobogeniepc
 
February 15, 2016
9:29 am

Dulaari asks Dhaani why she is washing clothes early in the morning. Dhaani says Kanak was washing clothes so I said I will wash it. Badi Amma asks where is she? Dhaani says she went to temple.


Posted by: http://ishqkarangsafedtv.com
 
January 20, 2016
9:22 am

nice article


Posted by: meizu m2 note hard reset
 
January 11, 2016
4:57 am

Mobdro is like a must have for anyone who likes to watch TV shows online. let us know your doubts about this amazing app through the comment section below if you’ve any. Thanks and keep visiting for more.mobdrodownloadapp


Posted by: mobdrodownloadapp
 
December 25, 2015
10:10 am

thanks for giving information in this blog.


Posted by: vidmate downloader
 
December 23, 2015
12:54 pm

hanks for this amazing post I liked it a lot man


Posted by: wwe smackdown biss key
 
November 17, 2015
7:15 am

Thanks for this amazing post I liked it a lot man


Posted by: shareit download
 
September 17, 2015
7:23 am

Thanks for the spacious information in the article very well


Posted by: Exam results
 
July 24, 2015
5:12 am

Interesting developement. If the technology can truly predict and analyze the various climate factors, would it be possible to predict precipitation as well? That could prove very valuable since the weather extremities could be somewhat countered by advanced warning for example.


Posted by: M Juhasz
 
July 23, 2015
1:02 pm

Incredible work!!! I’m excited to see where this genius project goes!! Keep going! All the best!


Posted by: T.Chaves
 
July 21, 2015
11:02 am

Great work! Still lot to be done, keep going..All the best!


Posted by: Jitendra
 
July 17, 2015
4:10 pm

IBM is great! IBM sets benchmarks for others to chase. What an amazing pool of talented innovators.


Posted by: Ashwani
 
July 16, 2015
1:32 pm

Solar forecasting combined with better solar panels will create a new era of energy management.


Posted by: InterceptCS
 
7 Trackbacks
 
October 1, 2015
12:13 am

[…] our work with the US Department of Energy’s SunShot initiative, and, in collaboration with ISO-New England, a regional grid operator and other industrial […]


Posted by: Chasing the Sun–with Cognitive Technology
 
October 1, 2015
12:01 am

[…] our work with the US Department of Energy’s SunShot initiative, and, in collaboration with ISO-New England, a regional grid operator and other industrial […]


Posted by: Chasing the Sun--with Cognitive Technology « A Smarter Planet Blog A Smarter Planet Blog
 
September 12, 2015
10:07 am

[…] This article is originally posted A Smarter Planet, you can also read the story here. […]


Posted by: IBM Improves Solar Forecasting Technology | TTalents
 
August 3, 2015
1:20 pm

[…] Here Comes the Sun: Why Forecasting Will Enable a Greener Energy Future —  IBM have made a few announcements lately about their work in and application of weather forecasting.  It is interesting what they are doing here: “The forecast technology uses a combination of machine learning, big data analytics and mathematical modeling of complex weather systems.”  As I read it, they are using historical and recent data as well as historical performance to, in some sense, create an “optimal” forecast from an ensemble of systems (including multiple numerical models.) […]


Posted by: WeatherConnector
 
July 24, 2015
5:00 pm

[…] Building a Smarter Planet: Here comes the sun – Why forecasting will enable a greener energy f… Until now, most experts agree that because of intermittency, solar energy won’t supply more than 20 to 30 percent of the United States’ energy. However, there is good reason to believe that with better forecasts, it might be possible to push solar’s energy contribution up to 50 percent. […]


Posted by: Smart Grid Above the Fold: 7/24/15 | Smart Grid Consumer Collaborative
 
July 17, 2015
3:47 am

[…] program funded the by the U.S. Department of Energy’s SunShot Initiative, the breakthrough results suggest new ways to optimize solar resources as they are increasingly integrated into the […]


Posted by: Machine Learning Helps IBM Boost Accuracy of U.S. Department of Energy Solar Forecasts by up to 30 Percent | leonhenr
 
July 16, 2015
3:17 pm

[…] program funded the by the U.S. Department of Energy’s SunShot Initiative, the breakthrough results suggest new ways to optimize solar resources as they are increasingly integrated into the […]


Posted by: Machine Learning Helps Boost Accuracy of U.S. DOE Solar Forecasts by up to 30 Percent - Solar Thermal Magazine
 
Post a Comment