Building a Smarter Planet. A Smarter Planet Blog.

Over the years, the game of cat and mouse between cyber attackers and the people charged with defending networks against their advances has become increasingly more complex. Every new advance in defensive technologies has forced attackers to adopt new tactics, and every new attack technique has produced a new response.

We’re at the point where some of the most diligent and advanced security organizations in the world have deployed over 60 different security products; products that, unfortunately, infrequently communicate with one another. Realistically, we can’t rely on these disconnected technologies to be successful 100 percent of the time, especially when they operate in isolation. We need a different, foundational approach.

Fortunately for security professionals, even the most advanced attackers share the same human limitations as the people defending the networks: they are not perfect and they will leave clues about their presence in a network. The enduring challenge is to figure out how to identify and combine those subtle indicators of an attack. Today, more advanced organizations are turning to Big Data in search of evidence of security breaches. 

These data sources may include full email text, business process data, network and flow data, communications channels scrapes and a whole host of others. Some organizations want to do things such as look at 50 years of transactional data to create models of typical behavior so they can better understand deviations from the norm.

As attackers continue to evolve their targets and tactics, it seems more likely that the number of sources will continue to increase. While we could continue to apply security technology to each new area of vulnerability, it would only defeat the underlying goal of a long-term security strategy – one that is based on the data rather than the source of the data.

We need data from everywhere. Tomorrow we’ll need data from sources we don’t have today. The question will be, “does my security strategy change just because we have added another piece to the puzzle?” IBM has designed systems over the years, ones that have processed and analyzed tons of data, Big Data, and the interesting thing that we discovered was that the more data we put in, the more quickly and accurately we got answers. In other words, the security strategy of the future will be built on the underlying premise that every new source of data is a blessing and not a curse.

To do this requires not only new thinking, but new capabilities. The good news is that this capability is something that can be derived from existing technology. For the last year or so we have been talking about the notion that business and security intelligence were on a collision course. The teams in business intelligence and analytics have developed ways to visualize and extract insights from extremely large data sets in every industry imaginable, working on everything from traffic patterns to consumer shopping trends.

At the same time, security professionals have been developing technology to make sense out of the millions, sometimes billions, of security events that organizations see every day. This technology was purpose-built for security challenges and the data associated with them. Now, as more advanced organizations begin to focus on what such Big Data is telling them, they are looking for a combination of what these two technologies can provide.

More than 10 years ago we began work on a simple security log management tool that over time evolved into something that could correlate and analyze security events and information (most notably things like network, firewall and users logs) in real-time. It was then expanded even further to include capabilities that enabled people to better understand and analyze network flow data. and then today, where we announce the combination of security intelligence with big data. 

Today, we are announcing the next step on that journey, the combination of security intelligence with Big Data and business intelligence. By combining these worlds of business and security intelligence in new ways, organizations are able to detect and remediate threats that they had once missed.

All of this is made possible by widening the scope and scale of investigation, and analyzing more data – Big Data – more flexibly, and ultimately delivering more accurate and timely results than ever before.