John  W. Thompson

Software-defined everything: Revolution or evolution?

August 01, 2013 6:00 AM EDT

Enterprise IT has always been on a never-ending quest to become more efficient, more flexible and more agile. If you look back over the transformations our industry has gone through over the past few decades, it’s clear that the need to do more with less and move faster has underpinned many, if not all, of the shifts. Talk of the software-defined data center, software-defined networking and software-defined storage dominate headlines these days as everyone is cloaking themselves in the “software-defined” label and touting these solutions as the latest answer to this ongoing challenge.

It is easy to get caught up in the hype cycle and believe the buzz about the benefits of new technologies. Software-defined solutions certainly hold a lot of promise -- an automated, dynamic infrastructure, business-aligned SLAs, simpler operations and lower costs. But, technology shifts like this don’t come without risks or unintended consequences that we -- as an industry and individual companies -- will need to manage.

The Never-Ending Quest

It’s clear why IT is looking for solutions that automate many of their day-to-day tasks. Today, IT professionals spend a lot of their time manually configuring and managing a vast amount of resources. Their days are full of repetitive tasks such as logging onto individual switches (or servers or storage devices) to make changes. In large infrastructures with thousands of individual devices this adds up quickly and often leaves little to no time for IT to focus on projects that could transform the business.

The need to be able to adapt quickly to changing business requirements is real. Take storage for example.  With data growing at an explosive rate -- 40% - 50% each year -- storage is quickly becoming the largest and fastest growing budget item for enterprise IT. Yet, in many organizations storage provisioning is still a slow process and it can take weeks to fulfill requests from the business.

The Need for a Closed Feedback Loop

The over-riding objective in the move to software-defined solutions is to run the infrastructure at higher levels of utilization and support faster changes to support a more agile business environment. Software-defined solutions are managed by intelligent, policy-driven software that runs independent of the underlying hardware, enabling IT to dynamically configure and reconfigure the infrastructure based on changing application and user demands.

This dynamic, flexible infrastructure in theory is great for the overall business. Yet, the constant changes add a lot of complexity to the infrastructure and drive IT’s risk profile through the roof. So many moving parts increase the chances that a transient problem -- or a ‘ghost in the machine’ -- will be harder to find and have a bigger impact on the business.

How do organizations effectively monitor software-defined solutions and ensure that each and every change is working as it is supposed to? I think the industry still has work to do here to effectively address what I consider the unintended consequence of software-defined solutions -- performance and availability monitoring.

As automation increases and IT makes real-time changes to the infrastructure, IT needs real-time information. They need ongoing insight to let them know that each change is having the right, or the desired, effect on the infrastructure. And, if it’s not, IT needs to know immediately so that it can step in and correct an issue before it becomes a major problem.

That requires a very granular level of monitoring that provides insight into what is happening across the entire, system-wide infrastructure. I think of this as a closed-loop process where a performance monitoring solution is constantly evaluating the outcome of the changes and reporting on the actions of the software layer and the impact on the underlying physical infrastructure. 

While effective performance and availability monitoring is critical to the success of software-defined solutions, today it is largely an afterthought. As IT leaders adopt software-defined solutions they need to have a plan in place for monitoring the performance of these new technologies.

The shift to software-defined solutions represents an evolution, not a revolution. These solutions will help IT leaders build the dynamic infrastructure that the business requires, but only if we understand the consequences and work to solve them. Without effective performance monitoring the software-defined data center, software-defined networking solutions and software-defined storage are flying blind.