I recently posted a blog sharing my opinion that Software-Defined Storage (SDS) was doomed. This idea provoked a visceral response from several corners of social networking. Many people commented that the EMC definition of SDS was doomed, but “real” SDS was alive and well. In this blog I’ll revisit my arguments and the arguments of others.
First, let’s get the initial point of view out of the way, and the one most seem to agree with – that EMC has taken the wrong approach to SDS. According to this article from 2013:
EMC unveils ViPR software-defined storage platform
“…On the one hand customers can choose to use ViPR’s so-called control level, which will unify the management and provisioning of storage from heterogenous storage arrays/media from one screen. This could mean EMC or third-party storage vendor arrays or even commodity hardware…On the other hand customers can make use of deeper controls in the so-called data plane. Here, ViPR uses its own Object Data Services that can be accessed via Amazon S3 or HDFS (Hadoop Distributed File System) APIs to enable management and analytics of data where it resides on the storage media…”
Two years later, there is not much left of EMC’s original promise. 3rd party and commodity storage have been left behind as EMC focused ViPR on VNX, VMAX, Isilon, ScaleIO, XtremIO, Data Domain, VCE, and ECS – all products within their stable. ViPR’s vision of being the control plane for all storage has transitioned into becoming a management interface for EMC’s 8 separate storage product lines. Daunting, yes, but software-defined storage? No.
So, then, how should SDS be defined, and does this definition have life? As we know, the exact definition of SDS varies from person to person, and from vendor to vendor, but any definition of SDS always seems to include the notion of a control plane and a data plane. This idea of “planes” came from the networking industry (specifically from Nicira) and was adopted by the storage industry a few years later. But; and this is a big but; it should not be assumed that ideas from SDN automatically carry into SDS. Here’s how another storage vendor reinforces this point:
“Unlike the networking industry, where systems from different vendors have communicated to form working fabrics, storage arrays have been dispersed into silos for disparate workloads. Without truly understanding what control plane commonly means across all array vendors, it is almost inane to apply the SDN concepts to SDS verbatim… Without a distributed data fabric (i.e., a clustered file system, or an industry-standard API for snapshots, clones, data protection, etc.) large companies are dangerously playing with fire… and customers’ emotions.”
OK, if not SDN-like, what exactly should be controlled within an SDS control plane? In my opinion, in order for a storage control plane to be called a control plane, four central points must be addressed:
- Logical storage device pooling
- RAID or Erasure Coding
- Load Balancing / Quality of Service
- Status Reporting
Other services, such as tiering, cloning, caching, snapshots, and others can be added, but the four basic requirements remain. With these four criteria met, the storage control plane is able to provision, protect, deliver, and report – the 4 basic services required from any data storage system.
After contemplating the basic storage control functions, and the idea of the storage control plane, it occurred to me that the location of the control plane seemed to define SDS or non-SDS systems. In a traditional (non-SDS) storage system, the control plane resides with each individual storage controller. For an up-and coming-class of server-attached storage, however, the control plane resides within a virtual machine providing storage services to a group of locally-attached storage devices.
Does this relocation of the control plane from the storage controller to a virtual server constitute SDS? Apparently, for many people, yes. Therefore, perhaps the best definition of SDS would be:
“Software-defined Storage: Separation of the data plane and control plane where the control plane resides outside of a proprietary storage controller.”
This SDS definition is very different from EMC’s, and holds much more promise. The benefit of this SDS approach is that sophisticated storage controls can be applied to inexpensive storage devices. Interestingly, EMC has products that meet the definition above (vVNX and VOneFS) but prefers to focus on ViPR as its SDS figurehead. Will we hear more about those software storage platforms in 2015, or will EMC stubbornly continue down the ViPR path? Time will tell…
From NetApp’s perspective, we’ve recognized the need to extend proprietary storage controllers into software storage control planes. As a result, our best-selling storage OS is today available in three variants: as a traditional controller-based OS (Data ONTAP) utilizing proprietary storage shelves, as a virtual OS residing within Amazon Web Services (Cloud ONTAP) utilizing AWS Elastic Block Storage, and as a vSphere GuestOS (Data ONTAP Edge) utilizing commodity server-attached storage devices. Each OS variant can operate independently or in conjunction with the other two variants, creating a data fabric with common storage commands and the combined intelligence to universally apply data policies across all variants.
The idea of virtualized storage, uncoupled from proprietary controllers and available via multiple connected software control planes, may just be the eventual path that SDS takes.