At NetApp’s recent Insight 2016 conference in Las Vegas, we offered an update on the massive progress we’ve made in both All-Flash arrays (see Gartner’s view here and IDC’s validation here ) and the significant work that’s been done to turn the Data Fabric vision first shared at Insight 2014 into today’s reality. We launched a variety of new products and features to enhance the Data Fabric, including ONTAP 9.1 and new All-Flash arrays (more color here from EVP Joel Reich and from Jeff Baxter).
In addition, Day 3’s general session offered a very special technology preview of FabricPool. FabricPool should provide the most substantial integration yet directly between the All-Flash revolution taking place in on-premises Data Centers and the massive disruptive economics of the Cloud, linking the two to produce a technically elegant solution with deep business impact.
FabricPool will be designed as an innovative technology that allows any workload (SAN or NAS) on an ONTAP system to keep hot data on all-Flash but tier inactive cold data off onto low-cost object storage, either on-premises or in the Cloud. This inactive data is often up to 80% of the overall storage footprint, leading to significant savings. FabricPool should be automated and transparent and thus will allow enterprises to capture the disruptive economics of the Cloud without sacrificing the performance and improved user experience of Flash or having to re-architect applications.
NetApp CTO Mark Bregman and Sr. Technical Director Joe CaraDonna presented this FabricPool sneak peek from the main stage at Insight 2016:
Let’s dive a little deeper by focusing on two topics in this blog:
- Enabling the All-Flash Data Center by building a bridge to the Cloud
- No Compromise Enterprise Implementation
FabricPool: Enabling the All-Flash Data Center by building a bridge to Cloud
Over the last year, the phrase “All-Flash Data Center” has been thrown around in the IT industry with fervor. And indeed, the dramatic improvements have meant that All-Flash arrays have increasingly been the right choice for most workloads, driving significant improvements in application and database performance and end-user experience. NetApp itself has ridden that curve to an industry leading position.
Today, the “raw cost” of all-Flash is still a bit more than traditional 15K/10K SAS HDDs. But even a minimum 3:1 storage efficiency ratio could bridge that gap, and customers are quickly adopting all-Flash due to the overall TCO including reduced floor space, power, cooling, plus the user experience.
But as we look at SATA drives, there’s still a sizeable (~10x) $$/GB price difference and the roadmap for archival drives for at least the next five years seems to indicate that gap won’t be closing immediately. These archival drives may be next-generation hard drives today and some form of dense solid state storage in the future, but the one thing they will not be built for is high-performance continuous random reads and writes.
An “All-Flash Data Center” will not be able to leverage these disruptive archival technologies and thus will be a less efficient and competitive resource as a result. The fact is most Data Centers will utilize disk for years to come, either on their floor, or indirectly on the Cloud.
And yet with FabricPool, an “All-Flash Data Center” should be a possibility. How? By redefining what “All-Flash Data Center” means to what really matters: That all applications, databases, and workloads rest on all-Flash storage systems. This ensures that applications are performant and end-users are delighted. But with FabricPool, cold/inactive data could be stored on lower-cost object storage using archival HDDs on-premises or in the Cloud. As previously noted, up to 60 – 80% of data is typically cold or inactive, leading to substantial savings.
Thus, with FabricPool building a Bridge to the Cloud, in the future you could transform your data center as shown:
FabricPool: No Compromise Enterprise Implementation
As we’re designing FabricPool for potential future delivery, we’re making sure it’s a comprehensive enterprise solution.
Transparent for Any Workload
FabricPool is being designed to monitor user activity on each data block and marks them either hot or cold. The meta data will be left behind on SSDs and all the cold data (packaged into larger objects) tiered to object storage, without creating any stubs or smartlinks in the primary storage. The complete process is planned to be transparent to end users or applications. All the applications will continue to access through either NAS or SAN protocols, with no re-architecting required.
Flexible Cloud Integration: Choose Your Cloud
FabricPool is planned to support StorageGRID for private object storage and Amazon S3 for public object storage. Additional on-premises S3-compliant object storage and other Cloud hyperscalers will likely be supported in the future. The choice of Cloud provider and on-premises or off-premises is yours.
Efficient On Flash; Efficient On Cloud
FabricPool will preserve ONTAP’s leadership in storage efficiency, with All-Flash FAS’s inline deduplication, compression, compaction, and thin-provisioning technologies (among others) all unaffected. This will minimize the amount of on-premises All-Flash necessary and minimize the amount of data sent or retrieved from the Cloud, thus lowering the overall cost of the solution.
ONTAP 9 has expanded security features built-in, among them data-at-rest encryption via NetApp Storage Encryption (NSE via encrypted drives) or NetApp Volume Encryption (NVE via software encryption new in ONTAP 9.1). FabricPool will not change this encryption for the primary data storage. When sending the inactive/cold data off the array, the data-in-transit is will be protected by TLS 1.2 encryption. At the capacity/cold tier in the Cloud or on-premises, data could be protected by object storage support encryption features, available in NetApp StorageGRID or Amazon S3.
Simple To Activate; No Ongoing Management
FabricPool will be set up in two simple steps. The video earlier in this blog demonstrates a potential view of the process. Step one will be to provide access details to the object storage bucket to be used for cold/inactive data. Step two will be to apply the tiering policies to the applicable aggregates and volumes. The tiering policy is dynamic and will automatically start to tier data to the cold/inactive object storage tier when the all-Flash tier begins to fill up, ensuring that you don’t run out of primary storage nor pay to store data in the Cloud before it’s necessary to tier it there.
A Unique Approach
In closing, some have asked what differentiates Fabric Pool from competitive offerings. After all, tiering of data within a storage array is a concept that goes back at least a decade, and there are some offerings now that tier some data to the Cloud. There are three main differentiators:
- FabricPool should be available for all ONTAP workloads & protocols (SAN & NAS), while maintaining all the enterprise features, efficiencies, and all-Flash performance w/ AFF & ONTAP 9.
- The tiering is planned to go to object storage on a Cloud, including StorageGRID and AWS S3 in the first release, with more heterogeneous Cloud options to be considered for future releases.
- You should be able to have multiple AFF arrays and aggregates all pointing to the same object store using FabricPool in the future. This means the days of having to size and guess exactly how much Flash and HDD to put in each Array for tiering will be at an end once FabricPool arrives. Get one or more AFFs with enough space for your active/hot data. Expand them easily if you need to. Then either point at the Cloud or build one big on-premises object store and simply add HDD capacity as it starts to fill out.
As you can see, we’re tremendously excited about FabricPool. As noted up front in the disclaimer, the information in this blog is intended to outline our general product direction, and is subject to change. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. But we are hard at work on the FabricPool technology, and we look forward to sharing more details in the future.
The information in this blog is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. NetApp makes no warranties, expressed or implied, on future functionality and timeline. The development, release, and timing of any features or functionality described for NetApp’s products remains at the sole discretion of NetApp. NetApp’s strategy and possible future developments, products and or platform directions and functionality are all subject to change without notice. NetApp has no obligation to pursue any course of business outlined in this blog or any related presentation, or to develop or release any functionality mentioned therein.