Over the past few years, the enterprise data center has been transformed with regard to compute and storage. However, despite the emergence of flash storage and, more recently, the NVMe stack, external storage systems are still orders of magnitude slower than server memory technologies (RAM). They can also be a barrier to achieving the highest end-to-end system performance.

 

But now, a new frontier in storage media bridges the latency gap between server storage and external storage: storage-class memory (SCM). This new class of memory technology has performance characteristics that fall between DRAM and flash characteristics. Figure 1 highlights where SCM fits into the storage media hierarchy.

 

Figure 1) SCM in the storage hierarchy.

SCM Basics

Let’s start by comparing SCM with DRAM and flash technologies:

  • DRAM offers 100 times lower latencies than SCM and provides significantly higher IOPS. The DRAM performance advantage makes SCM less attractive as a potential replacement for DRAM, even though suppliers plan to release SCM in dual inline memory module (DIMM) format. On the plus side, SCM offers persistent data storage, has a higher storage density for a given die area, and is less expensive.
  • SCM read and write speeds are over 10 times faster than flash and can support higher IOPS while offering comparable throughput. Furthermore, data access in flash is at the block and page levels, but SCM can be addressed at the bit or word level. This granularity eliminates the need to erase an entire block to program it, and it also simplifies random access and wear leveling. However, because the price per gigabyte is expected to be substantially higher, SCM is unlikely to be a replacement for flash in enterprise storage. With new storage media, price per gigabyte is a key contributor to adoption. For example, in spite of the clear advantages of flash over HDDs, the industry hasn’t yet completely converted from HDDs to flash.

Currently, two major suppliers offer competing approaches to SCM: Intel and Micron with their 3D XPoint® and Samsung with its Z-SSD®. Although both offerings are considered to be SCM and they provide comparable performance metrics, they are based on fundamentally different technologies. In 3D XPoint®, the bit storage is based on the difference in resistance of the bulk material. Z-SSD®, however, is based on the widely used floating-gate transistors to store bits: single-level cell (SLC) and multilevel cell (MLC).

 

Other persistent memory technologies are also in development, some with the potential for broad adoption in enterprise and embedded applications, such as nanotube RAM (NRAM) and resistive RAM (ReRAM).

 

With most enterprises still in the midst of modernizing their data centers with flash and NVMe, and given the advantages of DRAM and flash in certain areas, it’s reasonable to ask: Where will SCM fit in?

SCM in the Enterprise Data Center

Applications with large datasets can exploit the value propositions of SCM especially for random read and write performance, enabling manipulation of large datasets in real time.

 

The total market for low-latency applications and workloads is huge, in particular:

  • Traditional applications like OLTP, fraud detection, financial trading, and enterprise resource planning (ERP). The market opportunity here is big but is growing relatively slowly.
  • Emerging applications like in-memory databases, artificial intelligence (AI) workloads, high-performance computing (HPC) workloads, and Internet of Things (IoT) analytics. At present, this opportunity is small but is growing rapidly.

So, where does SCM fit into enterprise storage architectures? There are three basic approaches for using SCM:

Read cache in shared storage environments.

This solution is low in complexity from a software and hardware implementation perspective. In an enterprise storage architecture, SCM-based read cache can be externally attached as an add-in card through a PCIe interface in the storage controller (Figure 2). This solution can cache hot data and metadata, thereby enabling random read performance with latencies in the sub-80µs range (including media and software protocol latencies). The read latency is therefore reduced by half, even with the fastest SSDs in the bulk storage tier.

Figure 2) SCM as an external cache.

Storage tier in shared storage environments.

SCM in DIMM format that is installed in the storage controller can be used as a storage tier, thereby improving read and write performance (Figure 3). This solution brings large datasets and applications closer to compute, driving significant performance gains (latencies in the sub-20µs range). Because disaster recovery and high-availability solutions must be architected, however, the software implementation in this case is relatively complex. For example, the solution must manage node failures, especially in the case of pending writes.

 

Another variant of this solution can use SCM-based SSD arrays as a storage tier (attached through a high-speed fabric). This solution is subject to path length delays, adding to the latency, but it is less expensive than using DIMMs as a storage tier.

Figure 3) SCM DIMMs attached to the storage CPU.

Storage accelerator in shared and direct-attached storage (DAS) environments.

The use of SCM DIMMs on the host controllers as a storage tier is a disruptive innovation that results in latencies in the sub-10µs range! (Figure 4). Because the data that resides on this layer doesn’t have to travel over the network, the read and write latency benefits are immense.

 

This solution also fits well in DAS environments and is suitable for applications that require extreme performance.

Figure 4) SCM DIMMs attached to the server CPU.

SCM is expected to play a vital role in enhancing storage performance. According to some industry analysts, revenues from SCM in the enterprise storage industry are expected to reach $1 billion by 2021. By using SCM in conjunction with existing storage technologies, your enterprise can build prescriptive solutions that meet the needs of both emerging and traditional applications.

 

At NetApp, we have a vision for new solid-state technologies like SCM. We’re developing strategies for how to use these new technologies to accelerate the digital transformation in companies like yours.

 

Find out how we plan to non-disruptively integrate these disruptive technologies. Check out this short video to see three proof-of-concept demonstrations of new storage technologies, including NVMe, SCM used as a read cache, and persistent memory (PMEM).

Sundar Ranganathan

Sundar Ranganathan is a Senior Product Manager for NetApp ONTAP based in Sunnyvale, CA. He currently focuses on introducing features and solutions for the ONTAP software suite to target AI/DL and low latency workloads. His experience includes product management and product development roles at Micron Technology, Qualcomm Inc., and Honeywell. He holds an MBA from the Marshall School of Business (USC), MS degree in Electrical Engineering from University of Southern California (USC) and a BS degree in Electronics Engineering from India.