Twenty years ago, with the introduction of AutoSupport®, NetApp became the first storage company to leverage telemetry data from its products to help customers improve the reliability and efficiency of their storage. Today, NetApp® Active IQ® builds on that legacy with big data analytics, artificial intelligence (AI), and machine learning.
Every day, Active IQ processes more than 70 billion data points from more than 300,000 Data Fabric assets around the world. These data points feed a 4PB data lake of rich telemetry data that is accessed by more than 1,500 engineers.
Originally the system was managed onsite in a Hadoop data lake running on NetApp systems because it allowed NetApp to right-size the storage by disaggregating compute and storage and to provide nonstop performance even when disks fail. The traditional method of commodity direct-attached storage (DAS) could not provide the same level of scale and availability.
To evolve Active IQ, NetApp needed to act on the emergence of AI and machine learning, and that was going to take a significant amount of compute power. Therefore NetApp took its first steps to the hybrid cloud by migrating its real-time Hadoop analytics cluster to a hybrid flash cluster combining NetApp FAS and AFF systems running ONTAP. Accelerating the real-time Hadoop cluster with AFF gave NetApp a big increase in efficiency, speed, and flexibility to leverage the cloud while keeping the data secure across the hot, warm, cold, and archival Hadoop tiers.
The faster performance with NetApp all-flash systems for the real-time analytics environment means that alerts are acted on as they come in, without delay. The moment an issue is detected, Active IQ processes the message and runs analytics to determine whether it matches known predictive risks or is a new event. Recommended resolutions are instantly provided to customers, or technical support is engaged.
A growing set of cloud-based analytics powers the Active IQ machine-learning algorithms. Data is replicated from the on-premises AFF storage system to an AFF in an Equinix data center. Using NetApp Private Storage for Cloud, Active IQ can leverage compute from multiple clouds at the same time by using a direct connection to secure, private NetApp storage in Equinix.
NetApp FlexClone® technology is used to seamlessly create copy-on-write versions of data for test and development, resulting in lower management overhead while quickly getting the team all the data they need. The storage efficiency capabilities of NetApp ONTAP® have also helped the company lower data center costs while supporting exponential data growth. Overall, NetApp has derived the following benefits:
- 10x data reduction, from 12PB on DAS to 1.2PB on NetApp AFF
- $600 million in support costs saved annually
- Development cycle for new analytics services reduced from 6 months to 4 weeks
To learn more about NetApp data analytics solutions, visit NetApp.com/BigData.