Twenty years ago, with the introduction of AutoSupport®, NetApp became the first storage company to leverage telemetry data from its products to help customers improve the reliability and efficiency of their storage. Today, NetApp® Active IQ®  builds on that legacy with big data analytics, artificial intelligence (AI), and machine learning.

 

Every day, Active IQ processes more than 70 billion data points from more than 300,000 Data Fabric assets around the world. These data points feed a 4PB data lake of rich telemetry data that is accessed by more than 1,500 engineers.

 

Originally the system was managed onsite in a Hadoop data lake running on NetApp systems because it allowed NetApp to right-size the storage by disaggregating compute and storage and to provide nonstop performance even when disks fail. The traditional method of commodity direct-attached storage (DAS) could not provide the same level of scale and availability.

 

To evolve Active IQ, NetApp needed to act on the emergence of AI and machine learning, and that was going to take a significant amount of compute power. Therefore NetApp took its first steps to the hybrid cloud by migrating its real-time Hadoop analytics cluster to a hybrid flash cluster combining NetApp FAS and AFF systems running ONTAP.  Accelerating the real-time Hadoop cluster with AFF gave NetApp a big increase in efficiency, speed, and flexibility to leverage the cloud while keeping the data secure across the hot, warm, cold, and archival Hadoop tiers.

 

The faster performance with NetApp all-flash systems for the real-time analytics environment means that alerts are acted on as they come in, without delay. The moment an issue is detected, Active IQ processes the message and runs analytics to determine whether it matches known predictive risks or is a new event. Recommended resolutions are instantly provided to customers, or technical support is engaged.

 

A growing set of cloud-based analytics powers the Active IQ machine-learning algorithms. Data is replicated from the on-premises AFF storage system to an AFF in an Equinix data center. Using NetApp Private Storage for Cloud, Active IQ can leverage compute from multiple clouds at the same time by using a direct connection to secure, private NetApp storage in Equinix.

 

NetApp FlexClone® technology is used to seamlessly create copy-on-write versions of data for test and development, resulting in lower management overhead while quickly getting the team all the data they need. The storage efficiency capabilities of NetApp ONTAP® have also helped the company lower data center costs while supporting exponential data growth. Overall, NetApp has derived the following benefits:

  • 10x data reduction, from 12PB on DAS to 1.2PB on NetApp AFF
  • $600 million in support costs saved annually
  • Development cycle for new analytics services reduced from 6 months to 4 weeks

To learn more about NetApp data analytics solutions, visit NetApp.com/BigData.

Mike McNamara

Mike McNamara is a senior leader of product and solution marketing at NetApp with 25 years of data management and data storage marketing experience. Before joining NetApp over 10 years ago, Mike worked at Adaptec, EMC and HP. Mike was a key team leader driving the launch of the industry’s first cloud-connected AI/ML solution (NetApp), unified scale-out and hybrid cloud storage system and software (NetApp), iSCSI and SAS storage system and software (Adaptec), and Fibre Channel storage system (EMC CLARiiON). In addition to his past role as marketing chairperson for the Fibre Channel Industry Association, he is a member of the Ethernet Technology Summit Conference Advisory Board, a member of the Ethernet Alliance, a regular contributor to industry journals, and a frequent speaker at events. Mike also published a book through FriesenPress titled "Scale-Out Storage - The Next Frontier in Enterprise Data Management", and was listed as a top 50 B2B product marketer to watch by Kapos.

Add comment