By Matt Watts, Director, Technology and Strategy EMEA, NetApp
As the reality of the Internet of Things (IoT) edges closer, organizations will need to move past strategic and theoretical discussions to something more substantive. While every mention of IoT seems to start with the game of who can cite the highest possible number of connected IoT devices, the focal point should really be how to work with the vast amounts of new data IoT devices are poised to generate. One well-known car company built up about 14 petabytes of storage over several years. But in just the last few months, its driverless car program created an additional 4.5 petabytes.
This unbridled data growth will make an already challenging IT problem – organizing the data to make it valuable – even greater. Advanced data management and organization is essential to effectively assimilating and using the flood of data that will come from IoT. However, given the scale and scope of this data, we need new thinking and approaches beyond traditional solutions.
The IoT Layer Cake
NetApp’s strategy to address the IoT is Data Fabric. Data Fabric is a broad vision for enabling data to be in the right place at the right time with the right performance and cost, for the IoT we see this as a ‘Layer Cake’ approach, so what does the ‘IoT Layer Cake’ look like? It’s a set of integrated functions/technologies to meet the challenge of IoT.
There are four fundamental layers in the “cake”:
- Partnerships and Solutions
- Common APIs
- Data Fabric
- Data Platforms/Data Lakes
Each of these four elements provides a critical part of a holistic solution that will empower the organization to use IoT data and provide IT with strategic infrastructure that isn’t prohibitively expense or too limited to support huge incoming data flows.
Partnerships and Solutions
NetApp partnerships and solutions are focused on providing customers with complete, fully integrated approaches that solve common problems or needs. At this time NetApp is currently working with two key partners on IoT solutions. Virdata, a Technicolor company, provides the ability to capture any type of data from any type of device and deliver real-time insights and analytics. The Virdata service offering is developed on NetApp FlexPod for OpenStack. The other partner, IBM Cloud (SoftLayer) provides the compute platforms, cloud infrastructure and turnkey big data solutions for deploying Virdata’s service offering. These two partnerships provide strong examples of how NetApp can offer provide complete solutions for specific applications or scenarios.
Common APIs are essential to make the layered approach work across a broad base of IT infrastructure, products and platforms. APIs are the preferred approach for interaction with key components of the layer cake, and individual developers or organizations will have their favorites, making broad API support a critical function.
Given that many IoT solutions are in their infancy, it is very likely that many new and as of yet undiscovered APIs will play an important role in the future. Simply put, we just don’t know all the answers at this time, so ensuring that there is a focus on API support is essential to the layer cake’s long-term validity. Some existing APIs will obviously play an important role. OpenStack, for example, will be a very important set of APIs for delivering value from IoT data. OpenStack is already included in the NetApp IoT layer cake approach.
NetApp’s Data Fabric provides single, standardized data management and storage solution that can work across different architectures and platforms. Data Fabric provides a common method of making data available without putting it in silos or limiting flexibility.
The Data Fabric acts as the link between the data platform, data lakes, APIs and key software products that will be used at a higher level to create IoT applications. Data Fabric provides a consistent interface that makes it easy to provide IoT data to specific workloads or applications.
The need for this type of solution is magnified when the huge amount of data from IoT starts to become part of an organization’s overall data asset. In addition, using Data Fabric allows IT organizations to store data on the most appropriate and cost-effective type of physical storage will become critical as the volume of IoT data explodes. Data Fabric does this with a “single pane of glass” management console for all data, regardless of what platform it is stored in or where it might be located.
Data Platform & Data Lakes
The data platform layer is designed to provide an enhanced approach to Platform as a Service (PaaS) for IoT. The platform is made up of autonomous components including capabilities such as compute, object store, networking, orchestration and other functions. The key to IoT support is that these components are designed to support massive scale out to ensure necessary throughput.
The data platform also provides simplified integration with key software products and environments. A good example of this is the NetApp NFS Connector, which allows analytics to run natively on NFS data with tools such as Hadoop. The data platform also supports enterprise software solutions from vendors such as Oracle, Cisco, VMware, Red Hat, Microsoft and CA.
The term data lake refers to a set of global “object pools” that provide broad access to large amounts of data. Data lakes use a flat architecture to store vast amounts of raw data, precisely the type of requirement that will be common in the IoT. Each data element in the lake has a unique identifier and an extended set of metadata tags. The use of Hadoop with data lakes is already increasing as IT organizations looking higher levels of data scaling and accessibility. However, data lakes are not specific or limited to Hadoop.
Using Data Fabric to Enable IoT Innovation
Data Fabric is an ideal approach for creating a flexible, scalable and adaptable means of managing and protecting data, both now and in an IoT-driven future. With IoT data coming from so many devices and endpoints, Data Fabric’s ability to centrally store and deliver data to any infrastructure component or workload becomes increasingly important. In addition, as the size and number of data lakes grows exponentially in an IoT-driven world, Data Fabric is an excellent solution for managing and protecting these huge data stores.
The age of IoT requires fundamental change to older, traditional data storage and management approaches. NetApp has developed a solution that bridges current and future requirements, allowing your organization to create value from IoT without storage bottlenecks or limitations.
Matt Watts will be speaking more about the Internet of Things at IBM InterConnect 2016 in the following sessions: