Apprenda is a popular container management platform also know as Platform as a Service (PaaS) that provides private cloud services to developers for developing and deploying applications on .NET and Java.
NetApp ONTAP provides a standard data management layer that provides persistent data store for containers and protects data during the CI/CD process.
Apprenda PaaS uses the some of the native ONTAP features to accelerate the CI/CD workloads like software builds and databases in the Software Development Life Cycle (SDLC) to expedite the application release in to production. For the purpose of this blog we focus on the process of providing production like databases to developers to test and validate applications during the development and staging process.
Database is often one of the most overlooked and complex parts of Software Development Life Cycle (SDLC). It is critical for quality testing to have a clean and realistic data. Setting up a production-like database for testing purposes is typically a manual and time-consuming process. It involves data copies, migration scripts, obfuscating of sensitive data, and multiple meetings between application developers, DBAs and IT – people who do not speak the same language.
Once a copy of the production database for Dev/Test is created, it quickly becomes “polluted” and less reliable with every test run. The more it is touched by instances of the applications under development, the greater is the discrepancy between the reality and the desired state of the data, due to errors in the code base. When developers fix the implementation issues, it is often necessary to restore the test data to its original state right before every new or updated instance of the application is deployed. This requirement increases the amount of complexity and overhead exponentially, especially in the organizations with frequent CI/CD practice. Because of these complexities and the time that it takes to produce a fresh data for the preproduction purposes, organizations often forego this process, which puts the code quality at risk and and impacts the delivery.
NetApp and Apprenda created an automated solution that simplifies the process of generating multiple copies of production like database on-demand for pre-production purposes. The cloning of a pristine database for Dev/Test is instrumented automatically at the application deployment time, using native ONTAP features like snapshots and FlexClones. The solution is enabled through a direct integration between the Apprenda platform and NetApp SnapCenter REST APIs. It works with a wide variety of DB engines. We will use MySQL in the demonstration.
At the compute layer, the integrated solution relies on Apprenda deployment policy engine and the abstraction model that frees the developers from the need to know intricacies of database cloning and reduces the need in meetings. Apprenda policy engine allows the operators to securely segment the platform based on various needs, SDLC environments being one of them. The established policies determine where applications instances are deployed and what databases they are connecting to. The databases similar to the applications themselves are segmented based on the deployment policies ensuring secure separation of pre-production and production environments.
Apprenda also takes the concept of the “serverless computing” down to the data tier. Multiple servers can be setup to host cloned databases, in which case Apprenda will be controlling the placement of NetApp FlexClones based on the CPU and memory utilization.
At the data management layer, the solution hinges on the capabilities of ONTAP to provide on-demand, instant, thin provisioned FlexClones of the databases that the application can quickly connect to for development and testing purposes. Through policy automation between the Apprenda platform and NetApp SnapCenter REST API, data management operations, including application consistent backups, cloning, mounting, starting the database automatically and instant DB restore, are performed with zero storage touch.
The database cloning is governed by Apprenda deployment policies and metadata, which is defined and configured by Apprenda platform operators.
For example, here the platform operators can specify what databases are available for cloning, the location of the production DB, various cloning policies, and the set of servers where the clones can be mounted. The operators also pre-configure several metadata properties that may be set by the developers or DevOps prior to the deployment. DB Cloning type is one of them.
By simply setting this property Developers can influence how the cloning is performed without the need to know any details of the process. There are three options:
- CloneOriginal uses the original version of the database (a production copy or a specially prepared data set) for the cloning. The application consuming the database gets the most up-to-date data in this case.
- RestoreClone makes an attempt to restore the last available snapshot of the cloned DB. This option is set as a default because the restore process is much faster than the initial cloning of the original.
- In both of these cases, the application starts from a clean slate as far as the data is concerned. All changes to the data that were made while the application was tested are overwritten by the recent production data or the original state of the earlier clone.
- In some cases, it is, however, desirable not to refresh the data.
- KeepExisting option allows developers to maintain the state of the data between the test runs.
This automated database provisioning is conducted in two steps. First, Apprenda Extension for SnapCenter is invoked by Apprenda Deployment Pipeline. It communicates with SnapCenter REST API to perform the desired type of cloning and mounting based on the metadata that accompanies the application. At a later stage of the pipeline, Apprenda Bootstrapper for SnapCenter is called to update the application configuration file with the new connection information, so that the application, once it is containerized by the platform, can connect to the cloned database.
Click here to watch a full demonstration of the solution.
The Apprenda plugin that integrates with NetApp SnapCenter tool for managing the databases like MySQL in the persistent data layer, enables developers to test production like databases instantly and on-demand to release the application faster and safely into production. This integration allows developers to migrate risk from production databases using native ONTAP features at the data layer from Apprenda with Zero Storage touch.
This integrated solution can serve as an enhancement to the existing CI/CD automation. It helps to increase the quality of testing by automatically refreshing application data, without delaying the verification process by manual data migrations. It helps minimize the number of meeting between developers, IT and DBAs, with reduced internal friction from negotiations about providing production-like DB to developers.
The plugin that enables the DBaaS with Apprenda on NetApp ONTAP using SnapCenter along with the relevant documentation can be downloaded from netapp.io for MySQL databases.