In order to truly adopt a continuous integration (CI) process developers are challenged with building easy and efficient tools to store, change, test and integrate code into the main code base. When addressed, developers can easily identify and fix bugs early in the development lifecycle improving code quality and time to market.

 

Luckily, developers have access to tools like JFrog Artifactory and Jenkins to help manage such complications. JFrog Artifactory acts as a universal binary repository manager enabling developers to pre-build artifacts, like libraries, tools and compilers to run simultaneous builds. GIT acts as a source-code repository. Together Jenkins and JFrog enable developers to make changes at any stage of code development, test throughout development, and more easily deploy code changes when most appropriate.

Figure 1) CI workflow with JFrog Artifactory

After you compile the code, you package it with all the dependent components (such as libraries and tools) from the respective Artifactory repositories, promote it to quality assurance or user acceptance testing, and release it into production in a continuous delivery (CD) process. During the entire CI/CD workflow different versions of artifacts are required for packaging the binaries. In an enterprise, this CI/CD workflow results in exponential data growth, and all this data needs to be comprehensively managed and protected against corruption and loss.

 

Figure 2) Different versions of Artifacts used for packaging the build images

To handle this CI/CD data growth, JFrog Artifactory integrates with NetApp® ONTAP® 9 data management software for better performance, scalability and availability of data during the CI/CD process. All the data volumes—for multiple, large code bases, running parallel builds during the CI process, and pushing the successful builds to Artifactory—can be provisioned on ONTAP 9 for reliable data management and protection like backup, replication and cloning.

 

JFrog Artifactory consists of a database and a repository. The binaries are stored in the respective repositories such as Docker, Maven, NPM, Python etc. Artifactory creates an entry that maps the file’s checksum to the repository path it was uploaded to. Artifactory can connect to external databases such as MySQL to provide the performance and reliability necessary for the large workloads of enterprises that follow CI/CD process. Shared data volumes over NFS for MySQL and checksum repositories on ONTAP 9 provide scalability, resiliency, and high availability of data for Artifactory.

Figure 3) Application-consistent backup of Artifactory with SnapCenter 2.0.

For better data control and protection, MySQL and Artifactory home directory are stored in data volumes in ONTAP 9. SnapCenter simplifies backup, restore, and clone from a backup with application-integrated workflows. Leveraging storage-based data management, SnapCenter enables increased performance and availability and reduced testing and development times. Artifactory integration with NetApp SnapCenter® 2.0 provides the ability to backup, clone production databases and roll-back to an earlier state in case of any failures during the testing  and staging phase.

 

  • SnapCenter 2.0 has the ability to remotely install the agent for MySQL in the database location . The MySQL agent helps SnapCenter 2.0 to take an on-demand application-consistent snapshots of the database and the repository data volumes that are part of a consistency group.
  • If data is lost or corrupted, SnapCenter 2.0 can instantly restore to a previous state of the database and checksum repository.
  • NetApp snapshots are thin provisioned, so you can clone a consistent database instance—for example, for testing after database refactoring or applying patches—before promoting the changes to the production MySQL database. This capability mitigates risk and prevents potential errors when changes are applied to the production database.
  • For disaster recovery, SnapCenter 2.0 can also replicate the repository to another file system or S3 object store in Amazon Web Services (AWS).

The key to data protection is to create application-consistent snapshots for the Artifactory data volumes in ONTAP 9. JFrog and NetApp have developed a plug-in that lets developers generate snapshots of these volumes and the MySQL database by accessing SnapCenter APIs. This plug-in can run from the virtual machine (VM) or physical node on which Artifactory is configured. By using this plug-in, developers can keep this data in sync without the help of infrastructure administrators. Administrators can use SnapCenter 2.0 to manage other data protection capabilities such as restoring, cloning, and disaster recovery for Artifactory.

 

Meet Us At swampUP, the JFrog User Conference – May 24-26, Napa Valley, CA

If your company is a JFrog customer, but not currently using StorageGRID, then swampUP is a fantastic opportunity to learn more about NetApp integration with Jfrog Artifactory. Hear directly from our experts how you can instantly share new and updated code artifacts with development teams around the globe. To learn more, make sure you register for our session:

 

This solution offers greater synchronization speed at a lower cost, without the need to rely on cloud-based object repositories, allowing your company to maintain full control over valuable binary assets. Read more about it in this tech report:

 

Bikash Roy Choudhury

Bikash Roy Choudhury is a Principal Architect at NetApp. He is responsible for designing and architecting solutions for DevOps workflows relevant across industry verticals including high tech, financial services, gaming, social media, and web-based development organizations that address customer business requirements in these markets. He also works on validating solutions with Red Hat (RHOSP-IaaS), Apprenda (PaaS), Docker Containers, CloudBees Jenkins, IBM Bleuemix PaaS and Perforce Helix using RESTful APIs and integrating them with NetApp ONTAP software in private, hybrid, and public clouds. In his current role, Bikash drives integrations with strategic DevOps partners, including Red Hat, CloudBees, Perforce, Apprenda,
JFrog Artifactory, IBM, and Iron.io.

Bikash has over 16 years of data management platform experience. Before joining NetApp, he worked for eight years at key accounts as a professional services engineer. For three years he was a systems administrator working on various UNIX platforms. Bikash received an MSCIS from the University of Phoenix, San Jose, and a BSc in computer science engineering from a distinguished engineering college in India.

View all posts

Janish Shah

Jainish Shah is Software Solution Developer @JFrog and is passionate about SAAS Solutions and Container Orchestration tools like DC/OS, Kubernetes, RedHat OpenShift, Docker Swarm.

He has hands-on experience with CI/CD pipelines on cloud (AWS, GCP, Azure). To look at his contribution to the community like (puppet, homebrew, helm, mesosphere), connect with him on GitHub.

View all posts

Add comment