One complexity of software development is the fact that a developer can be responsible for every aspect of an application. Or the job can be parsed out to specialists, automated, or adopted from existing code pools — thus including artifacts and build repositories accessed locally or across remote sites. Moreover, managing the multitude of artifacts and builds in different repositories, such as Docker, Maven, Gradle, etc., is a nightmare without a universal binary repository manager like JFrog Artifactory. JFrog Artifactory functions as the system of record and source of truth, organizing all the binary resources including proprietary libraries, remote artifacts, and other third-party resources.

 

Enterprises and their transforming IT organizations are challenged with teams that have different responsibilities in the design and development cycle of software releases. They also deal with different content and services and are usually located in different geographical locations. The development teams, for example, can be located in various parts of the world while working on modules of a single application or different applications that work collectively in production. This process paves the way to continuous streams of code development where artifacts are required to compile and package applications. The produced binaries need to be injected back into an automated maturity pipeline and eventually released in production in a continuous delivery (CD) process.

 

This ongoing process leads to the creation of massive amounts of data at every location that needs to be stored, managed, modified, protected, and analyzed, with the ability to reuse data at scale. At some point hardware sprawl (servers storing data) becomes unmanageable and does not provide an adequate solution for scalability and data protection while collaborating between remote sites. It may be argued that caching and mirroring the data from the central location can solve the problem. However, there is a communication cost over WAN when multiple users access one or many files from a central repository. Additionally, mirroring does not provide a read/write copy of the files, and most of the time are asynchronous in nature (meaning the updates are not instantaneous).

 

NetApp provides the data management platform for agile application development through its unique data access, control, continuity, and protection capabilities in a more secure and efficient manner. Data should scale and move beyond the walls of the physical constructs of storage controllers into a storage resource pool that can be provisioned on demand. NetApp products, including StorageGRID Webscale, provide a stable and standard storage platform that shares data globally. StorageGRID does not allow data to be locked in a specific format on heterogeneous storage silos.

 

Figure 1) Global site collaboration with JFrog Artifactory and NetApp StorageGRID

 

However, storage array-based replication across the remote sites does not allow JFrog Artifactory to update its database and repository nondisruptively. Integrating JFrog Artifactory with StorageGRID provides an application-aware data replication that syncs nearly instantly with the remote sites without any disruptions for the end user and the applications that depend on the data. All the file-level operations, such as read, write, and delete, are synced across all StorageGRID instances located in every remote site based on the replication policy defined in JFrog Artifactory.

 

As shown in Figure 1, JFrog Artifactory database and StorageGRID syncs the metadata changes nearly instantly with all remote sites whenever an update happens in any of the locations that are part of the GRID. The users or developers in the remote locations can now list all the updated content locally in their respective sites. StorageGRID starts the file replication instantly. Downloading the actual files locally at every remote site depends on the size of the objects and the WAN speed. The download speed of the replicated objects with a WAN latency of 170 ms between the sites provided 6x better performance with StorageGRID compared to popular hyperscaler Cross Region Replication (CRR). Downloading files locally from StorageGRID provides ~300MB/sec depending on the storage node configurations.

 

Figure 2) Performance results from Cross Site replication (CRR)

 

StorageGRID provides 1c per GB/month for storing and replicating files/objects of any size across sites compared to 2.25c per site for storing, 2c copying between sites, and 9c per GB for downloading <10TB size files/objects by a popular like Hyperscaler. The joint solution with JFrog Artifactory and NetApp StorageGRID Webscale not only provides better performance but also improved return on investment (ROI) with lower costs that can be used for collaboration as a service (CollabaaS).

 

If your organization is currently using StorageGRID and has a similar need to manage your continuous delivery of software builds and updates (and you want insight and transparency over the entire process), you should learn more about JFrog Artifactory.  Read about how to manage binaries across multisite topologies in this white paper published by JFrog.

mm

Bikash Roy Choudhury

Bikash Roy Choudhury is a Principal Architect at NetApp. He is responsible for designing and architecting solutions for DevOps workflows relevant across industry verticals including high tech, financial services, gaming, social media, and web-based development organizations that address customer business requirements in these markets. He also works on validating solutions with Red Hat (RHOSP-IaaS), Apprenda (PaaS), Docker Containers, CloudBees Jenkins, IBM Bleuemix PaaS and Perforce Helix using RESTful APIs and integrating them with NetApp ONTAP software in private, hybrid, and public clouds. In his current role, Bikash drives integrations with strategic DevOps partners, including Red Hat, CloudBees, Perforce, Apprenda,
JFrog Artifactory, IBM, and Iron.io.

Bikash has over 16 years of data management platform experience. Before joining NetApp, he worked for eight years at key accounts as a professional services engineer. For three years he was a systems administrator working on various UNIX platforms. Bikash received an MSCIS from the University of Phoenix, San Jose, and a BSc in computer science engineering from a distinguished engineering college in India.

mm

Janish Shah

Jainish Shah is Software Solution Developer @JFrog and is passionate about SAAS Solutions and Container Orchestration tools like DC/OS, Kubernetes, RedHat OpenShift, Docker Swarm.

He has hands-on experience with CI/CD pipelines on cloud (AWS, GCP, Azure). To look at his contribution to the community like (puppet, homebrew, helm, mesosphere), connect with him on GitHub.