Docker has made persisting data in a container environment easier by supporting storage plug-ins, which can be used to to dynamically create, attach, and delete volumes. However, Docker is facing pressure to move enterprise data storage features upstream into its platform, similar to pressures OpenStack has faced. Docker should resist the pressure and opt instead to keep things simple for users.  

The evolution of the next generation data center continues to accelerate

Just a few short years ago, “cloud” was all the buzz; some people got on board early, while some waited until it was viewed as more “mainstream.” Make no mistake: I love cloud and OpenStack, and I loved the vision it had for the future of the data center.


We used to talk about things like cloud native, scale out, and ephemeral, and for some, it really resonated. At the same time, however, cloud became a bit, well, cloudy. It sort of lost its identity and started trying to cater to any and all applications and use cases. In other words, it wants to be all things to all people. That’s great — nothing wrong with it. And the fact is that not everybody is ready or able to reinvent how they do things and throw away their existing investments.


Now, let’s consider the latest phenom in the IT industry. Some of you may remember it as chroot, others maybe as jails. Regardless of what it started as, or even when it started, chances are more likely than not you’ve heard of containers and Docker. I’m not going to go into Docker and what’s cool about it or what the use cases might look like. Instead, I want to focus on one small component of Docker and give my view on what’s good about it. That one little feature, if you haven’t already guessed, is persistent storage (aka “volume plug-ins”).

Persistent storage or the “stateless” world?

If you’ve looked at Docker, experimented, or even used it extensively over the past couple of years, you’re probably already familiar with that one thing folks always found difficult to deal with at scale: persistent data. (The other is networking. Of course, it’s always networking!!)


Okay, I can hear some of the sighs already. “Persistent storage … I’m building 12 factor apps … the world is stateless.” Okay if that’s what you’re doing then great. I’d argue that the world isn’t really stateless, even though sometimes we’d really like it to be. Sometimes you just can’t get around the fact that you need to persist data. Databases are, of course, always the primary example here, and, well, that’s because they’re pretty predominant alongside just about any application.


Okay, so let’s get back on topic here. Up until the 1.10 release, the primary mechanisms available with Docker if you had persistent data were to either use a data container or to pass a local directory into your container as a data directory when you launch it. For a lot of people that was enough; it worked, it provided persistence of data, and it wasn’t really difficult.


Let’s talk about scale out, though, and throw in things like clusters of Docker nodes. Now things get a bit more difficult. If you were using something like an NFS or ZFS share across multiple nodes as that mount point you were passing into your container, then you’re still probably in okay shape. If you were just using a local disk on the node, you’re kinda sunk. That’s where folks like ClusterHQ came in with Flocker. They made things like scaling out across multiple nodes possible and relatively easy to manage.

Adoption and scale of Docker deployments

The adoption and scale of Docker deployments has been growing at a pretty phenomenal rate. We’re not talking about one or even 10 Docker hosts now. People are talking about hundreds of nodes in their clusters or, in some cases, even thousands. (Yes, really … it’s not just hype. There are people running Docker deployments at this scale.)


Now you have a problem. Trying to move all of that data with your containers is a bit tricky. Even if you’re using something like a shared file system, there are a number of things that start to break down when you start to really scale.


This is where things get really interesting and cool. One thing Docker released in 1.10 was a framework for volume plug-ins!

Docker 1.10 and volume plug-ins

As of 1.10, you have the ability to write or obtain a plug-in for *insert-storage-device-here* and use that to dynamically create, attach, and delete volumes to use as persistent storage for your container applications.


The best part: If you do this correctly, your applications are still stateless.


You can still watch them die, restart on other nodes, and do everything that you’re taught in the 12-factor app philosophy. The only difference is that now you also can have persistent data (remember our database) that can move around with your application while being completely decoupled from it.


So here’s what we have today: We’ve got a really great design via Docker, which is easy to deploy and run. Even better, we have a plug-in architecture for volumes that lets you plug in a storage service. Docker doesn’t provide a ton of stuff here, and that’s good. It just provides an API to do things like create, inspect, list, and remove. That’s really about it. The beauty in the architecture is they leave the bulk of the work up to the driver or plug-in itself. That includes how to communicate with the device, attach and mount volumes, etc. This means you avoid things like vendors trying to push in features to differentiate themselves that then spoil the Docker interface.

NetApp SolidFire’s Docker Volume Plug-in — Not just another “Cinder wannabe”

There are lots of folks that have written volume plug-ins, including some that provide entire abstraction layers for devices (I call them the “Cinder wannabes” of Docker). Anyway, there’s a lot going on there, and rather than try and go through all of the various options, I’ll just focus on the SolidFire plug-in. The SolidFire plug-in enables volume creation, mounting/unmounting, and deletion — all through Docker and even Docker Compose. Let’s use Docker Compose as an example and walk through what happens.


Let’s say you have a good old-fashioned LAMP stack you’re running. You want to persist your MySQL database, but, of course, you want to actually run your MySQL app in a container that’s able to be restarted on different nodes (or even the same node) but still have everything work. You want it to be stateless and be a good 12-factor citizen.


You can simply specify in your docker-run or docker-compose file that you want a volume attached to a data directory in your container. That mount point in the container just happens to be where MySQL is going to be configured to store its data. So you start your application, pass in the mounted volume (the magic is in the driver, which will create the volume for you if it doesn’t exist already), and you’re up and running.


Your app is doing it’s thing, writing some data to the database, fetching tables, indexing all the wonderful kitten GIFs you are collecting. Now … something goes horribly wrong. The power supply in the server you’re hosting your super cool kitten app on dies. This is where all your hard work and adherence to converting your application to microservices and being stateless pays off. You (or whatever orchestration you subscribe to) notices that container is toast, so it just starts it back up on another node in your Docker cluster.


The start up works the same way, but this time when it queries the volume driver, the driver notes the volume already exists. Rather than creating a new one, it just mounts the volume to the new node and returns the information of the existing volume back to Docker which passes it into the app. Things start up, and since your app is stateless, it doesn’t even know anything happened. And the best part is that all of your data in your MySQL database is right there, so you just pick up where you left off.


It’s important to keep in mind, however, that just slapping in any old SAN isn’t really the answer. You need to be careful here: Choose something that can scale and is fully automatable. Above all, you want storage that is easy to manage, reliable, and can handle multi-tenancy.

PLEASE … Leave it in the driver!

Now I know some are thinking, “Neat, but I want replication, snapshots, backups, and all that other traditional enterprise IT type stuff.” My first and most natural response is to say, “No, you don’t.” My next response is “Okay, you may want it, but you’re better off without it.” And then finally I come around to meeting in the middle and saying, “Okay … but PLEASE leave it all in the driver.” So that’s what we do.


The driver itself provides a CLI (and soon some Docker-aware SDKs) to let you do some of those enterprise things that make you sleep well at night. The key here, though, is we try to keep Docker SIMPLE. Personally, I don’t want Docker to know anything about replication, consistency groups, or any of those sorts of things. It should be agnostic and simple.


There’s some danger here, and I’ll be the first to admit it. With OpenStack, for example, we wanted to do almost the exact same thing: provide portable persistent data objects that could be moved around from one ephemeral instance (stateless application) to another, and completely decouple the application and its data. We even set out initially to do it the same way Docker is doing it now. We offered volume types for custom/fancy things that a backend could do, just like Docker offers generic options. In both cases, it’s just metadata that a driver can act on.


In the case of OpenStack, things sort of got out of control, though. Vendors pushed more and more of the “special” stuff into the actual Cinder API. At the same time, many adopters just went back to the “old” way of doing things and decided to skip over the whole stateless/ephemeral idea. It’s too bad really, because it sort of devalues a lot of what OpenStack set out to accomplish. The way I see it, Docker is most likely going to reach a similar crossroads, sooner rather than later.


The question is: Will folks remain steadfast in trying to change the future of application writing and embrace the next generation data center, or will tradition prevail and water things down? Honestly, I don’t think either outcome is bad, but I do think one is better than the other. See if you can guess which.


I see some parallels between OpenStack/cloud and Docker/containers. I fully believe they have their own specific use cases and can complement each other. At the same time, I also think of containers as a sort of “do over” opportunity. It’s another chance to change how we write apps and truly revolutionize IT. I hope this time we can all embrace it a bit better than we did the last time. Maybe if we all get on the train — thinking less about where we’ve been and more about where we’re headed — we can experience something not just evolutionary but revolutionary.

John Griffith

Principal Software Engineer at SolidFire, opinionated, sometimes outspoken, but well intentioned. Passionate about open source technologies, particularly OpenStack.

Add comment