In version 3.0 of the Blockbridge Volume Driver, we introduced Storage Profiles. This post goes into detail on what Storage Profiles are and how to use them.

What are Storage Profiles?

Storage Profiles provide a way to templatize resource constraints, provisioning parameters and volume options. You can specify storage requirements in a profile and then reference the profile to provision similar storage from any docker host running the Blockbridge volume driver. For example, you may create a profile to ensure that storage and compute are collocated in the same L2 networking domain. Or, create tiers of storage ranging from old-school EMC spinners to high performance software-defined storage running on commodity NVMe.

Storage Profiles can be referenced by and managed from any Docker host without the need for additional clustering software or setup. There are currently 2 ways to interact with them. You can use the driver’s command-line utility or communicate directly with driver’s RESTful API. Yes, the Blockbridge Volume Driver now exports an API that can be accessed via Unix Domain Socket! From simplicity, the examples below use the volume driver’s built-in command-line utility.

Profile Definition

Each profile specifies a Blockbridge volume type and user, at minimum. The type argument specifies the high-level data service provided by the volume driver. The user argument specifies the Blockbridge tenant storage account that owns the data (recall that Blockbridge is natively multi-tenant). Additional parameters and provisioning constraints are supplied depending on the volume type.

As of 3.0, the Blockbridge Docker volume driver implements 3 distinct volume types. AutoVol provides dynamically provisioned / assembled persistent storage. AutoClone provides storage that is dynamically cloned from a snapshot (i.e. writable snapshots). Snappy provides automated provisioning and scheduled snapshots with integrated snapshot management. The table below summarizes the required and optional parameters for each volume type.

Provisioning Attributes

Provisioning attributes define the storage, workflow and environmental requirements of a volume. The attributes that you can select reflect the organization and configuration of your Blockbridge deployment. If you are running a basic single node / single datastore setup, you can safely ignore these. If you are running a more sophisticated deployment, perhaps with mixed-vendor storage, multiple networks, or multiple datacenters, attributes give you complete control over placement.

In a Blockbridge deployment, attributes are used by administrators to describe and catalog storage assets via user-defined keywords: things such as compute/network location, media type, performance, data protection, power domain, datacenter, customer, compliance requirement, storage vendor, etc. The native Blockbridge API allows applications to specify resource constraints using a simplified query language. 

The attribute parameter in the profile definition specifies a resource provisioning constraint as a query string. The query string takes the form of “+attr” to require an attribute and “-attr” to forbid an attribute (e.g., “+ssd +qos -production)”.

Creating a Storage Profile

The easiest way to create a profile is via the command-line interface in the Blockbridge volume driver. Note that you can create profiles from any Blockbridge Volume Driver, running on any Docker host. The create operation is atomic and newly created profiles can be referenced immediately via other Docker hosts. The syntax for the create operation appears below:

... profile create --name profile [options] [attributes]

# docker exec blockbridge-volume-driver \
  profile create --name profile --user block --capacity 32GiB -- +ceph -production
[
{
"name": "profile",
"user": "block",
"capacity": "32GiB",
"type": "autovol",
"attributes": "+ceph -production"
}
]

Default Storage Profile

When a volume is created without specifying volume options, it will inherit the options specified in default profile. Therefore, the default profile is implicitly referenced for anonymous built-in volumes, and all other implicitly created volumes. If a default profile does not exist, implicit volume creation will fail.

# docker exec blockbridge-volume-driver \
  profile create --name default --user block --capacity 32GiB -- -production

[
{
"name": "default",
"user": "block",
"capacity": "32GiB",
"type": "autovol",
"attributes": "-production"
}
]

Volume Creation using a Profile

To create a volume using a named storage profile, you must explicitly provision the volume using the docker volume create command. If the profile argument is not supplied, the default profile is selected. Note that all implicitly created volumes automatically select the default profile.

# docker volume create --name myvol --driver blockbridge --opt profile=profile

Listing Storage Profiles

To retrieve a list of storage profiles and details, use the profile ls command implemented by volume driver command line interface, as shown below.

# docker exec blockbridge-volume-driver profile ls
[
{
"name": "profile",
"user": "block",
"capacity": "32GiB",
"type": "autovol",
"attributes": "+ssd"
},
{
"name": "default",
"user": "block",
"capacity": "32GiB",
"type": "autovol",
"attributes": "-production +rack42"
},
]

Viewing Profile Information

To retrieve information on a specific storage profile, use the profile info command implemented by the volume driver command line interface, as shown below.

# docker exec blockbridge-volume-driver profile info --name default
[
{
"name": "default",
"user": "block",
"capacity": "32GiB",
"type": "autovol",
"attributes": "-production"
},
]

Removing a Storage Profile

If a profile is no longer needed, use the profile remove command implemented by the volume driver command line interface, as shown below. Note that if a volume is created using a storage profile, it does not create a dependency on the profile’s existence; it will not prevent removal.

# docker exec blockbridge-volume-driver profile remove --name profile

Conclusion

The 3.0 release of the Blockbridge Volume Driver drastically simplifies persistent storage management for Docker. It improves on our previous management capabilities by leveraging more of our core infrastructure APIs. We have reworked our volume driver to provide both a command-line utility and RESTful API to enhance visibility and storage management while improving the overall stability of the solution.

A key feature in this release is storage profiles. Storage profiles extend service-oriented storage concepts to Docker, better address the semantics of implicit volume creation, and further reduce the complexity of multi-host workflows. From an implementation perspective, storage profiles represent an architectural shift away from distributed persistent state (the old description files), toward the use of globally managed configuration objects via API.

Enjoy!