How Storage Software Layers Facilitate Cloud Native Development
This article is the first of a multipart series on cloud native storage. Check back on The New Stack later this week and next for future installments.
You need storage — every function needs to be able to access some kind of storage. The consequences of problems with either the storage itself or the ability of applications to access the storage are serious. “When apps fail, it’s often the storage causing problems,” explains Irshad Raihan, director of product marketing at Red Hat.
Regardless of where your application is being deployed, there are three major issues that can arise from storage problems. The most serious are data loss or data unavailability. Only slightly less serious are performance problems or inability to handle spikes in demand. In this article, we’re taking a look at how to best connect cloud native workflows to storage from the perspective of companies who provide cloud native (also called “container-native”) software-based storage, as well as Minio, an object storage server designed specifically for cloud infrastructure.
“My single most important takeaway is for developers to consider storage attributes up front,” says Alex Chircop, founder and chief technology officer of Storage OS. Whereas in the “old days” most developers depended on a storage admin to provide resources and had to build applications around the available storage, now developers have the freedom and responsibility to control the storage provisioning process — and to do so in minutes rather than weeks. There might not be one single best way to connect cloud native workflows to storage — unlike in the past, developers have the ability to provision storage based on the specific application needs, so each application can connect to data in an optimal way. We can, however, discuss the specific issues everyone should be aware of as they consider the best storage options for cloud native projects.
What Exactly Is Cloud Native? What Does it Mean for Storage?
The Cloud Native Computing Foundation defines cloud native systems as container packaged, dynamically managed and microservices oriented. Being “cloud native” ultimately has nothing to do with where the application is deployed — a monolith lifted-and-shifted to AWS will never be cloud native.
According to Chircop, storage in a cloud native environment should have essentially the same attributes as anything else that is part of the same environment. Gou Rao, co-founder and chief technology officer of Portworx, echoes the overall sentiment. “Customers care more that the software they’re buying is in line with the standards being set around the cloud native ecosystem,” Rao says.
Let’s look at how those three pillars of cloud native systems apply to the storage landscape — and then address portability and Day Two issues, both of which are key concerns when thinking about how to best connect your cloud native workflow to storage in real life.
Containers
“It’s not that cloud providers aren’t providing a reliable service, it’s that Kubernetes forces you to use those services in a way they weren’t designed for,” says Michael Ferranti, vice president of marketing at Portworx, about the challenges when connecting containers directly to storage options provided by the major cloud vendors.
Container-based environments are dynamic. As each container moves around the cluster, it needs to maintain its connection to the storage volume. If you are connecting to EBS directly, this requires an error-prone, manually-managed process of detaching and reattaching the volume to new hosts.
Kubernetes clusters are also generally deployed over several availability zones, sometimes even over multiple clouds. But cloud providers’ native storage options don’t allow volumes to attach to a host running across several availability zones. Unless you have a software layer over EBS, if a container fails over to the second availability zone it would not be able to connect with its data.
Microservices
Running application made up of hundreds or thousands of microservices, all or most of which need to be connected to storage, directly on EBS — or directly on your on-prem server — would not be practical because of the hard limits on the number of volumes you can attach to a Linux or Windows instance (40 for Linux, 17-26 for Windows). “If you want to run, say 100 pods, or 150 pods, or 250 pods on a VM and each of those pods needs a volume, you simply cannot do it using native EBS,” explains Ferranti.
Building a cloud native application requires densely packing your microservices — and their associated data. Doing so requires a software layer over the cloud providers’ native storage.
Dynamic Management
Removing opportunities for human error by handling as much as possible through APIs is a key part cloud native best practices. Not only should you be able to declare your storage needs through an intuitive UI, cloud native storage should require minimal human intervention to detach and reattach volumes as containers move in a cluster, to handle scaling, to adapt in case an application fails over to another availability zone and to handle storage degradation as the application runs.
Portability
“When you move from one cloud to another cloud the part that you realize change is required is actually storage not compute,” explains Anand Periasamy, CEO of Minio. “The APIs are not compatible,” Periasamy says that portability between public clouds and between public and private clouds is among the top concerns Minio hears about from clients — Portworx’s industry surveys have shown similar concerns about lock-in. Especially considering that running multicloud environments is common, your storage solution should allow data to move between public clouds and private clouds easily.
And On Day Two
As applications run, things go wrong: networks are partitioned, disks fail and containers move around. When thinking about storage, it’s important to consider how you will maintain performance as there is wear and tear on your storage system — and how as much as possible of the maintenance will be handled through automation.
“Just because you can connect a container to a storage system does not mean you get the operational characteristics that you want in a cloud native application,” stresses Ferranti. Getting those operational characteristics requires a software layer between your application and your storage resources. Think of this software layer as a way to translate between your containerized, API-driven, microservices-based application and the storage resources, which even in a cloud environment are connected to a machine and still have many of the same limitations as on-premise storage options.
Cloud native applications require storage that behaves in cloud native ways.
The Cloud Native Computing Foundation and Portworx are sponsors of The New Stack.
Feature image via Pixabay.
