Internal Developer Platforms Are for DevOps too
The rise of platform engineering is changing DevOps. DevOps have the task of creating internal developer portals to help others (mostly developers) consume services. But this isn’t the entire story.
What we’ve discovered is that DevOps themselves stand to benefit from internal developer portals, especially DevOps groups that need a strong software catalog. Even DevOps teams that start with developer self-service (the services developers consume) discover the strengths of the software catalog for DevOps purposes. Let’s explore this further.
Is DevOps in Trouble?
How do DevOps track the state of distributed applications and the systems they run on? How do they understand what’s going on with microservices? How do they know what has changed in an environment with many deployments each day, or get a sense of what it all costs?
You’re in trouble if all this information exists in a huge manual csv, if it is a git file with combined software and infrastructure data or if it resides in some super-engineer’s mind. You’re also in trouble if it takes too long to answer simple questions such as which Kubernetes clusters are running where, or which microservice version is currently running in production.
This lack of control is what drives DevOps and platform teams to adopt internal developer portals, with an initial focus on the software catalog. But aren’t internal developer portals supposed to be about developers — the developer experience, reusable and abstracted self-service elements for developers? They are. But one important use case is overlooked by platform engineering folks: internal developer portals for DevOps.
Wait, Isn’t This for Developers? The Case for DevOps Visibility
DevOps can benefit greatly from internal developer platforms because they too need a single place to access data about the software and infrastructure, from environments, deployments, regions and cloud resources to microservices. Despite owning many DevOps tools, they don’t necessarily have the capacity to bring all the data together, from the git provider, CI/CD, different cloud vendors and more.
DevOps need a single place to see changes and dependencies with all metadata on hand, as well as be able to tell what it all costs. We discovered this almost by chance, when showing Port’s platform-to-platform teams within DevOps organizations. Instead of looking to simplify developers’ lives, they focused on their own, first making DevOps’ lives better and only then looking to make the developer experience better.
Guardrails and Training Wheels
An industry veteran told me that the main reason why they developed an in-house developer portal was to ensure that whenever self-service for developers was enabled, the right golden paths were set, with guardrails (or training wheels, depending on the developer and team). They knew they couldn’t expect developers to write a Terraform file that meets their DevOps standards. This means that whatever the developer does — from microservice scaffolding to Day 2 actions or provisioning a temporary environment — they would need guardrails. This is provided through the internal developer portal with a product-like self-service interface that not only reduces the developer’s cognitive load, but makes life better for DevOps — fewer tickets and less mess.
“Developers were driving us crazy with requests,” he told me.
It’s interesting to note that if setting guardrails is what developer portals are all about, then scorecards (or what Spotify’s Backstage calls “soundcheck”) reflect post-facto whether those requirements were met. Scorecards are important, but covering all self-service actions and setting guardrails for each, in advance, is what really makes the platform engineering effort a success.
Making Sense of the Mess: The Software Catalog
It sounds odd, but the many tools that offer partial “catalogs” — think of a container orchestration tool — show data across one element, such as a cluster, and not many clusters, and sometimes several clusters are exactly what DevOps would like to see, since they manage more than one.
DevOps need access to the broadest possible software catalog, as we’ve defined here: “The software catalog is a visibility layer to the infrastructure and the software deployed over it. An ideal software catalog should show the entire ecosystem surrounding the SDLC [software development life cycle]: CI/CD flows, dev environments, data pipelines, deployments and anything cloud.”
We’ve seen cases where DevOps couldn’t see all AWS Lambdas across regions (AWS doesn’t offer a cross-region view). Sometimes DevOps need to see all deployments across all services that are running now. There’s no way to get this from GitHub and see all deployments for all services in different repos. This list goes on.
For instance, there is no one place to see all relevant data about all running services if they don’t all have the same runtime (for example, I cannot see all the running services that run on K8s and Lambda in the same place). This also happens where DevOps can’t see all Argo CD applications for the entire organization.
Similarly, DevOps also need to see the activity log that comes with an internal developer portal, so they can see who did what and when, especially as they made services consumable by developers, in case of issues or for audit purposes.
And the best part? The same software catalog will be used by developers, with some level of abstraction and with access control, so they won’t be overwhelmed by unnecessary data.
In short, the more complex the world becomes, the more DevOps need developer portals too.
Workflow Automation
In this case we’re talking about using developer portals as part of machine-driven workflows. The idea is really the same. It assumes that the internal developer portal is the single source of truth for the software and infrastructure in the organization, a real-time reflection of microservices and all the resources and infrastructure they run on.
In this case, DevOps uses the internal developer portal to inform workflows of locked services, fail builds in case a minimum level of quality isn’t met or terminate an ephemeral environment after a certain TTL. They can also use high cost as an input for termination.
Package Management
Microservice complexity is one of the drivers behind the recent rise of developer portals. Microservices are modular units of code designed for reuse by other software elements. Packages aren’t that different, and the associated security vulnerabilities are a huge headache. There are a lot of security solutions on the market for this problem, but internal developer portals seem to provide a superior solution since they provide package visibility, migration and dependency management, similar to what software catalogs offer for issues of drift.
There are additional security-related benefits to using internal developer portals. The fact that all self-service actions and DevOps actions are tracked in one central place, and that there is an activity log, makes it easier to pass SOC2 compliance. Additionally, should a security incident occur, solving it would become easier.
Of course, to do all this you need a good internal developer portal, with a robust software catalog. Try Port’s free version here.
