OpenTelemetry Gaining Traction from Companies and Vendors
AMSTERDAM — At its heart, OpenTelemetry is about identifying problems in complex systems to create a better user experience. But to do that, it has to make sense of petabytes of data across thousands, even hundreds of thousands, of nodes.
Essentially, telemetry monitors three sources:
- Logs
- Metrics
- Traces (which monitor the pathways for interactions, such as end-to-end transactions and what happens between services)
Having competing standards for dealing with this information obviously becomes a hindrance to interpreting that information. And yet that’s historically what has happened, said Austin Parker, head of developer relations at Lightstep and a maintainer with OpenTelemetry, a CNCF incubating project.
“What we’re trying to do is bring all those people together, because all these projects have more in common than they have apart,” Parker told The New Stack. “Observability requires you as a developer or an operator to bring together all these different types of data and these different technologies and blend them to really get insights about what’s going on.”
Merged Standards Announced
That’s why it’s significant that one of the more popular standards — Elastic Common Schema, by the observability and security company Elastic — is merging into the OpenTelemetry Semantics Convictions. The Elastic Common Schema defines a common set of fields to be used when storing event data in Elasticsearch, such as logs and metrics. OpenTelemetry will create a dedicated working group to maintain the merged standard. The group announced the merger at Tuesday’s Observability Day, held in Amsterdam as part of the KubeCon+CloudNativeConEurope 2023 conference.
The result, wrote Microsoft’s OpenTelemetry project representative Reiley Yang, will be “more consistent signals across a different pillar of observability and security events.”
Adoption Growing for OpenTelemetry
OpenTelemetry is gaining traction across industries, with most companies now having some level of adoption, Parker said.
“We’re moving from smaller pre-production or internal workloads to go into full production, more important workloads and beyond those smaller Kubernetes clusters into bigger ones,” Parker said. “That’s because the project is still growing and changing pretty rapidly.”
Telemetry vendors are also joining the effort. Recently, observability company Sumo Logic announced it was “all in” on OpenTelemetry and on Tuesday Lightstep/ServiceNow said it will “deepen” its commitment to the OpenTelemetry project by “enabling customers to run non-vendor-specific components across environments when they adopt native OpenTelemetry tooling for complete telemetry portability.” Specifically, ServiceNow’s Lightstep team will commit to open source and upstream all OpenTelemetry enhancements generated from the Lightstep engineering team.
“At Lightstep, we’ve seen many organizations grapple with ‘cloud native sticker shock’ as they come to understand that these complex systems require sifting through massive amounts of data across architectures and proprietary solutions,” the company stated in its announcement. “In today’s macroeconomic environment, organizations are looking to reduce costs while driving innovation, especially when it comes to cloud native applications.”
Cutting Costs with OpenTelemetry
OpenTelemetry can help companies find those savings, which may account for its booming rise in popularity given the macroeconomic climate.
“What we’re seeing, especially in these very large orgs that are going through cloud transformations, is Kubernetes is kind of an unknown for them,” Parker said. “As they add more services, and they get more complex, it just gets more and more confusing, and their existing tools don’t really cover it.”
That’s where OpenTelemetry comes into play by providing a standard for collecting data, modeling data and bringing together different data sources.
“Observability isn’t a set of products. It’s this practice of how are you using all this data to understand your system, and to understand your system, you need more than just logs, you need more than just metrics, you need more than just traces,” Parker said. “We bring those together, and help you layer those different types of data together so that you can get better answers to your questions so you can really understand what’s going on.”
All of which can lead to a better end user experience because it provides insight into what matters, he added.
“By understanding, you can really focus on how is performance connected to my end user, how am I ensuring that my customers are having a great time using my application — that I’m actually making them happy versus having a bunch of disconnected performance metrics that don’t really tell me what’s going on,” Parker said.
CNCF paid travel and accommodations for The New Stack to attend the KubeCon+CloudNativeConEurope 2023 conference. Check back often this week for all things KubeCon+CloudNativeCon Europe 2023. The New Stack will be your eyes and ears on the ground in Amsterdam!
