With ‘first-mile observability,’ Calyptia helps companies collect and analyze system data at source

Let the OSS Enterprise newsletter guide your open source journey! Sign up here.

“Observability” describes the ability to measure the internal state of a software system by analyzing the raw outputs. The resulting metrics, logs, and traces help developers and engineers understand what may be impacting an application’s performance, through analyzing data such as response time, memory consumption, uptime, and more.

While there are countless tools designed to iron out the kinks before a software update is pushed into the wild, observability is all about the production stage — that is, software that is already in the public domain. Thus, it’s important to get insights as early as possible, as even a slight system lag could translate into increased churn and lost revenue — this is where so-called “first-mile observability” enters the fray.

First-mile observability is concerned with deriving immediate insights from observability data at the source of where the data is created, rather than having to pool it first in a centralized “last-mile” conduit which takes more time and consumes more resources. But more than that, first-mile observability acknowledges the more modern distributed IT stack, with IoT and edge applications spread across disparate public and private infrastructure.

“First-mile observability is about developers and practitioners being able to get insights about their systems as soon as possible,” explained Anurag Gupta, CEO and cofounder of Calyptia, an enterprise-focused first-mile observability platform. “The quicker an organization can diagnose, troubleshoot, and respond to any issue, the better their systems will perform — and the business as a whole will benefit.”

That’s not to say that first-mile observability tools are looking to supplant more traditional observability products from long-established incumbents such as New Relic, Datadog, and Splunk — it’s more about complementing them with speedier insights.

Founded in 2020, Calyptia is the handiwork of the creators and maintainers of the open source projects Fluent Bit and Fluentd, which are touted as “fast, lightweight, and highly scalable” log and metrics data collectors that support widely-used cloud and container technologies such as Kubernetes and Red Hat OpenShift. Both Fluent Bit and Fluentd are now hosted by the Cloud Native Computing Foundation (CNCF).

To help power its mission to usher “immediate systems insight” deeper into the enterprise, Calyptia this week announced that it has raised $5 million in a seed round of funding led by Sierra Ventures and Carbide Ventures.

Faster observability

Calyptia essentially builds on top of Fluent Bit and Fluentd, offering a hosted cloud solution that collects and aggregates logs and metrics closer to their source for quicker observability and security analysis — it is deployed where the data is first generated, be that a Kubernetes node, a virtual machine (VM), a bare-metal server, or an IoT device. From there, Calyptia can parse the data in real-time, and also route it on to other cloud services or on-premise databases for longer-term data analysis.

Elsewhere, Calyptia Fluent Bit Enterprise — which recently entered public beta — is pitched as its flagship product, one that enables companies to deploy first-mile observability smarts inside their own environment “…to ease operations, decrease observability costs, and manage observability at scale,” according to Gupta. “Fluent Bit is the engine for first-mile observability, and Calyptia Enterprise for Fluent Bit ensures that any organization can realize all the benefits.”

But how does Calyptia save on observability costs, exactly? Well, a lot of it comes down reducing back-end costs and so-called “egress” fees, which are often “hidden” fees cloud providers charge whenever a company wants to lift data from one place and move it elsewhere. By analyzing data at the source, this saves on at least some of these costs. Moreover, users can minimize the amount of raw or “poorly-filtered” data they channel into pricey backend storage services — with Calyptia, they can choose which backends are best suited for a particular log data stream, and even discard “bad” or irrelevant data before it consumes too many backend resources.

“As you can route to multiple backends, users can choose the right cost / value for their observability data,” Gupta explained. “Additionally, we’ve had users simply filter out logs they know are not valuable — for example, we’ve seen users save 78% on data egress and backend costs when users exclude noisy notice-level logs from firewalls.”

The open source factor

As with just about every other commercial company built on an open source foundation, one of the core selling points of Calyptia is that it’s vendor agnostic and will generally play nice with whatever other technologies a company has in place. And given that Fluent Bit and Fluentd are governed by the CNCF, this gives (prospective) customers peace-of-mind from a licensing perspective.

“Users who have multiple backends or tools do not have to worry that their data might be favored to a single endpoint over another,” Gupta said. “Additionally, being part of the CNCF means that users can adopt and embed this technology within their own solutions without having to worry about proprietary licensing or any potential licensing changes.”

In terms of customers and revenues, Calyptia is keeping its cards fairly close to its chest, but it did say that it has a “growing number of enterprise customers from a diverse set of verticals,” spanning cloud providers, financial services, banking, messaging services, and more.

Moreover, Gupta said that they’re seeing some interest from organizations that have previously used the open source Fluent Bit project, and who are now looking to scale things with the enterprise edition.

“What we’ve done with Calyptia Enterprise is take the most powerful open source tool for first-mile observability — Fluent Bit — and make it fully accessible for the enterprise,” Gupta said. “The quicker an organization can diagnose, troubleshoot, and respond, the better their systems will perform — and the business as a whole will benefit.”