ML observability platform WhyLabs raises $10M to monitor models and data in production

WhyLabs, a startup building what it calls “an interface between humans and AI applications,” last week announced that it raised $10 million in a series A funding round co-led by prolific data scientist Andrew Ng’s fund and Defy Partners, with participation from Madrona Venture Group and Bezos Expeditions. The company says that the capital will be used to further develop its platform as WhyLabs looks to grow both its workforce and customer base.

WhyLabs occupies a segment of the industry known as “MLOps,” a newer discipline involving collaboration between data scientists and IT professionals with the goal of productizing machine learning algorithms. The market for such solutions could grow from a nascent $350 million to $4 billion by 2025, according to Cognilytica. But certain nuances can make implementing MLOps a challenge.

WhyLabs was spun out of the Allen Institute for AI, a fundamental AI research institute in Seattle, Washington. Alessya Visnjic, who spent nine years at Amazon developing machine learning infrastructure, founded the company in 2019 with Andy Dang, Sam Gracie, and Maria Karaivanova. Dang worked on Amazon’s machine learning platforms, including Sagemaker, while Gracie was a principal user experience designer with Amazon’s machine learning group. Karaivanova, who’s also an investor, previously served in an executive role at Cloudflare.

“Software failures are an unavoidable fact of life in any modern enterprise. But the weird thing about AI failures specifically is that most issues originate in the data that the models consume,” Visnjic told VentureBeat via email. “It quickly became apparent to me that the kinds of tools people rely on in DevOps are not suitable for AI applications. AI needed its own tooling ecosystem.”

AI observability

WhyLabs is designed to enable AI practitioners to monitor the health of data and models in a platform-agnostic, decentralized way. Available as a self-service software-as-a-service offering since October, the platform provides tools for monitoring models and data streams in production for ranking, recommendations and personalization, document understanding, image understanding, forecasting, and fraud detection scenarios.

WhyLabs alerts data science teams of data quality issues, data drift, and other model behavior deviations. (In machine learning, “data drift” refers to changes in the statistical properties of what the model is trying to predict over time, which causes problems because the predictions become less accurate.) Once an alert is identified, the platform’s debugging features help with root-cause analysis of the issue, including remediation.

“With WhyLabs, machine learning and data teams are able to automate a significant portion of their day-to-day operations tasks and minimize the time to resolution of machine learning and data failures.  Ultimately, the benefit of using WhyLabs is that teams are able to focus on building more and better models, improving customer experience and business operations,” Visnjic said.

WhyLabs also offers an open source package for logging in machine learning applications, called Whylogs, which Visnjic claims has been downloaded over 100,000 times since its September 2020 launch. She added: “Industry thought leaders like Stitch Fix and Yahoo Japan collaborate with WhyLabs on building out Whylogs and using it to streamline machine learning logging and monitoring for their in-house machine learning platforms.

Competition

WhyLabs competes with a number of startups in the MLOps and data observability market, including Aporia, Monte Carlo, Cribl, Acceldata, and Bigeye. But the startup claims to have added two dozen new organizations to its client base since October, including brands in logistics, fintech, martech, retail, and health care.

If the digital transformation wave holds, WhyLabs will be well-positioned for growth in the coming months. Survey results point to the need for improved observability as companies adopt AI technologies. A recent report from DataIQ found that one-third of companies spent months getting models into production. Visibility into machine learning projects remains limited, with over 45% of companies saying they receive no updates or periodic updates. In another study, 47% of projects never get out of the testing phase. And of those that do, another 28% fail anyway.

“[We have] a mix of enterprise and self-service customers spanning from AI-first startups to Fortune 500 companies … Our goal is to equip every practitioner with AI observability tools and to monitor every production machine learning model,” Visnjic continued. “The product roadmap includes many exciting features based on customer demand, such as further enhancing the platform’s support for unstructured data use cases — specifically for image, audio and natural language processing.”

Eighteen-employee WhyLabs’ total raised stands at $14 million to date.