Evidently AI is a technology company that develops open-source tools for monitoring and diagnosing machine learning models in production. Founded in 2020 by Emeli Dral and Michael Tsanev, the company provides a suite of libraries and services that help data scientists and engineers detect data drift, model degradation, and other issues that can arise after deployment. The platform is widely used in industries such as finance, healthcare, and e-commerce to ensure the reliability and fairness of AI systems.
The core of Evidently AI's offering is its open-source Python library, which generates interactive reports and dashboards from model predictions and data. These reports cover a range of analyses, including data drift detection, model performance evaluation, and feature importance. The library integrates with popular machine learning frameworks and data processing tools, making it a practical choice for teams already using Machine learning workflows. In 2022, the company introduced a commercial platform that adds monitoring, alerting, and collaboration features on top of the open-source library.
History and Funding
Evidently AI was founded in 2020 by Emeli Dral and Michael Tsanev, both of whom had prior experience in data science and engineering. The company is headquartered in London, United Kingdom. In 2021, Evidently AI raised a $2.5 million seed round led by the venture capital firm Crane Venture Partners, with participation from other investors. This funding was used to expand the team and accelerate development of the commercial platform. By 2023, the company had grown to serve thousands of open-source users and dozens of enterprise customers.
Core Features
The Evidently AI platform offers several key capabilities for machine learning monitoring. Data drift detection is a primary feature, using statistical tests and distance metrics to compare current data distributions against a reference baseline. This helps identify when the input data changes over time, which can lead to model performance degradation. The platform also tracks model quality metrics such as accuracy, precision, recall, and F1 score, allowing teams to monitor performance on a rolling basis.
Another important feature is the ability to generate detailed reports that combine numerical metrics with visualizations. These reports can be exported as HTML or JSON files, making it easy to share findings with stakeholders. The open-source library supports both tabular data and text data, and it can be used with Large language model based systems for tasks like sentiment analysis or text classification. Additionally, the platform includes a dashboard that provides a real-time overview of model health, with alerts triggered when metrics exceed predefined thresholds.
Integration and Use Cases
Evidently AI is designed to integrate seamlessly into existing machine learning pipelines. It works with popular data processing and orchestration tools such as Apache Airflow, Prefect, and Dagster, as well as with cloud services like Amazon Web Services and Google Cloud. The library can be used in Jupyter notebooks for ad-hoc analysis or deployed as part of a continuous monitoring system. Common use cases include monitoring customer churn prediction models, fraud detection systems, and recommendation engines.
In practice, teams use Evidently AI to validate that their models remain accurate and unbiased over time. For example, a financial institution might use the platform to monitor a credit scoring model for drift in applicant demographics, ensuring that the model does not inadvertently discriminate against certain groups. Similarly, an e-commerce company could track changes in user behavior that might affect a product recommendation model.
Open Source and Community
The open-source nature of Evidently AI has contributed to its popularity. The library is available on GitHub under the Apache 2.0 license, and it has received contributions from a community of developers and data scientists. The project has gained significant traction, with thousands of stars on GitHub and adoption by companies worldwide. The company also maintains documentation and tutorials to help users get started, and it hosts community events such as webinars and meetups.
In addition to the core library, Evidently AI has developed integrations with other open-source projects, such as MLflow and Kubeflow, to facilitate model tracking and deployment. The company's commitment to open source is reflected in its business model, which offers a free tier for the open-source library and a paid tier for the commercial platform with advanced features like team collaboration and priority support.
Future Directions
As machine learning models become more complex and widely deployed, the need for robust monitoring solutions is expected to grow. Evidently AI is positioned to address this demand by expanding its capabilities to cover new types of models, including Deep learning and Transformer (architecture)-based architectures. The company is also exploring ways to incorporate explainability and fairness metrics into its platform, helping organizations build more trustworthy AI systems.
In 2023, Evidently AI announced a partnership with Alibaba Cloud to offer its monitoring solutions on the Alibaba Cloud marketplace, extending its reach to the Asia-Pacific region. The company continues to release regular updates to its open-source library, adding new features and improving performance. With the increasing emphasis on responsible AI, Evidently AI is likely to play a key role in helping organizations maintain the health of their machine learning systems.