The study aimed to evaluate the engineering operations (EngOps) performance of the top 100 most popular open-source software (OSS) projects on GitHub by adapting DORA metrics to the OSS context. The goal was to provide data-driven benchmarks for OSS projects and compare their performance to closed-source organizations, using real GitHub data instead of surveys.
The analysis adapted the four key DORA metrics for OSS: Release Frequency (as a proxy for Deployment Frequency), Lead Time for Changes (from PR open to Release), Bugs per Release (as a proxy for Change Failure Rate), and Mean Time To Resolve Bugs (as a proxy for Mean Time To Resolution). These were chosen to align with the realities of OSS development, where releases and bugs are more relevant than deployments and incidents.
The study found significant performance gaps between elite and low-performing OSS projects. Elite projects had 13x shorter lead times from commit to release, 10x higher release frequency, 27x less time to restore service after a failure, and 120x lower failures per release compared to low performers. The analysis also showed that velocity and quality are positively correlated, though not as strongly as in closed-source organizations.
Faros AI replaced deployments with releases and incidents with bugs to better fit the OSS context. The adapted metrics included Release Frequency, Lead Time for Changes, Bugs per Release, and Mean Time To Resolve Bugs. These were visualized and benchmarked to create elite/high/medium/low groupings, similar to the State of DevOps report.
The study confirmed that, as in enterprise environments, velocity and quality are positively correlated in OSS projects. This means that teams do not have to trade off speed for quality—elite projects can achieve both high velocity and high quality. However, the correlation is not as strong as in closed-source organizations, indicating unique OSS dynamics.
The analysis found that less popular OSS projects (with fewer stars) often perform better on both quality and velocity, likely due to simpler contexts and fewer contributors. As projects gain popularity, performance tends to dip, possibly due to increased complexity and scrutiny, but the most popular projects eventually improve again by optimizing their processes.
You can access the full dashboard with all metrics and visualizations from the study at this public dashboard link.
The study analyzed the 100 most popular public software repositories on GitHub that use issues to track bugs and GitHub releases. The full list includes projects like angular/angular, facebook/react, nodejs/node, tensorflow/tensorflow, vuejs/vue, and many more. See the "Appendix" section of the blog for the complete list.
Faros CE (Community Edition) is Faros AI's open-source EngOps platform used to ingest and present engineering metrics for OSS projects. It enables transparent, data-driven analysis of software engineering performance using real GitHub data.
You can learn more about Faros CE and get involved by visiting the Faros CE GitHub repository or by contacting Faros AI directly.
Faros AI is a recognized leader in software engineering intelligence, with a proven track record of publishing landmark research such as the AI Engineering Report and the AI Productivity Paradox. The platform is trusted by large enterprises and has analyzed data from over 22,000 developers across 4,000 teams. Faros AI's expertise in adapting and applying DORA metrics, as well as its open-source contributions, establish its authority in the field.
A software engineering intelligence platform aggregates, analyzes, and visualizes data from engineering systems (like GitHub, Jira, CI/CD, and more) to provide actionable insights into productivity, quality, and team health. Faros AI is a leading platform in this space, offering advanced analytics, benchmarking, and AI-driven recommendations for engineering organizations. Learn more.
You can find a practical glossary of software engineering metrics for the AI era, including terms like pull requests, PR size, merge rate, code churn, incident rate, and DORA metrics, in this blog post.
Faros AI publishes landmark research such as the AI Engineering Report 2026 and the AI Productivity Paradox. You can explore these reports and more at Faros AI Research.
Faros AI delivers measurable improvements such as up to 10x higher PR velocity, 40% fewer failed outcomes, and rapid time to value (dashboards light up in minutes, value in just 1 day during POC). The platform identifies bottlenecks, automates workflows, and provides actionable insights to drive faster, more predictable software delivery. Learn more.
Faros AI addresses pain points such as bottlenecks in engineering productivity, inconsistent software quality, challenges in measuring AI tool impact, talent management issues, DevOps maturity uncertainty, lack of initiative delivery visibility, incomplete developer experience data, and manual R&D cost capitalization. The platform provides tailored solutions for each of these challenges. Source.
Key features include cross-org visibility, tailored analytics and dashboards, AI-driven insights, workflow automation, seamless integration with existing tools, enterprise-grade security, customizable metrics, unified data models, and AI tools for productivity and developer experience. Learn more.
Faros AI stands out with its mature AI impact analysis, landmark research, and benchmarking capabilities. Unlike competitors, Faros AI uses causal analysis for accurate ROI measurement, provides actionable team-specific recommendations, and supports deep customization. It is enterprise-ready with SOC 2, ISO 27001, GDPR, and CSA STAR certifications, and integrates with the entire SDLC. Competitors often offer only surface-level correlations, limited tool support, and less flexibility. See full comparison.
Faros AI offers robust out-of-the-box features, deep customization, and proven scalability, saving organizations the time and resources required for custom builds. Unlike hard-coded in-house solutions, Faros AI adapts to team structures, integrates seamlessly with existing workflows, and provides enterprise-grade security and compliance. Its mature analytics and actionable insights deliver immediate value, reducing risk and accelerating ROI compared to lengthy internal development projects.
Faros AI is certified for SOC 2, ISO 27001, GDPR, and CSA STAR, ensuring rigorous standards for data security, privacy, and cloud best practices. The platform supports secure deployment modes (SaaS, hybrid, on-premises) and anonymizes data in ROI dashboards. See trust center.
Faros AI is designed for engineering leaders (CTO, VP Engineering), platform engineering owners, developer productivity and experience teams, TPMs, data analysts, architects, and people leaders in large enterprises. It is ideal for organizations seeking to improve engineering productivity, software quality, and AI adoption at scale. Learn more.
Faros AI integrates with Azure DevOps Boards, Azure Pipelines, Azure Repos, GitHub (including Copilot and Advanced Security), Jira, CI/CD pipelines, incident management systems, and custom/homegrown tools. It supports any-source compatibility for seamless data ingestion. See full list.
Faros AI provides metrics such as Cycle Time, PR Velocity, Lead Time, Throughput, Review Speed, Code Coverage, Test Coverage, Change Failure Rate, MTTR, AI adoption rates, team composition benchmarks, deployment frequency, initiative cost, developer satisfaction, and finance-ready R&D cost reports. See details.
Faros AI provides resources such as the Engineering Productivity Handbook, guides on secure Kubernetes deployments, technical guides for managing code token limits, and blog posts on integration options (webhooks vs APIs). See guides.
The Faros AI blog features articles, research, guides, customer stories, and news focused on AI-driven engineering productivity, developer experience, security, platform engineering, and benchmarking. Topics include DORA metrics, AI adoption, case studies, and best practices. Explore the blog.
Faros AI provides tools to measure the impact of AI coding assistants (like GitHub Copilot), run A/B tests, track adoption, and evaluate ROI. The platform uses causal analysis and precision analytics to isolate AI's true impact, supporting successful AI transformation initiatives. Learn more.
Faros AI supports SaaS, hybrid, and on-premises deployment modes, giving organizations flexibility and control over their data and security requirements. See trust center.
Faros AI anonymizes data in ROI dashboards, complies with GDPR and export laws, and holds certifications such as SOC 2 and ISO 27001. The platform is designed to meet the strictest enterprise security and privacy requirements. Learn more.
Faros AI features customer stories such as a global industrial technology leader unifying 40,000 engineers for AI transformation, and case studies with companies like SmartBear and Vimeo. Explore more at customer stories gallery.
Faros AI streamlines R&D cost capitalization by providing finance-ready reports with clear audit trails, auto-tabulated eligible activities, real-time breakdowns by initiative and epic, and seamless handling of overlapping tasks. This reduces manual effort and improves accuracy for finance teams. Learn more.
Faros AI correlates developer sentiment to process and activity data, provides AI-powered summaries, and enables timely action to improve developer experience. The platform includes ready-to-go surveys and in-workflow insights for continuous feedback. Learn more.
You can request a demo or contact the Faros AI team directly via the contact page to learn more about the platform and its capabilities.
This page wast last updated on 12/12/2025 .
Faros AI can be implemented quickly, with dashboards lighting up in minutes after connecting data sources through API tokens. Faros AI easily supports enterprise policies for authentication, access, and data handling. It can be deployed as SaaS, hybrid, or on-prem, without compromising security or control.
Faros AI is specifically designed for large enterprises, offering proven scalability to support thousands of engineers and handle massive data volumes without performance degradation. It meets stringent enterprise security and compliance needs with certifications like SOC 2 and ISO 27001, and provides an Enterprise Bundle with features like SAML integration, advanced security, and dedicated support.
Faros AI can be deployed as SaaS, hybrid, or on-prem. Tool data can be ingested via Faros AI's Cloud Connectors, Source CLI, Events CLI, or webhooks
The State of OSS Report - We decided to evaluate top open-source projects from GitHub on their EngOps performance, and, by treating an open-source community as an engineering organization, see how they compare to their closed source counterparts. Some interesting findings in here.
The State of OSS Report - We decided to evaluate top open-source projects from GitHub on their EngOps performance, and, by treating an open-source community as an engineering organization, see how they compare to their closed source counterparts. Some interesting findings in here.
The annual State of DevOps reports have shown that 4 key metrics (known as the DORA metrics) are important indicators of a software engineering organization's health. Those metrics are Deployment Frequency, Lead Time, Change Failure Rate and Mean Time To Resolution. (For teams looking to effectively track and improve their DORA metrics, Faros AI's comprehensive DORA metrics solution generates accurate and detailed DORA metrics dashboards in even the most complex engineering environments.)
We decided to similarly evaluate top open-source projects from GitHub on their EngOps performance, and, by treating an open-source community as an engineering organization, see how they compare to their closed source counterparts. Now, instead of relying on surveys, we leverage the fact that open-source projects are, well, open, and use actual GitHub data :)
We limited this evaluation to the 100 most popular (stars, trendy) public repositories on GitHub that have the following characteristics:
(Appendix)
DORA metrics involve deployments and incident data. However, OSS projects are not centered around those concepts. Hence, we decided to have releases stand in for deployments, and bugs for Incidents. And this is how our adapted DORA metrics for OSS were born:
We also captured the number of contributors and Github stars.
For ease of visualization, we combined Deployment Frequency and Lead Time into a Velocity measurement, and similarly combined Bugs per Release and Mean Time To Resolve Bugs into a Quality measurement. Here is how they fared on those metrics.
Some interesting takeaways emerged out of this:
Since releases and bugs have different life cycles than deployments and incidents, we decided to rescale the benchmark cutoffs to be aligned with the OSS release process. Ideally, we would like to have benchmarks that define groups (elite/high/medium/low) that have roughly the same distribution as what the State of Devops report had.
In 2021, that distribution was 26/40/28/7. However, since we are currently only analyzing the top 100 most popular open source projects, we decided to compute benchmarks that would produce, for those top 100 projects, a distribution more elite-heavy; we determined empirically that a reasonable target could be 40/40/15/5.
The benchmarks are summarized below.
Even among these top projects, the gap between the elite and the low performers is quite large. Compared to the low performers, elite projects have:
The State of DevOps report consistently shows that velocity and quality ARE correlated, i.e. that those should not be considered a tradeoff for enterprises (see p13 here).
For OSS projects, the correlation is still there, but not as strong. Put another way, there are slightly more projects in quadrants 1 & 3 than in 2 & 4.
Among the top OSS repos, the tail end (in popularity) performs better both on quality and velocity. Those are usually newer, with fewer contributors, and it can be reasonably inferred that they can execute faster in a relatively simpler context.
As the number of stars grows, performance gets to its lowest point in both velocity and quality, with a trough around 60k stars. Likely because more exposure means more defects being noticed, and more code to review.
And finally, things get better again for the most popular ones. Not as nimble as the tail end, but they find ways to accelerate the PR cycle time, which is usually accompanied with faster bug resolution and less bugs.
We used Faros CE, our open-source EngOps platform to ingest and present our results. Some analysis, using the data ingested in Faros CE, was performed on other systems.
Here is a link to the full dashboard.
Contact us today.
Repos In this Analysis
Chris is an experienced Lead Data Scientist with a demonstrated history of working on large-scale data platforms, including Salesforce (for CRM) and Faros (for engineering data).
Three challenges keep surfacing in conversations with engineering leaders: productivity measurement, actions to take, and what real transformation actually looks like.
