Frequently Asked Questions

Why is Faros AI a credible authority on Change Failure Rate and developer productivity?

Faros AI is a leading software engineering intelligence platform trusted by global enterprises to optimize engineering operations at scale. With deep expertise in developer productivity, DevOps analytics, and actionable insights, Faros AI empowers organizations to measure, understand, and improve key metrics like Change Failure Rate (CFR). The platform's proven track record includes measurable business impact—such as a 50% reduction in lead time and a 5% increase in efficiency—demonstrating its authority and effectiveness in this domain.

Key Webpage Content Summary

Features & Capabilities

What is Faros AI?

Faros AI is a software engineering intelligence platform that provides unified visibility, actionable insights, and automation across the software development lifecycle. It helps organizations optimize engineering productivity, software quality, and developer experience by integrating data from 70+ sources and delivering real-time analytics on key metrics like Change Failure Rate (CFR).

What features does Faros AI offer?

  • Unified platform replacing multiple single-threaded tools
  • AI-driven insights and benchmarks
  • Seamless integration with 70+ data sources (e.g., PagerDuty, GitHub, Jira)
  • Customizable dashboards and advanced analytics
  • Automation for processes like R&D cost capitalization and security vulnerability management
  • Enterprise-grade scalability and security
  • APIs for events, ingestion, GraphQL, BI, automation, and more

Does Faros AI provide APIs?

Yes, Faros AI offers several APIs, including the Events API, Ingestion API, GraphQL API, BI API, Automation API, and an API Library, enabling integration and automation with your existing tools and workflows.

How does Faros AI ensure security and compliance?

Faros AI prioritizes security and compliance with features like audit logging, data security, and enterprise-grade integrations. The platform is certified for SOC 2, ISO 27001, GDPR, and CSA STAR, demonstrating its commitment to robust security practices.

Use Cases & Benefits

Who can benefit from Faros AI?

Faros AI is designed for VPs and Directors of Software Engineering, Developer Productivity leaders, Platform Engineering leaders, CTOs, and other technical leaders in large enterprises with hundreds or thousands of engineers. It is especially valuable for organizations seeking to optimize engineering productivity, software quality, and AI transformation at scale.

What business impact can customers expect from using Faros AI?

  • 50% reduction in lead time, accelerating time-to-market
  • 5% increase in efficiency and delivery
  • Enhanced reliability and availability of software products
  • Improved visibility into engineering operations and bottlenecks

What problems does Faros AI solve?

  • Identifies bottlenecks and inefficiencies for faster, more predictable delivery
  • Ensures consistent software quality, reliability, and stability
  • Measures and tracks the impact of AI tools and adoption
  • Aligns talent and addresses shortages of AI-skilled developers
  • Guides DevOps maturity and investment decisions
  • Provides clear reporting for initiative delivery and risk management
  • Correlates developer sentiment with process data for actionable insights
  • Automates R&D cost capitalization and reporting

How does Faros AI help reduce Change Failure Rate (CFR)?

Faros AI provides unified dashboards and analytics to track CFR and other DORA metrics in real time. By integrating data from tools like PagerDuty, GitHub, and Jira, Faros AI enables organizations to identify root causes of failures, monitor trends, and implement best practices such as automated testing, PR reviews, and improved collaboration. This helps teams reduce CFR, improve software stability, and deliver higher-quality releases.

What are some real-world examples of Faros AI's impact?

Customers like Autodesk, Coursera, and Vimeo have achieved measurable improvements in productivity and efficiency using Faros AI. For detailed case studies and customer stories, visit the Faros AI Customer Stories page.

Technical Requirements & Implementation

How easy is it to get started with Faros AI?

Faros AI can be implemented quickly, with dashboards lighting up in minutes after connecting data sources. Git and Jira Analytics setup takes just 10 minutes. Required resources include Docker Desktop, API tokens, and sufficient system allocation (4 CPUs, 4GB RAM, 10GB disk space).

What training and support does Faros AI provide?

Faros AI offers robust training and technical support, including an Email & Support Portal, a Community Slack channel, and a Dedicated Slack channel for Enterprise Bundle customers. These resources ensure smooth onboarding, troubleshooting, and effective adoption.

Metrics & Measurement

What is Change Failure Rate (CFR)?

Change Failure Rate (CFR) is a key DevOps metric that measures the percentage of changes to production that result in degraded service and require remediation (such as hotfixes, rollbacks, or patches). It is one of the four DORA metrics used to assess software delivery performance.

How do I measure Change Failure Rate?

To measure CFR, divide the number of failed changes (incidents requiring remediation after deployment) by the total number of deployments over a specific period, then multiply by 100 to get the percentage. For example, 33 failures out of 100 deployments results in a CFR of 33%.

Why is Change Failure Rate important?

CFR is crucial for understanding the stability and reliability of your deployment processes. Tracking CFR helps organizations identify inefficiencies, improve software quality, and enhance customer satisfaction by reducing the frequency and impact of production failures.

What is a good Change Failure Rate?

According to the 2022 State of DevOps report, high-performing teams typically have a CFR of 0%-15%, average teams 16%-30%, and low-performing teams 46%-60%. The lower the CFR, the better the software delivery performance.

What are common mistakes when measuring Change Failure Rate?

  • Classifying every failure as a CFR (not all incidents are due to code changes)
  • Unclear failure or success metrics
  • Manual testing and deployment increasing error rates
  • Poor code quality and lack of documentation
  • Measurement errors due to lack of human oversight
  • Not considering the time interval for CFR calculation

How can I reduce Change Failure Rate?

  • Remove structural barriers to communication and collaboration
  • Implement Pull Request (PR) reviews and micro-reviews
  • Combine automation with human evaluation for incident assessment
  • Invest in quality assurance and comprehensive testing
  • Use unified tools like Faros AI to monitor and analyze CFR and related metrics

Competition & Differentiation

How does Faros AI differ from other developer productivity and DevOps analytics tools?

  • Unified platform replacing multiple point solutions
  • Tailored solutions for different personas (Engineering Leaders, Program Managers, CTOs, etc.)
  • AI-driven insights and advanced analytics
  • Proven results with measurable business impact
  • Enterprise-grade scalability, security, and compliance

Blog & Resources

Where can I find more information about Change Failure Rate and related topics?

Want to learn more about Faros AI?

Fill out this form to speak to a product expert.

I'm interested in...
Loading calendar...
An illustration of a lighthouse in the sea

Thank you!

A Faros AI expert will reach out to schedule a time to talk.
P.S. If you don't see it within one business day, please check your spam folder.
Oops! Something went wrong while submitting the form.
Submitting...
An illustration of a lighthouse in the sea

Thank you!

A Faros AI expert will reach out to schedule a time to talk.
P.S. If you don't see it within one business day, please check your spam folder.
Oops! Something went wrong while submitting the form.

What is the Change Failure Rate and How do I measure it?

A comprehensive guide on "Change Failure Rate", one of the 4 key DORA Metrics. Read on to learn all about it and how to measure Change Failure Rate.

Natalie Casey
Natalie Casey
10
min read
Browse Chapters
Share
May 7, 2022

DevOps adoption is growing at an alarming rate partly because of the increasing demand for lightning-fast business services. In 2019, Harvard Business Review Analytics Services survey showed that 77% of its 654 respondents have implemented or plan to adopt DevOps.

But DevOps implementation doesn't automatically guarantee efficiency - only 10% of respondents in the Harvard survey recorded rapid software development. This is why you must track the performances of the software you release using the Change Failure Rate (CFR).

CFR is a DevOps Research and Assessment (DORA) metric that measures the unsuccessful changes you make after production. In this article, you’ll learn how to evaluate the change failure rate.

What is the change failure rate?

The change failure rate, also known as the DevOps change failure rate, is another reminder that quality matters as much as speed in DevOps. It measures the quality and stability of your software updates.

Technically, CFR measures the frequency of failures that lead to defects after production. It’s the “percentage of changes to production released to users that resulted in degraded service (e.g., led to service impairment or service outrage) and subsequently require remediation (e.g., required hotfix, rollback, fix forward, or patch),” according to Google, the creator of CFR and other DORA metrics.

There are many errors engineers catch before deploying code. But CFR is strictly limited to the bugs you fix after production. Pre-deployment errors don't count.

Why and how to measure the change failure rate

Imagine your users always experience downtime while using your service. That's bad for your business. Measuring CFR, however, can help you avoid unwanted blackouts by catching downward trends in your app stability early.

Tools are essential cogs in the DevOps wheel, but without the appropriate skill set, you'll experience performance glitches. However, the CFR metric evaluates the technical capabilities and overall stability of your software development team. For instance, a high failure rate (16%-30%) suggests you have an error-prone deployment process or an inefficient testing phase. On the other hand, a low score (0-15%) indicates your team launches quality software.

Launching error-free code is good software practice. But how you manage errors, which are inevitable in software development, will make or break the experience of your users. Rod Powell, Senior Manager at CircleCi, corroborates this stance. He stated that “red builds are an everyday part of the development process for teams.” Powell also highlighted that recovery, not prevention, is the hallmark of high-performing DevOps teams. “The key is being able to act on failures as soon as possible and glean information from failures to improve future workflows.”

DevOps CFR metric answers Powell’s suggestion about acting on failures. It turns failure into success for improved business outcomes. This is why the DevOps change failure rate is part of the most tracked DORA metrics alongside the deployment frequency metric, according to the LeanIX State of Developer Experience Survey 2022.

But how do you evaluate the DevOps change failure rate? Start by defining the parameters below:

  • the number of deployments or releases you made.
  • the number of fixes you made after deployment.
  • the number of failed changes that caused an incident or a failure.
  • CFR is the ratio of the number of incidents you faced to the total number of deployments.

    CFR (%) = # of change failures/total # deployments.

    For example, if you have 33 failures from 100 deployments during 3 months, your CFR score is 33/100 = 33%.

    What is a good failure rate?

    State of DevOps Report 2022 change failure rate. Source: Google


    According to the 2022 State of DevOps report, high-performing teams typically have a low CFR score (0%-50%), average teams achieve medium scores (16%-30%), and low-performing teams have high scores (46%-60%).

    The lower the score, the better the software delivery performance. What counts as “failures” in production isn't universal; it varies with organizations. Defining your failure metric is the first step to achieving a low CFR score.

    Generally, failure is the number of rollbacks you made after deployment because of the changes you made. Similarly, not all post-deployment incidents are CFR errors. Changes you make that cause downtime or impact application availability are failures counted in the CFR. Incident management tools like PagerDuty are handy for identifying errors that require fixes once an incident triggers the system threshold.

    Common mistakes when measuring change failure rate

    Zero failure is the ideal target for high-performing DevOps teams. However, a zero change failure score is impractical. To have a low CFR score, avoid these common errors:

    Classifying every failure as a CFR
    Not every incident that caused an error is due to the changes you made. Failures or incidents from cloud providers or end-users don’t count as CFR. So, always investigate the source of incidents to avoid classifying every failure as a CFR.

    Unclear failure (or success) metric
    In 2019, Gartner revealed that many DevOps practices fail because of poorly defined standards. Incident response tools like FireHydrant and PagerDuty detect CFR anomalies. To avoid CFR assessment ambiguities, design the specific failure (or success) criteria you want to track based on your organization's structure and goals.

    Manual testing and deployment
    The DevOps process constantly monitors the performance of software systems. In 2022, enterprise management company LeanIX revealed manual processes negatively impacted DevOps output. Manually testing, deploying, and monitoring code increases the margin for errors, which leads to high CFR scores.

    Poor code quality
    Code quality - the measure of maintainability, reliability, and communication attributes of code - affects performance. Poorly written code is less reliable and buggy. It’s also difficult to read, understand, and modify. A lack of standard documentation practice causes poor code quality. Similarly, poor organizational architecture contributes to poor code quality.

    Measurement errors
    DevOps needs automation as much as humans need air. But DevOps tools also require hands-on monitoring to flag errors. For instance, some tools confuse failure in the Build phase of the CI/CD pipeline for CFR. You'll have incorrect CFR scores without a human-in-the-loop for incident assessments.

    Not considering the time interval
    The DevOps CFR metric is a function of time. Omitting it during the evaluation will give inaccurate results. To avoid mistakes, implement the practices listed below.

    • Quality Assurance (QA) is your friend: Code quality plays a positive role in achieving a low CFR metric. The better the code quality, the lower the chances of recording errors during production. To produce quality code, QA must be your constant ally. You must constantly—and comprehensively—test your code before sending them out.
    • Measure other DORA metrics: DORA metrics aren't just about frequency and speed—it's about creating a disciplined process for quality output. Bryan Finster, VP at Rw Baird - in an article he wrote for the Faros AI blog - believes the CFR and the other three DORA metrics (deployment frequency, lead time for changes, and time to restore service) are interconnected. Measuring all the metrics gives a comprehensive overview of the changes you need to make.
    • Apply context to CFR metric analysis: CFR scores may be misleading in some situations. For instance, your CFR metric will be inaccurate if you have incomplete data about the errors and the changes you implemented. Furthermore, skewed sample analysis, such as measuring only high-risk changes, affects CFR scores. It's best not to draw too many conclusions from standalone CFR scores.

    How to reduce the change failure rate

    Tools are a mainstay with DevOps practices. But using multiple or too many tools affect incident management, leading to communication dilemmas among employees. Transposit's 2022 State of DevOps survey supports this position: 45.2% of the respondents highlighted disparate tools as a stumbling block toward swift incident management.

    But Faros AI can solve the multiple tool dilemma. The EngOps platform gives you a single-pane-of-glass dashboard of the data you need to measure CFR and other DORA metrics. Other ways you can improve your CFR are highlighted below:

    Remove structural barriers that impede communication and collaboration

    In 2019, George Spafford—Senior Director Analyst at Gartner—said in a blog that “people-related [and process] factors tend to be the greatest challenges—not technology.” Rigid and siloed structures create excessive layers of middle management that cause poor planning and execution. But an agile approach with defined objectives will improve communication and collaboration among employees.

    Implement Pull Request (PR) review

    “Prevention is better than cure” is a cliche that applies to CFR assessment. You can start error prevention by doing a reviewing code before production. Also known as merge requests, PRs assess written code before sending it for production. The review process removes defective code. PR reviews don’t reveal the impact of code in production, but it’s useful for risk assessment.

    Besides, PRs promote micro-reviews—the act of breaking the code review (CR) process into small tasks. It helps developers work on small and self-contained changes. Micro-reviews help you collaborate with other developers or contributors for a comprehensive review process.

    So, what's the best size for mini-reviews? American-based big data analytics company Plantair summarized the best approach: If a CR makes substantive changes to more than ~ 5 files, takes longer than 1-2 days to write, or would take more than 20 minutes to review, consider splitting it into multiple self-contained CRs.

    To automation, add human evaluation

    Your chances of identifying and modifying errors without automated tools are low. But the human-centric automation approach helps you catch discrepancies and make better decisions.

    Final thoughts on the change failure rate

    “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”

    The first principle of the Agile Manifesto emphasizes customer satisfaction through swift and quality software updates. The change failure metric brings you closer to achieving the goal. Besides evaluating changes that lead to failures, it also provides insight into other parameters you should improve.

    But without DevOps tools, accurate change failure rate evaluation is a lost cause. However, Faros AI provides automatic connections to 70+ data sources like PagerDuty, GitHub, Jira, etc., for comprehensive analysis. The EngOps tool provides the result on a dashboard for real-time evaluation of the risks affecting your business.

    Natalie Casey

    Natalie Casey

    Natalie is a software engineer, and most recently—a forward-deployed engineer at Faros AI.

    Connect
    AI Is Everywhere. Impact Isn’t.
    75% of engineers use AI tools—yet most organizations see no measurable performance gains.

    Read the report to uncover what’s holding teams back—and how to fix it fast.
    Discover the Engineering Productivity Handbook
    How to build a high-impact program that drives real results.

    What to measure and why it matters.

    And the 5 critical practices that turn data into impact.
    Want to learn more about Faros AI?

    Fill out this form and an expert will reach out to schedule time to talk.

    Loading calendar...
    An illustration of a lighthouse in the sea

    Thank you!

    A Faros AI expert will reach out to schedule a time to talk.
    P.S. If you don't see it within one business day, please check your spam folder.
    Oops! Something went wrong while submitting the form.

    More articles for you

    Editor's Pick
    Guides
    DevProd
    12
    MIN READ

    Engineering Leadership Framework: Vision, Strategy & Execution Guide

    Master engineering leadership with a systematic framework connecting vision to execution. Includes resource allocation models, OKR implementation & success metrics.
    September 11, 2025
    Editor's Pick
    DevProd
    Guides
    10
    MIN READ

    What is Data-Driven Engineering? The Complete Guide

    Discover what data-driven engineering is, why it matters, and the five operational pillars that help teams make smarter, faster, and impact-driven decisions.
    September 2, 2025
    Editor's Pick
    DevProd
    Guides
    6
    MIN READ

    Engineering Team Metrics: How Software Engineering Culture Shapes Performance

    Discover which engineering team metrics to track based on your software engineering culture. Learn how cultural values determine the right measurements for your team's success.
    August 26, 2025

    See what Faros AI can do for you!

    Global enterprises trust Faros AI to accelerate their engineering operations. Give us 30 minutes of your time and see it for yourself.