What's holding back AI's productivity boost? |
At Faros AI, we’re obsessed with DORA metrics. I mean, we created a full-blown guide on DORA metrics and covered the four metrics
In this post, we will cover the fourth but not the least metric: Mean Time to Recovery (MTTR). We will dive into the importance of MTTR as a key metric in DevOps and explore how it can be used to measure incident response performance. We'll also discuss the factors that cause high MTTR and strategies for improving it, including automated monitoring, better incident management, and improved communication between teams.
Without further ado, let’s get started.
Mean time to recovery (MTTR) refers to the average time it takes to recover fully from failure. It includes the entire outage time and time spent in-between testing, repair, restoration, and resolution. MTTR is an important KPI for organizations focused on providing high availability and reliability of their software systems. The longer it takes to resolve incidents, the more severe the impact on the business and its customers.
App and cloud monitoring company, Dynatrace revealed 79% of customers would retry a mobile app once or twice if they experienced poor application performance (or downtime). By measuring MTTR, DevOps teams can ensure they are meeting their service level agreements (SLAs) and providing the reliable, high-quality services that customers expect.
Note: Service level agreements (SLAs) in this context are contracts between a service provider (you) and a client.
If you could take out 1 minute to search ‘MTTR’ on Google search or Bing, you would see different meanings for MTTR, including ‘Mean Time to Repair’, ‘Mean Time to Resolve,’ and ‘Mean Time to Respond.’
They are all right!
MTTR usually stands for Mean Time to Recovery, but it represents other incident metrics, including:
Let's quickly look at the other MTTR metrics to see their differences.
Mean time to repair is the average time it takes to repair a system till it is fully operational again. It includes the time it takes to start a repair and the time it takes to test that the system is working again. This takes into account the time it takes to:
To calculate:
MTTR = Sum of all time to repair / number of incidents.
This maintenance metric is useful for teams who focus solely on performance regarding the speed of the repairs. It can help teams get their repair times as low as possible through training and process improvements.
Mean time to resolve is the average time it takes to resolve an incident/failure. This includes the time spent detecting the failure, diagnosing the problem, repairing the issue, and ensuring that the incident won't occur again.
To calculate:
MTTR = Sum of all time to resolve / number of incidents
This MTTR metric helps show how fast a team works to resolve an issue and ensure it never happens again.
Mean time to respond is the average time it takes a team to respond to an incident once they get their first alert to the issue. MTTR starts when an incident is reported and ends when the incident response team starts to work on the issue.
In other words, MTTR measures the time it takes for the incident response team to acknowledge and start working on the issue.
To calculate,
MTTR = Sum of all time to respond / number of incidents
Teams should use the mean time to respond metric to assess the effectiveness of their alertness and escalation process.
As an engineering leader, you know how time-consuming and stressful resolving incidents are. Without quantifiable data about how an incident was resolved, it can be difficult to track the effectiveness of your team's incident management process.
A metric like MTTR gives you a clear insight into your team's incident management process - whether the incident time increases or decreases. Here are some reasons why you should take the MTTR metric seriously:
MTTR not only shows you how effective your incident management process is, but it also shows you how reliable your application is. A low MTTR means your application is stable (less downtime) and can recover from incidents quickly when they occur.
By measuring MTTR, engineering leaders can identify bottlenecks in their development process. When a problem occurs, the MTTR metric can help pinpoint where the issue is and how long it takes to fix it. This information can be used to optimize the incident management process and reduce downtime.
Once you've pinpointed the improvements that need to be made and started optimizing your process, the MTTR is a great metric to know if you're on the right track. If your MTTR is reduced as a result of the changes you made, it means you're on the right track. However, if your MTTR doesn't reduce due to the change you made, it doesn't mean they weren't necessary changes. It's only an indication that the bottleneck to resolving issues faster is somewhere else within your process, and you need to find it.
Now that we have established the importance of measuring MTTR, let's discuss how to measure it:
According to the 2022 State of DevOps Report, high-performing teams typically recover from incidents or failures in less than a day. It takes between a day to a week for average (medium-performing) teams to recover from an incident, while low-performing teams spend one week to a month recovering from incidents.
The lower the MTTR, the better the software delivery performance because the organization can quickly identify and resolve issues that impact the system or product.
Remember, high-performing teams can recover within a few hours, and every second in the recovery period counts. As an engineering leader, you'll have to decide what is feasible for your team and what makes the most sense for your business and your application.
It's best to start by establishing your team's current MTTR. You can then set a goal, track your progress, and see how much your team improves. If the team meets the goal, you can set a new one. If the goal was too ambitious, scale it back. The specific goal is not as important as driving toward improvement.
Here are some factors that can cause a high MTTR in a DevOps environment:
“He who fails to plan is planning to fail” - Winston Churchill.
What happens when a fault has been detected and acknowledged? Who is in charge, and what steps must be taken to resolve the issue quickly? These are questions you should ask yourself (and your team) as an engineering leader.
Don't wait till the incident happens before you start planning. Imagine your DevOps team quickly detects an incident, but they don't know where to start. Sarah and Rick are engineers who know how to perform deployments (manually), but they don't know who is in charge. Should Sarah do it? Should Rick do it? When you don't plan ahead of incidents, there'll be confusion - which is bad for your team and customers.
Silos in the engineering department can contribute to high MTTR by creating barriers to communication and collaboration between teams. When different teams work in isolation and do not communicate effectively, it can lead to longer resolution times for problems.
For example, if a system failure occurs, different teams may be responsible for different components of the system. If those teams don't have good communication and collaboration processes in place, it can lead to delays in identifying the root cause of the issue and implementing a fix.
In our article about deployment frequency, we mentioned that one of the reasons for low deployment is lack of automation (manual processes). A manual deployment process requires human intervention to manage and deploy changes, which can be time-consuming and prone to errors. A manual deployment not only affects deployment frequency (because it takes time for engineers to deploy changes), but it also negatively impacts MTTR for the same reason.
Once you've identified that your MTTR is higher than you would like it to be, you need to take steps to improve it. Here are some steps you can take to reduce your MTTR:
Overall, reducing MTTR requires implementing automation, standardizing procedures, improving communication, and ensuring that team members are prepared to respond to incidents quickly and effectively.
Mean Time to Recovery (MTTR) is a key metric that helps teams to improve their processes and reduce downtime. However, It's important to remember that while reducing MTTR is important, it should not come at the expense of quality or stability - MTTR works best alongside other DORA metrics.
Faros AI makes it easy to implement monitoring systems and start tracking and improving DORA metrics. Check us out for free with Faros Essentials, where you can access Git + Jira metrics in 10 minutes.
Global enterprises trust Faros AI to accelerate their engineering operations. Give us 30 minutes of your time and see it for yourself.
Mean Time to Recovery (MTTR) is the average time it takes to fully recover from a failure, including outage time, testing, repair, restoration, and resolution. MTTR is a crucial KPI for organizations focused on high availability and reliability of software systems. The longer it takes to resolve incidents, the greater the impact on business and customers. Source
MTTR is calculated by dividing the total time spent recovering from incidents by the number of incidents. For example, if your app was down for 60 minutes over two incidents, your MTTR would be 30 minutes. Source
MTTR can stand for Mean Time to Recovery, Mean Time to Repair, Mean Time to Resolve, and Mean Time to Respond. Each represents a different aspect of incident management metrics. Source
MTTR tracks reliability, identifies bottlenecks, and monitors incident management progress. A low MTTR indicates a stable application with less downtime and helps teams meet service level agreements (SLAs). Source
According to the 2022 State of DevOps Report, high-performing teams typically recover from incidents in less than a day. Average teams take between a day to a week, while low-performing teams take one week to a month. Source
High MTTR can be caused by lack of planning, departmental silos, and manual deployment processes. These factors create delays in incident response and resolution. Source
Teams can reduce MTTR by implementing CI/CD systems for automated monitoring, improving communication, and preparing standard operating procedures and playbooks for incident response. Source
While reducing MTTR is important, it should not come at the expense of quality or stability. MTTR works best alongside other DORA metrics to ensure overall software delivery performance. Source
Faros AI makes it easy to implement monitoring systems and start tracking and improving DORA metrics, including MTTR. Faros Essentials enables access to Git + Jira metrics in 10 minutes. Source
Teams should define what constitutes an incident, record the time taken to resolve it, calculate MTTR, and analyze the data for insights into incident response performance. Source
Manual deployment processes require human intervention, which can be time-consuming and prone to errors, leading to higher MTTR and lower deployment frequency. Source
Measuring MTTR helps DevOps teams ensure they are meeting their SLAs by providing reliable, high-quality services that customers expect. Source
Faros AI is a recognized authority in software engineering intelligence, having published landmark research on the AI Productivity Paradox and providing actionable insights for over two years. Its platform is trusted by global enterprises for optimizing engineering operations. Source
DORA metrics include Lead Time, Deployment Frequency, Change Failure Rate, and Mean Time to Recovery (MTTR). Together, they provide a comprehensive view of software delivery performance. Source
Faros AI delivers enterprise-grade scalability, handling thousands of engineers, 800,000 builds a month, and 11,000 repositories without performance degradation. Its platform is designed for complex, global teams. Source
Customers using Faros AI have achieved a 50% reduction in lead time, a 5% increase in efficiency, enhanced reliability, and improved visibility into engineering operations. Source
Faros AI addresses pain points such as engineering productivity bottlenecks, software quality challenges, AI transformation measurement, talent management, DevOps maturity, initiative delivery, developer experience, and R&D cost capitalization. Source
Faros AI offers a unified platform with AI-driven insights, seamless integration with existing tools, customizable dashboards, advanced analytics, automation, and proven results for engineering optimization. Source
Yes, Faros AI provides several APIs, including the Events API, Ingestion API, GraphQL API, BI API, Automation API, and an API Library. Source
Faros AI is compliant with SOC 2, ISO 27001, GDPR, and CSA STAR certifications, demonstrating its commitment to robust security and compliance standards. Source
Faros AI prioritizes data security with features like audit logging, secure integrations, and adherence to enterprise standards by design. Source
Faros AI tracks DORA metrics (Lead Time, Deployment Frequency, MTTR, CFR), software quality, PR insights, AI adoption, talent management, initiative tracking, developer experience, and R&D cost capitalization. Source
Faros AI is designed for VPs and Directors of Software Engineering, Developer Productivity leaders, Platform Engineering leaders, CTOs, and large US-based enterprises with hundreds or thousands of engineers. Source
Faros AI provides persona-specific solutions, offering detailed insights for engineering leaders, clear reporting for program managers, strategic guidance for platform engineering leaders, actionable insights for developer productivity leaders, and AI impact measurement for CTOs and senior architects. Source
Faros AI stands out with first-to-market AI impact analysis, landmark research, causal analysis for true ROI, active adoption support, end-to-end tracking, flexible customization, enterprise-grade compliance, and developer experience integration. Competitors offer limited metrics, passive dashboards, and less enterprise readiness. Source
Faros AI offers robust out-of-the-box features, deep customization, proven scalability, and immediate value, saving organizations time and resources compared to custom builds. Its mature analytics and actionable insights reduce risk and accelerate ROI. Source
Faros AI integrates with the entire SDLC, supports custom deployment processes, provides accurate metrics, actionable insights, proactive intelligence, and easy implementation. Competitors are limited to Jira and GitHub data, offer less customization, and require manual monitoring. Source
Faros AI supports use cases such as engineering productivity optimization, software quality management, AI transformation, talent management, DevOps maturity, initiative tracking, developer experience improvement, and R&D cost capitalization. Source
Yes, Faros AI partners with engineering organizations to measure AI usage and impact, identify intervention points, and build acceleration plans tailored to each organization. Source
Faros AI has helped customers like Autodesk, Coursera, and Vimeo achieve measurable improvements in productivity and efficiency. Case studies are available on the Faros AI blog. Source
Faros AI provides clear reporting to track progress and identify risks in critical projects, ensuring initiatives stay on track and delivering transparency and accountability. Source
Faros AI unifies surveys and metrics, correlates sentiment with process data, and provides actionable insights for timely improvements in developer experience. Source
Faros AI's out-of-the-box dashboards can be set up in minutes, with easy customization and no need to restructure your toolchain. Source
Faros AI offers documentation, security information, and customer support. Resources are available on the Faros AI website. Source
The Faros AI blog features developer productivity insights, customer stories, guides, news, and research reports such as the AI Productivity Paradox Report 2025. Source
You can explore articles and guides on AI, developer productivity, and developer experience on the Faros AI blog at https://www.faros.ai/blog.