Shocking Results (or not so...) from the State of DevOps 2022 Survey

The 2022 Accelerate State of DevOps Report by Google Cloud’s DevOps Research and Assessment team (DORA) came out just a few weeks ago and the results are honestly quite shocking (or maybe not so after all?) - let’s discuss.

Over the past eight years, more than 33,000 professionals around the world have taken part in the Accelerate State of DevOps survey, making it the largest and longest-running research of its kind. Year after year, Accelerate State of DevOps Reports provide data-driven industry insights that examine the capabilities and practices that drive software delivery, as well as operational and organizational performance.

About This Year’s Report

This year’s report was focused more around security, owing to the numerous data breaches that have come to light in recent years and malicious attacks increasing ever so frequently. However, the core focus around software delivery and operational performance is what we will be talking about here.

DevOps teams were classified using four key metrics: deployment frequency, lead time for changes, mean-time-to-restore, and change failure rate, as well as a fifth metric that was introduced last year, reliability.

Here is how teams were ranked on the 4 key metrics:

As shown in the percentage breakdowns in the table below, Elite performers are Simply Non-Existent, High performers are at a four-year low and Low performers rose dramatically from 7% in 2021 to 19% in 2022! - Shocking?

The Medium cluster grew notably to 69% of respondents. However, when you look at the data more carefully - you’ll see that there is a shift toward slightly higher software delivery performance overall.

Why are these results Not So Shocking After All?

You can blame the ongoing pandemic for starters. With remote work becoming a norm, teams no longer have the same efficiency that allowed many of them to score in the Elite category a few years ago. Teams' ability to share knowledge, collaborate, and innovate, are severely hampered today due to the lack of water-cooler conversations or face-to-face whiteboard sessions and that is directly contributing to a decrease in the number of High performers and an increase in the number of Low performers.

The more important reason however is lack of visibility into engineering operations. This problem gets exaggerated even more considering the new reality that we all live in today with “fully remote” or “hybrid” becoming the modus operandi going forward.

Even with all the data expertise that lives in engineering organizations, it is a sad reality that engineering teams have not been able to fully leverage all the data in a unified manner. Why is that so? Data is often scattered across disparate systems.

Engineering leaders are often forced to cobble data together in spreadsheets in order to perform meaningful analysis. Take Lead Time for Change as an example, one of the 4 DORA metrics that research suggests is meaningful to track for engineering organizations: not only do you need to ETL data from multiple systems (commits, pull requests, build, artifacts, deployments) to compute it, the collected data needs to link properly together. You need a robust data system to gracefully deal with missing data and out-of-order data ingestion. Most likely, you will also need to capture changesets for your deployments. A very tall order.

A better data-driven approach is a must if we want to move from gut-feeling and guesswork to intelligent actions that impact real business outcomes.

It is not All Doom and Gloom Though

At Faros AI, we put a lot of thought into making it super easy for engineering teams to connect up their individual data sources to our EngOps Platform. Faros then does the hard work of connecting the dots between the data sources automatically. Hooking up known vendors such as GitHub, BitBucket, Jira, Jenkins etc. to the Faros AI Platform is as simple as clicking a button on the UI; custom home-grown systems can also be easily integrated with the Faros SDK. Faros AI munges all the data, imputes changesets, correlates incidents with deployments, and so forth, to build a complete trace of every change from idea to production and beyond (and every stage in between). The result is a single-pane view of your entire software development lifecycle, including DORA metrics out of the box with no change in process.

If the out of the box modules don’t cover your organization’s needs, build your own custom charts and dashboards. From data ingestion to transformation to visualization, Faros AI is easy to integrate, API driven and extensible at every level.

Continuous improvement with data

With live DORA dashboards in place, engineering organizations can start to see where they stand relative to other engineering organizations, and what the scope for improvement is in their software delivery processes. The ability to slice and dice lead time or failure recovery time by application, team, and stage helps in identifying bottlenecks in processes — whether in code review, QA, build times, or triage. At the same time, trends over time enable organizations to assess the true impact of interventions — with data. More generally, engineering organizations can finally start to take a data-informed approach to improving the efficiency and effectiveness of their operations.DORA metrics is a good starting point for most organizations, however there are many other things engineering organizations need to instrument in order to truly become a data-driven organization.

As Mustafa Furniturewala, VP of Engineering at Coursera says:

“It’s important to not look at just one signal but rather have a holistic view that looks at developer activity but also other important metrics like developer satisfaction and the efficiency of flow of information in the organization. The DORA and SPACE frameworks are good starting points, but there are many other things that are important such as tracking the ratio of microservices to engineers, alerts to engineers, distribution of seniority across teams, and so forth to get a sense of how overwhelmed some teams might be.”

Read his entire post here on how Coursera is leveraging Faros AI to accelerate engineering operations and unlock developer productivity!  

See Faros AI in Action

Get Started for Free today and experience the magic of Faros AI first-hand.