Background
A legacy of innovation facing rising demands and modern challenges
Founded in the early 1980s, Autodesk boasts an impressive legacy in innovative design software. Their flagship products are used globally in the architecture, engineering, construction, media and entertainment, and manufacturing industries. Over the past decade, this Design and Make industry has grown rapidly while simultaneously undergoing a massive digital transformation disruption. The changing landscape elicited a host of modern sustainability demands, which continue to push and redefine the boundaries of these industries.
In parallel, Autodesk continued to grow, as did its software development workforce. In earlier years, Autodesk built its success through individual development teams’ self-governance; each product team would measure and evaluate its own productivity metrics while addressing bottlenecks, eliminating toil, and maintaining focus on value-adding work.
Yet, as Autodesk began its platform journey, it experienced new software development productivity challenges from increasing dependencies at scale. To unravel the complexity, leadership adopted a new, centralized approach to developer services and productivity.
Complexity at scale and the need for data-driven insights
To meet the growing demands of the industries it serves, Autodesk is building a Design and Make platform with the aim to provide the highest standards of resiliency, reliability, scalability, and security to its customers. This entails connecting systems, tools, and technologies, building platform standards and capabilities, and defining paved paths for streamlined development.
Autodesk established an internal Developer Enablement group and heavily invested in developer productivity to facilitate this transformation. While examining the maturity and complexity of their operations and tech stack, the leadership realized that development teams would be unable to achieve their ambitious productivity goals without the use of insights. This recognition of “you can’t improve what you can’t measure” led them to evaluate how best to create data visibility for their teams.
This visibility would not come easy, given the sheer complexity and scale of the Autodesk tech stack. Autodesk teams run hundreds of thousands of builds per month that span thousands of configurations on a combination of loads, technologies, and tools.
Autodesk initially attempted in-house instrumentation of standard productivity metrics. They turned to Faros AI, a software engineering intelligence platform, because it offered the flexibility to integrate data from many tools and the ability for development teams to parse and scope the metrics in many ways.
Solution
Tracking DORA metrics and identifying meaningful leading indicators impacting business outcomes
When selecting the gold standard metrics for Autodesk, the team consulted the DORA (DevOps Research and Assessment) research from Google for an external perspective on what it means to be productive and how to measure productivity.
DORA metrics, which include deployment frequency, mean time to recovery (MTTR), lead time, and change failure rate (CFR), became the foundation for Autodesk's productivity framework. DORA’s research showed that these metrics correlate best with desirable business outcomes.
The Developer Enablement group is leading the delivery of solutions to enable teams across Autodesk to set their excellence standards and provide actionable insights to achieve them.
Beyond DORA metrics dashboards, Faros AI provides detailed insight into the contributing factors of each performance dimension. If a metric like lead time is too high, teams can see exactly why — for example, is it due to build time or code review time? This enables teams to autonomously improve their performance.
Tulika Garg, Director of Product Management for Developer Enablement and Ecosystem at Autodesk, says this visibility is crucial to help teams swiftly identify areas for improvement, make data-driven decisions, and deliver high-quality software faster.
In a talk at the 2024 Gartner® Application Innovation and Business Summit, Tulika shared a powerful example. Mean Time To Resolve (MTTR) measures how long it takes an organization to resolve an outage. Outages have a huge impact on customer loyalty, brand reputation, and profitability — especially for companies operating under strict SLAs. While certain incident management tools can measure MTTR, they do not answer the question of how to improve it. With Faros AI, development teams now have the insights to pinpoint sources of issues, whether in time-to-detect or rollback speed, and can prioritize improvements better.
Leveraging data insights to navigate the adoption of AI coding assistants
Autodesk has found that its platform approach to developer productivity insights has prepared it to be data-driven in adopting AI coding assistants like GitHub Copilot.
Leveraging Faros AI features like A/B testing and before and after metrics, Autodesk can confidently pilot and roll out the tool while keeping a close watch on adoption and usage, shifting bottlenecks, and unintended consequences. With Faros AI in place, Autodesk has holistic visibility into GitHub Copilot’s real impact on velocity, quality, and developer satisfaction, and has a framework in place for ROI analysis of any new AI-driven technology down the line.
Looking ahead
Autodesk's platform approach to developer productivity exemplifies the power of innovation and transformation fueled by data-driven insights. With its Internal Developer Platform and integrated visibility plane, Autodesk is establishing a robust strategy for actionable insights for its development teams. The organization draws inspiration and best practices from leading industry frameworks while incorporating the needs of teams and internal stakeholders.
To promote developer productivity and well-being, Autodesk is pushing the boundaries of innovation while simultaneously enhancing its platform tooling and infrastructure. Fueled with actionable insights from Faros AI, Autodesk is cultivating an environment where engineering productivity, agility, and satisfaction will reach new heights as they continue to build world-class solutions for their customers.
