Why Autodesk chose a platform approach to developer productivity and GenAI impact
Since the 1980s, Autodesk has been changing how the world is designed and made. Autodesk’s software is used to make greener buildings, electric cars, blockbuster movies, and more. Its software development team of thousands of engineers builds the technologies for designers and innovators to literally “make anything.”
In the last few years, Autodesk has been building a Design and Make platform for the industries it serves. This has required a massive shift in developer productivity and impact for its software development team.
Given the organization’s size and complexity, how did Autodesk equip teams to improve their speed, efficiency, and quality, and confidently adopt GenAI developer tooling? It built an internal developer platform with an integrated visibility plane, fueled with insights into how to optimize the software development lifecycle (SDLC).
Now the company is sharing its story and key learnings from adopting a platform engineering approach.
Background
A legacy of innovation facing rising demands and modern challenges
Founded in the early 1980s, Autodesk boasts an impressive legacy in innovative design software. Their flagship products are used globally in the architecture, engineering, construction, media and entertainment, and manufacturing industries. Over the past decade, this Design and Make industry has grown rapidly while simultaneously undergoing a massive digital transformation disruption. The changing landscape elicited a host of modern sustainability demands, which continue to push and redefine the boundaries of these industries.
In parallel, Autodesk continued to grow, as did its software development workforce. In earlier years, Autodesk built its success through individual development teams’ self-governance; each product team would measure and evaluate its own productivity metrics while addressing bottlenecks, eliminating toil, and maintaining focus on value-adding work.
Yet, as Autodesk began its platform journey, it experienced new software development productivity challenges from increasing dependencies at scale. To unravel the complexity, leadership adopted a new, centralized approach to developer services and productivity.
Complexity at scale and the need for data-driven insights
To meet the growing demands of the industries it serves, Autodesk is building a Design and Make platform with the aim to provide the highest standards of resiliency, reliability, scalability, and security to its customers. This entails connecting systems, tools, and technologies, building platform standards and capabilities, and defining paved paths for streamlined development.
Autodesk established an internal Developer Enablement group and heavily invested in developer productivity to facilitate this transformation. While examining the maturity and complexity of their operations and tech stack, the leadership realized that development teams would be unable to achieve their ambitious productivity goals without the use of insights. This recognition of “you can’t improve what you can’t measure” led them to evaluate how best to create data visibility for their teams.
This visibility would not come easy, given the sheer complexity and scale of the Autodesk tech stack. Autodesk teams run hundreds of thousands of builds per month that span thousands of configurations on a combination of loads, technologies, and tools.
Autodesk initially attempted in-house instrumentation of standard productivity metrics. They turned to Faros AI, a software engineering intelligence platform, because it offered the flexibility to integrate data from many tools and the ability for development teams to parse and scope the metrics in many ways.
Solution
A visibility plane within Autodesk’s internal developer platform
To democratize data access, Autodesk’s Internal Developer Platform (IDP) was provisioned with a visibility plane where Faros AI feeds the data insights from some of the key SDLC tools. Autodesk aims to use the Faros AI platform beyond simply tracking metrics to enable teams to drill down into specifics and identify bottlenecks, based on which each team can prioritize improvements that are most impactful for them.
Tracking DORA metrics and identifying meaningful leading indicators impacting business outcomes
When selecting the gold standard metrics for Autodesk, the team consulted the DORA (DevOps Research and Assessment) research from Google for an external perspective on what it means to be productive and how to measure productivity.
DORA metrics, which include deployment frequency, mean time to recovery (MTTR), lead time, and change failure rate (CFR), became the foundation for Autodesk's productivity framework. DORA’s research showed that these metrics correlate best with desirable business outcomes.
The Developer Enablement group is leading the delivery of solutions to enable teams across Autodesk to set their excellence standards and provide actionable insights to achieve them.
Beyond DORA metrics dashboards, Faros AI provides detailed insight into the contributing factors of each performance dimension. If a metric like lead time is too high, teams can see exactly why — for example, is it due to build time or code review time? This enables teams to autonomously improve their performance.
Tulika Garg, Director of Product Management for Developer Enablement and Ecosystem at Autodesk, says this visibility is crucial to help teams swiftly identify areas for improvement, make data-driven decisions, and deliver high-quality software faster.
In a talk at the 2024 Gartner® Application Innovation and Business Summit, Tulika shared a powerful example. Mean Time To Resolve (MTTR) measures how long it takes an organization to resolve an outage. Outages have a huge impact on customer loyalty, brand reputation, and profitability — especially for companies operating under strict SLAs. While certain incident management tools can measure MTTR, they do not answer the question of how to improve it. With Faros AI, development teams now have the insights to pinpoint sources of issues, whether in time-to-detect or rollback speed, and can prioritize improvements better.
Leveraging data insights to navigate the adoption of AI coding assistants
Autodesk has found that its platform approach to developer productivity insights has prepared it to be data-driven in adopting AI coding assistants like GitHub Copilot.
Leveraging Faros AI features like A/B testing and before and after metrics, Autodesk can confidently pilot and roll out the tool while keeping a close watch on adoption and usage, shifting bottlenecks, and unintended consequences. With Faros AI in place, Autodesk has holistic visibility into GitHub Copilot’s real impact on velocity, quality, and developer satisfaction, and has a framework in place for ROI analysis of any new AI-driven technology down the line.
Future-proofing engineering visibility with a platform approach
Autodesk’s platform approach to accelerating engineering productivity is helping the organization equip its development teams with the insights they need to achieve their excellence goals and be prepared to embrace new technologies like AI with confidence.
The company is eager to share several of its valuable learnings with peers dealing with similar challenges.
- Identify a pressing challenge for the organization. Before your organization can rationally evaluate potential solutions, you must thoroughly understand what problem or challenge you are trying to solve. For Autodesk, the challenge stemmed from increasing dependencies and engineering complexity and the need for a unified view of SDLC across teams. With the challenge identified, they were able to tailor an approach to fit their needs.
- Start with the teams’ needs and use cases. Once you’ve identified your solution, it can be tempting to jump right to integrating every single data source into the data insights platform. But that would have delayed addressing the teams’ most immediate requirements. In Autodesk’s case, they prioritized integration of data sources that could provide line of sight to the most pressing needs. Gradually, they expanded to other data sources and use cases.
- Small but clean data is better than large, unclean data. When deciding whether to place more emphasis on data quality over data quantity, Autodesk recommends going with quality. Start with relatively clean data sources that help establish the validity of your use cases. Along the way, you may identify data gaps or data hygiene issues, which you can add to the backlog. The success of your early MVP will create an appetite for more clean data, which, in turn, will motivate teams to address the data hygiene issues.
- Your biggest challenge is building a data-driven mindset. The foundational piece of the entire transformation is the decision to embrace a data-driven mindset. Organizations cannot improve what they cannot measure. Therefore, collecting, measuring, and analyzing data is the only way to improve your company’s operations in ways that align with your business goals and desired outcomes.
Looking ahead
Autodesk's platform approach to developer productivity exemplifies the power of innovation and transformation fueled by data-driven insights. With its Internal Developer Platform and integrated visibility plane, Autodesk is establishing a robust strategy for actionable insights for its development teams. The organization draws inspiration and best practices from leading industry frameworks while incorporating the needs of teams and internal stakeholders.
To promote developer productivity and well-being, Autodesk is pushing the boundaries of innovation while simultaneously enhancing its platform tooling and infrastructure. Fueled with actionable insights from Faros AI, Autodesk is cultivating an environment where engineering productivity, agility, and satisfaction will reach new heights as they continue to build world-class solutions for their customers.
