On August 31, 2023, Ben Cochran, VP of Developer Enablement at Autodesk, sat down with Vitaly Gordon, Co-founder and CEO of Faros, at the San Francisco Engineering Leadership Community annual event for a conversation about developer productivity.

The interest in a developer productivity case study is high, given the potential impact such investments can have. In a recent article, McKinsey reported that its new approach to developer productivity has produced results like:

• 20 to 30 percent reduction in customer-reported product defects
• 20 percent improvement in employee experience scores
• 60-percentage-point improvement in customer satisfaction ratings

As Vitaly told the audience, the Autodesk team is one of the best developer productivity teams Faros has had the pleasure to work with. Keep reading to learn why Ben’s data-driven and outcome-based approach at Autodesk has been so effective.

Or watch the full video here.

‍
“If God Didn’t Make It, One of Our Customers Did”

Autodesk serves the market for design and make. Its technology spans architecture, engineering and construction, product design and manufacturing, and media and entertainment, empowering innovators everywhere to solve challenges big and small.

For more than forty years, Autodesk has helped customers to turn their designs into reality. As former Autodesk CEO, Carol Bartz, was fond of saying, “If god didn’t make it, one of our AutoCAD customers did.”

Ben Cochran has been an integral part of Autodesk for over two decades and has played a pivotal role in the company's transformation. “What customers do with our software is truly inspiring to me, and that's what keeps me here,” Ben said.

Ben's journey at Autodesk began as a developer on the AutoCAD team, where he contributed to the core of Autodesk's business at the time. His role then evolved into developing collaboration tools for sharing information, ultimately leading him to consider the critical aspects of workforce productivity.

Today, Ben leads the Developer Enablement group, reporting directly to Chief Technology Officer, Raji Arasu. His job is to “make sure that our growing [engineering] workforce becomes as productive as they can be while building solutions for our customers.”

A Platform Transformation Leads to New Productivity Challenges

Autodesk's decision to establish the Developer Enablement group and invest significantly in internal productivity was driven by the challenge of scale.

In the earlier years, Autodesk's market segments operated independently, which allowed individual engineering teams to manage their flagship products effectively. The scale at the time was manageable, and each team could inspect and improve productivity within their segment. They could address their bottlenecks, eliminate toil, and keep the teams focused on value-adding work.

Driven by the transformational emergence of digital devices, processes, and workflows in design and manufacturing, Autodesk embarked on a shift to a cloud-based systems architecture.

The shift itself introduced new software engineering challenges and dependencies at scale, where microservices have to be interoperable, secure, and compliant across different international and local standards. “If ten teams are doing well and only one team is doing poorly, you are only as good as your weakest link,” said Ben.

“I wish I could say we had the foresight to proactively focus on developer productivity,” he continued. “But in reality, we were more reactive to how we saw teams struggling with managing dependencies and doing things in too many different ways.”

As explained in Harvard Business Review, the most effective antidote to low productivity and inefficiency must be implemented at the system level, not the individual level. “We needed to take a step back and look at how we make sure our engineering teams are productive and building resilient, scalable, and sustainable software.”

The Decision to Create a Centralized Developer Enablement Team

As the challenges mounted, many teams began asking for resources to tackle their pain points in their narrow slice of the pie.

Recognizing the need for a centralized and scalable approach, Ben consulted the DORA (DevOps Research and Assessment) research for an external perspective on what it means to be productive and how to measure productivity.

DORA metrics, which include deployment frequency, mean time to recovery (MTTR), lead time, and change failure rate (CFR), became the foundation for Autodesk's productivity framework. DORA’s research showed that these metrics correlate best with desirable business outcomes.

Ben decided to adopt DORA metrics as the first set of consistently measured metrics across Autodesk’s engineering. The Developer Enablement group became accountable for delivering solutions that enable all teams across Autodesk to baseline their current state and become more productive along these key metrics — by building structures, platforms, and tools to enable them.

The Benefit of Baselining Developer Productivity

With Autodesk’s adoption of DORA Metrics, a common language for discussing productivity emerged. Teams were no longer left to solve problems in isolation, but rather part of a joint effort to invest in doing things better.

Tracking the same set of metrics consistently over time has another advantage: it enables learning and continuous improvement. “If something changes [in our productivity], it’s a great thing to be able to take a step back and ask why it happened and what can we learn from it,” said Ben. “We can take action to help the team be more successful.”

Autodesk's maturity and complexity, with products spanning cloud, desktop, and mobile platforms, presents some unique challenges. The company had traditionally focused on annual software releases tailored to specific customer usage patterns on desktop. However, new digital workflows powered by cloud-based engineering demand more frequent updates and greater flexibility.

Given the demand for “cloud speed”, regardless of whether the software is on a tablet, desktop, or in the cloud, every team is gaining experience in releasing faster and more frequently. Ben’s organization works with each team to meet them where they are and incrementally accelerate as it makes sense, from quarterly, to monthly, to more frequent releases.

Live Q&A: Driving Customer Impact and Managing the Cultural Shift

During the live Q&A, Ben had the opportunity to answer questions from the audience.

How do DORA metrics tie to business impact?

Ben was asked how engineering at Autodesk works with business and product stakeholders to measure complementary metrics that also connect the DORA metrics business impact and customer value.

According to Ben, “there's a magical thing that happens when you focus on outcomes versus work, which is that the team understands ‘the why’ — they understand the business objective they're trying to achieve.”

Excelling at the DORA metrics ensures Autodesk’s customers can do more amazing things, faster. The metrics help engineering teams identify areas where they need to invest, for example in tech debt, precisely in order to deliver more value for their customers.

How did engineers react to a new measurement framework?

When asked about the reaction teams had to being measured in this way, Ben explained his rollout strategy. He sought to connect with the teams’ inherent motivation to excel at engineering practices and the pride they take in delivering outcomes.

“When you’re asking people to do something different, it may at first feel like you’re making their job harder,” explained Ben. “As we rolled this out, we’ve emphasized the business outcomes we’re all aiming to achieve. We encourage teams to collaborate, learn from one another, and see the transition as a path toward improvement rather than a punitive measure.”

Vitaly added that the beauty of the DORA Metrics is that in order to game them, you do actually need to become more productive. As Ben explained, the commitment to achieving best-in-class DORA metrics helps dismantle some of the biggest obstacles to speed, like change review boards.

How do you find your blind spots?

One audience member asked how Autodesk uncovers their challenges and objectively prioritizes what to solve first, without letting the most vocal people dominate.

For Ben, the first step is objective analysis with tools. “You start to measure things and you start to look at data, which doesn’t take personality into play,” he explained.

But building relationships is just as important. “I make a point of meeting with the teams and investing in people. If there's a problem, most people want to do something about it.”

Autodesk’s Developer Productivity Case Study

Autodesk's developer productivity case study is a testament to the company's commitment to innovation and excellence. By addressing the challenges of scale, fostering cultural change, and focusing on outcomes, Autodesk has positioned itself to thrive in an ever-evolving technology landscape.

Their inspiring journey is an example of how a company can adapt and succeed in the face of rapid technological change.

To start your journey, contact the Faros.ai team.