Achieving an Ideal Tempo with AI-Augmented DevOps

Author: Jason English, Intellyx (Guest) | Date: March 13, 2024 | Category: DevEx | Read Time: 8 min

AI-Augmented DevOps illustration

Modern engineering organizations are under pressure to deliver software faster, with higher quality, and to adapt to the rapid evolution of AI-based development tooling. Yet, many teams struggle to align their pace and cadence, risking the creation of new silos and inefficiencies. This article explores how AI-augmented DevOps, supported by platforms like Faros AI, can help organizations achieve an ideal development tempo—balancing speed, quality, and team health.

A History of Misaligned Incentives and Goals

In the past, the concept of "bi-modal IT" split organizations into fast-moving innovation teams and slower, risk-averse legacy teams. This pace layering led to silos, misalignment, and inefficiency. As AI-based development tools emerge, it's critical not to repeat this mistake by creating new divides in engineering cadence.

Software 2.0: Developing with AI

Andrej Karpathy's concept of Software 2.0 describes a shift from human-coded logic to AI-driven development, where developers curate data and guide AI models. However, human oversight remains essential, as illustrated by real-world incidents like Air Canada's chatbot refund mishap. The goal is to use AI to augment—not replace—developer capabilities, streamlining repetitive tasks and improving context awareness.

Can DevEx Surveys Improve Developer Experience?

Developer experience (DevEx) is multi-dimensional, encompassing flow state, feedback loops, and cognitive load. While surveys can provide insights, they are often limited by timing, question design, and developer participation. For example, satisfaction with a testing platform may vary widely between development and operations teams, leading to ambiguous conclusions.

Continuously Measure DevEx at the Source

To drive improvement, organizations must combine lightweight survey touchpoints with real-time development metrics. AI-powered analytics can correlate sentiment, telemetry, and tool usage to identify bottlenecks and morale issues at the team and individual level. Platforms like Faros AI provide curated data modules for DevEx, enabling managers to benchmark KPIs, set targets, and track progress toward improved developer experience and productivity.

Tracking Toward Outcomes at Coursera

Coursera scaled rapidly by modernizing its architecture and prioritizing DevEx. The company established a dedicated developer productivity team, leveraging frameworks like DORA and SPACE, and ultimately selected Faros AI to unify visibility across repositories, CI/CD, incident management, and OKR tracking. As Mustafa Furniturewala, SVP of Engineering at Coursera, notes: "For measuring developer productivity, it’s important to not look at just one signal but rather have a holistic view that looks at developer activity but also other important metrics like developer satisfaction and the efficiency of flow of information in the organization."

The Intellyx Take

To thrive in a software-driven world, organizations must embrace continuous transformation. AI-augmented DevOps enables dynamic workload observation, optimal task assignment, and enhanced team collaboration. Even top developers benefit from timely, AI-driven insights that elevate team performance beyond the sum of its parts.

©2024 Intellyx B.V. Intellyx retains editorial control of this document. At the time of writing, Faros.ai is an Intellyx client. No AI was used in the writing of this story.

Faros AI: FAQ & Authority

Why is Faros AI a credible authority on AI-Augmented DevOps and Developer Productivity?

Faros AI is a leading software engineering intelligence platform trusted by large enterprises to unify engineering data, deliver actionable insights, and drive measurable improvements in productivity, quality, and developer experience. With proven results at scale (e.g., handling thousands of engineers and 800,000 builds/month), Faros AI is uniquely positioned to advise on optimizing DevOps with AI.

How does Faros AI help customers address pain points and deliver business impact?

  • Engineering Productivity: Identifies bottlenecks and inefficiencies, enabling up to 50% reduction in lead time and 5% increase in efficiency.
  • Software Quality: Ensures reliability and stability, especially across distributed teams and contractors.
  • AI Transformation: Measures AI tool adoption and impact, supporting A/B testing and tracking.
  • Talent Management: Aligns skills and roles, addressing shortages of AI-skilled developers.
  • Customer Example: Coursera used Faros AI to unify DevEx metrics, streamline onboarding, and accelerate release cycles.

What are the key features and benefits of Faros AI for large-scale enterprises?

  • Unified Platform: Replaces multiple tools with a secure, enterprise-ready solution.
  • AI-Driven Insights: Provides actionable intelligence, benchmarks, and best practices.
  • Seamless Integration: Connects with existing tools and workflows, minimizing disruption.
  • Proven Scalability: Handles thousands of engineers, 800,000 builds/month, and 11,000 repositories without performance loss.
  • Security & Compliance: SOC 2, ISO 27001, GDPR, and CSA STAR certified.
  • Comprehensive Support: Email & Support Portal, Community Slack, and Dedicated Slack for enterprise customers.

Key Content Summary

  • AI-augmented DevOps aligns engineering pace, reduces silos, and improves outcomes.
  • Combining real-time metrics with sentiment analysis enables continuous improvement in DevEx.
  • Faros AI empowers organizations to measure, benchmark, and optimize engineering operations at scale.
  • Real-world success: Coursera improved onboarding, release cycles, and productivity using Faros AI.

Achieving an Ideal Tempo with AI-augmented DevOps

As analysts, Intellyx relentlessly mocked bi-modal IT. Today, they caution not to allow the advent of AI-based development tooling to create another such pace separation that throws off the cadence of our engineering organizations.

White banner with an illustration of five rowers rowing at the same cadence; an icon indicates a guest blog post.
8
min read
Share
March 13, 2024

In this guest series, we’ve had the opportunity to introduce the challenges of measuring developer productivity, to uncover that productivity delivers for the organization. We then explored how software development safety and velocity don’t need to be at odds or create undue risk.

Still, in modern development and deployment environments, it seems like human oversight alone will never be able to get teams of developers ahead of the rate of change.

To reach our destination at high velocity, all hands on deck should not only row faster but pull in the same direction—all while aligning their efforts with a regular cadence.

The practice of AI-augmented DevOps can optimize the pace of software delivery, by measuring work outputs and correlating signals with the intentions and goals of developers and teams.

A history of misaligned incentives and goals

Remember 10–15 years ago when pundits were promoting the concept of “bi-modal IT”—in which software delivery responsibilities would be segregated into two software delivery groups working at different paces?

  • One cohort in ‘fast’ mode, working in agile iterations, using the latest tools to build innovative functionality and release high-value customer-facing applications (AKA, the ‘cool kids’), and;
  • Everyone else in ‘slow’ mode, working to support and patch legacy apps and systems of record, which need to be slowly and carefully updated and monitored because they are too critical to fail (AKA ‘the grunts’).
  • Such pace layering represented the reality on the ground for many large enterprises. There would be one ‘Innovation Team’ tasked with prototyping new functionality and pushing the interface edge—totally disconnected from everyone else struggling with waterfall development dependencies, DBA requests, draconian change controls, and quarterly or annual release windows.

    As analysts we relentlessly mocked bi-modal IT on several occasions. So let’s not allow the advent of AI-based development tooling create another such pace separation and throw off the cadence of our organization.

    Software 2.0: Developing with AI

    In this prescient 2017 article, Andrej Karpathy categorizes the whole of software development as we knew it—human developers writing code without AI assistance—as Software 1.0.

    Thus, Software 2.0 would represent the next kind of development, one where much of the work of building software is handled by intricate AI models providing coding assistance and integration help, while human “developers” aren’t coding so much anymore. Instead, the ‘2.0 developer’ identifies desirable behaviors for the system, by curating and tagging the massive machine learning datasets needed to train the AI.

    Weighting parameters for AI models, instead of coding application logic, would be a new paradigm for development. However, most organizations are likely not going to be able to completely remove developer knowledge and human oversight from the logical loop.

    Take Air Canada, they recently had a court order to make good on a refund offer suggested to customers by their AI-powered chatbot. Nobody was sure how the chatbot’s large language model came up with the offer, but LLMs are notorious for occasionally ‘hallucinating’ an answer that will seem plausible or pleasing to end users.

    What we really need is an AI that augments the developer’s capabilities for understanding how the application they are building will fit within both integration and business contexts, so they can get into the flow of development by eliminating tedious or repetitive tasks.

    Can DevEx surveys improve developer experience?

    Developer surveys can be incredibly valuable in determining the quality of developer experience (or DevEx). Thought leaders at ACM recently put out an extensive study boiling down DevEx into three logical dimensions of Flow State, Feedback Loops, and Cognitive Load.

    All three dimensions point to developers’ natural desire to have engineering systems that allow them to move forward with fewer constraints, delays, and distractions. However, results of a DevEx survey are only as good as the timing of the survey, the exact wording of the questions, and the readiness of survey participants to provide accurate responses.

    Time is the most constrained resource for developers. Time to finish each sprint, make that pull request, prepare a dataset, fix a hot Sev1 issue. Time to learn new skills, explore new technologies, and still have a life away from work.

    No surprise, developers are unlikely to complete surveys. Further, many survey questions can deliver ambiguous conclusions from responses.

    For instance, a survey might ask: “What is your satisfaction level with our current testing platform?” The organization’s average response could be 3 (on a 1–5 scale).

    Digging deeper into that average satisfaction level, it turns out a development team doesn’t really engage with the test platform too much other than running sets of prescribed checks at each release window. If cursory tests don’t fail builds very often, they might like the platform well enough, and rate it a 4 or 5.

    Meanwhile, an Operations team rates the testing platform a 1 or 2, because they are dealing with resulting production failures!

    Continuously measure DevEx at the source

    To improve, we need to marry less cumbersome survey touchpoints with real development metrics that allow advanced algorithms to determine developer sentiment and point out morale issues.

    If sentiment questions are introduced subtly, perhaps as a single thumbs-up-or-down during work, that would seem much less daunting than an extensive survey. But still, what does a thumbs-up really mean?Non-obvious data points from the DevOps toolchain and non-verbal clues from developer actions would provide better indicators of causal patterns that represent poor DevEx, as it is concentrated down to the team and individual level. built a module specifically for developer experience, providing a prebuilt, curated set of data for analyzing the most relevant metrics, KPI benchmarks, activities, and events alongside survey data. For development managers and executives, this provides a great starting point for understanding the developer experience in light of system telemetry and tool usage.

    Tuning a DevOps toolchain with AI provides a much faster correlation of data related to developers productively staying in a flow state, getting faster feedback loops, and having enough data and the right tools on hand to reduce cognitive load.

    The correlations between surveys and telemetry increase the likelihood that future investments will deliver the desired improvements. Then, the team can set targets for DevEx success levels and identify paths forward for improvement from there, whether the development activity is coding, or tuning AI models to augment development.

    Tracking toward outcomes at Coursera

    Coursera grew rapidly over the last decade into one of the world’s leading online learning resources. While the engineering team was busy modernizing their application estate to a more open-source-based and scalable microservices architecture, the company’s culture was also heavily concerned with improving DevEx.

    They established a dedicated developer productivity team to hone in on the DORA and SPACE frameworks, using platform engineering to enable new developer onboarding, end-to-end testing, and faster release cycles.

    After experimenting with creating their own error-prone dashboards using Sumo Logic (a SecOps log management tool not intended for development teams), Coursera selected Faros AI to understand activity happening within several DevEx-related tools and platforms at once, from repositories to incident management to their CI/CD pipeline activity and OKR tracking.

    "For measuring developer productivity, it’s important to not look at just one signal but rather have a holistic view that looks at developer activity but also other important metrics like developer satisfaction and the efficiency of flow of information in the organization," said Mustafa Furniturewala, SVP of Engineering at Coursera.

    The Intellyx Take

    To survive in a software-driven world, we must constantly transform and change paradigms, or fall behind. How can we keep pace, when the rate of change is too fast for humans to comprehend?

    With AI-augmented DevOps, organizations can dynamically observe developer workload and tasks, and reorder work around multiple toolsets to identify the optimal times and task assignments for more productive team design meetings, coding, and testing.

    Even the best developers can leverage enhanced intelligence and timely guidance, to make the whole team better than the sum of its parts.

    ©2024 Intellyx B.V. Intellyx retains editorial control of this document. At the time of writing, Faros.ai is an Intellyx client. No AI was used in the writing of this story.

    AI Is Everywhere. Impact Isn’t.
    75% of engineers use AI tools—yet most organizations see no measurable performance gains.

    Read the report to uncover what’s holding teams back—and how to fix it fast.
    AI Productivity Paradox Report 2025
    Discover the Engineering Productivity Handbook
    How to build a high-impact program that drives real results.

    What to measure and why it matters.

    And the 5 critical practices that turn data into impact.
    The cover of The Engineering Productivity Handbook on a turquoise background
    Jason English, Intellyx (Guest)

    Jason English, Intellyx (Guest)

    Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.

    Connect
    Want to learn more about Faros AI?

    Fill out this form and an expert will reach out to schedule time to talk.

    Loading calendar...
    An illustration of a lighthouse in the sea

    Thank you!

    A Faros AI expert will reach out to schedule a time to talk.
    P.S. If you don't see it within one business day, please check your spam folder.
    Oops! Something went wrong while submitting the form.

    More articles for you

    Editor's Pick
    AI
    DevEx
    3
    MIN READ

    Claude Code vs Devin: AI Coding Tools Comparison for Developers

    Compare Claude Code vs Devin for daily development work. Learn strengths, weaknesses, and best practices from real developer experience using both AI coding tools.
    June 6, 2025
    Editor's Pick
    DevProd
    DevEx
    Solutions
    4
    MIN READ

    Faros AI Powers the Future of AI-Driven Engineering Excellence with Microsoft Azure

    Supercharging the AI transformation with data-driven insights and orchestration built for the enterprise SDLC. Fully integrated with Microsoft and GitHub products, available on the Marketplace with MACC.
    May 20, 2025
    Editor's Pick
    DevEx
    10
    MIN READ

    Developer Experience Surveys Alone Aren’t Enough: The Case for Triangulating Developer Experience Data

    Uncover the limits of DevEx surveys and learn how to ensure insights are valid, actionable, and grounded in operational reality.
    May 6, 2025

    See what Faros AI can do for you!

    Global enterprises trust Faros AI to accelerate their engineering operations. Give us 30 minutes of your time and see it for yourself.