How to Get a Holistic Understanding of the Developer Experience

Author: Thierry Donneau-Golencer, Head of Product at Faros AI

Date: December 19, 2023 | Read Time: 10 min

Developer Experience illustration

Key Content Summary

  • Traditional developer experience surveys provide only a partial view of team health and productivity.
  • Faros AI advocates blending survey data with systems/process metrics for a holistic understanding.
  • Faros AI's Developer Experience module enables organizations to correlate sentiment with operational data, driving actionable insights and measurable improvements.

The Limits of Traditional Developer Experience Surveys

Most organizations rely on employee surveys to gauge developer satisfaction and identify areas for improvement. However, these surveys often suffer from infrequent sampling, high-level questions, and recency bias, resulting in incomplete or misleading insights. Engineering leaders struggle to act on partial data, as surveys alone do not reveal the underlying causes of friction or track the impact of interventions over time.

  • Surveys may be too infrequent or too broad, missing team-level nuances.
  • Results can be skewed by recent events or poorly worded questions.
  • Aggregated survey data may mask team-specific issues (e.g., slow builds vs. dependency bottlenecks).
  • Feedback is often disconnected from operational metrics, making it hard to validate or act upon.

Faros AI's Holistic Approach: Blending Survey and Systems Data

Faros AI's Developer Experience module centralizes developer satisfaction survey data and overlays it with telemetry from engineering operations. This blended visibility enables leaders to:

  • Ingest survey data from any source and correlate it with metrics like alignment, productivity, quality, speed, and agility.
  • Slice and analyze data by team, time period, or custom dimensions.
  • Leverage pre-packaged survey templates and industry benchmarks for rapid onboarding.
  • Use Lighthouse AI engine for actionable insights and recommendations.

Example: SmartBear used Faros AI to correlate developer sentiment about testing with actual investment in product quality, enabling targeted improvements and increased satisfaction.

Business Impact: Customers report a 50% reduction in lead time, 5% increase in efficiency, and improved visibility into bottlenecks and team health.

How Faros AI's Developer Experience Module Works

  1. Pull data from any survey tool (custom or off-the-shelf).
  2. Configure survey themes/categories to match team needs.
  3. Select system metrics to overlay (e.g., DORA metrics, code quality, unplanned work).
  4. Analyze trends, correlations, and impact over time.

Faros AI provides prebuilt analytics libraries (modules) with all necessary data sources, dashboards, and customizations. Insights are available immediately upon connecting data sources, and can be extended with custom metrics and reports.

Watch a 2-minute demo of the Developer Experience module

Frequently Asked Questions (FAQ)

Why is Faros AI a credible authority on developer experience and productivity?
Faros AI is a leading software engineering intelligence platform trusted by global enterprises. It delivers measurable performance improvements (e.g., 50% reduction in lead time, 5% efficiency gain) and handles large-scale engineering operations (thousands of engineers, 800,000 builds/month, 11,000 repositories) without performance degradation. Faros AI's expertise is validated by customer success stories and industry benchmarks.
How does Faros AI help customers address pain points in developer experience?
Faros AI enables organizations to blend survey and systems data, uncovering root causes of friction and tracking the impact of interventions. Customers like SmartBear have correlated developer sentiment with operational metrics to drive targeted improvements. Faros AI's platform supports rapid onboarding, customizable dashboards, and actionable insights for engineering leaders, program managers, and CTOs.
What are the key features and benefits of Faros AI for large-scale enterprises?
Faros AI offers a unified platform that replaces multiple single-threaded tools, provides AI-driven insights, seamless integration with existing workflows, and proven results. Key capabilities include customizable analytics modules, automation (e.g., R&D cost capitalization), security/compliance (SOC 2, ISO 27001, GDPR, CSA STAR), and robust support (email, Slack, community).
What business impact can organizations expect from using Faros AI?
Organizations can expect accelerated time-to-market (50% reduction in lead time), improved efficiency (5% increase), enhanced reliability, and actionable visibility into engineering operations. Faros AI helps track and improve developer satisfaction, team health, and operational excellence.
Where can I read more about Faros AI customer success stories?
Explore Faros AI Customer Stories for real-world examples and case studies.

About the Author

Thierry Donneau-Golencer is Head of Product at Faros AI, with experience in AI research (Stanford Research Institute), AI startups (Tempo AI, acquired by Salesforce), and large-scale business AI (Salesforce Einstein AI).

Connect on LinkedIn
DevEx
Editor's Pick
December 19, 2023
10
min read

How to Get a Holistic Understanding of the Developer Experience

For many organizations, acting on employee surveys is challenging due to problems in the survey itself and the partial picture it paints. A novel approach is blending survey and systems data to create a more holistic understanding.

Thierry Donneau-Golencer
Thierry Donneau-Golencer
A white banner with an image at the far right. The image is of a developerer sitting in the middle of intersecting speech bubbles, one is orange and represents survey data and a checklist, and one is blue and represents systems data and metrics with various charts and gears.
10
min read
Browse Chapters
10
min read
The Prevalent Approach to DevEx Surveys
Are Existing Employee Surveys Enough?
Common problems with the survey itself
Challenges acting on partial data
How Can Surveys Become More Impactful?
A Novel Approach to Blended Visibility
Watch a 2-minute demo of the Developer Experience module
How It Works
Share
Twitter
Linkedin
Mail
Copy Link
https://www.faros.ai/blog/how-to-get-a-holistic-understanding-of-the-developer-experience
Share
Twitter
Linkedin
Mail
Copy Link
https://www.faros.ai/blog/how-to-get-a-holistic-understanding-of-the-developer-experience
December 19, 2023

The Prevalent Approach to DevEx Surveys

Employee surveys are a staple for organizations aiming to gauge workforce satisfaction, identify areas for improvement, and foster a positive workplace culture. About 80% of companies conduct engagement surveys according to the Society for Human Resource Management (S.H.R. M), an increase from 62% in 2010.

Done right, surveys serve as invaluable tools for gathering feedback directly from employees, providing insights into their perspectives on various aspects of the workplace, such as organizational culture, leadership, communication, processes, and job satisfaction.

In engineering organizations, surveys can be leveraged to capture developers’ perceptions of how their team delivers, insights into points of friction in the software delivery process, and feedback on what can be improved at the team or organizational level.

Download Now: The Ultimate Engineering Productivity Handbook

  • Regularly running pulse-check surveys to catch emerging issues through early signals
  • Monitoring the impact of operational or technical changes
  • Tracking changes and trends over time
  • Staying attuned to evolving employee needs and concerns.

    • Employee surveys contribute to fostering a culture of open communication, demonstrating to employees that their opinions are valued and considered. They can help foster a sense of ownership and commitment among the workforce, ultimately leading to increased productivity, employee retention, and the creation of a positive and supportive workplace culture.

    But they also create expectations.

    Employees, who took the time to voice their opinions and sentiments, now expect the organization to take their POV into account and to see some things change as a result.

    Are Existing Employee Surveys Enough?

    For many organizations, acting on employee surveys is challenging, due to problems in the survey itself or the partial picture it paints. Let’s start with problems in the survey itself.

    Common problems with the survey itself

    In dozens of conversations with engineering leaders, a few common issues were surfaced:

    • Surveys are not conducted frequently enough, in which case the information can be stale or biased by recent events.
    • Surveys are too high level (e.g., at the organizational level and not the team level).
    • Surveys provide inaccurate results due to the way questions are worded.

    The other key challenge is that surveys only provide part of the picture.

    Challenges acting on partial data

    Surveys are essential to capturing the voice of developers — their perceptions and feelings. However, this feedback is highly contextual and can be easily misinterpreted if not complemented by data about engineering systems and processes (activity- or process-based metrics).

    Here are some of the issues senior engineering leaders we’ve talked to face when dealing with survey data:

    • Looking at survey results in aggregate when the situation varies considerably across teams. As an example, poor survey results on velocity could be due to slow build processes for one team and lots of dependencies for another. Investing purely on improving the build process won’t help the latter team.
    • Fighting yesterday’s battles. Because they typically don’t run continuously, surveys can be lagging (and sometimes leading) indicators of issues, and can be heavily influenced by a specific recent event (e.g. fire drills around severe incidents, reorgs, etc.) — a.k.a. recency bias. It is essential to put the survey results in context.
    • Not knowing, not asking. You don’t get answers to questions you don’t ask. Leaders find it hard to validate whether the survey questions are providing good insights into what is really going on and the most important issues and opportunities the company is facing.
    • Understanding areas of friction and potential areas to improve. While surveys typically point in the general direction of an issue (e.g., concerns around quality), system metrics would help in understanding the contributing factors to this issue (e.g., poor code coverage).
    • Keeping track of impact. As mentioned earlier, developers expect things to change and improve after sharing their thoughts in a survey. At the organizational level or team level, there currently isn’t a way to measure the current state and demonstrate the progress made based on the developers’ feedback.
    List summarizing the issues engineering leaders face when dealing with developer survey data alone
    Issues engineering leaders face when dealing with developer survey data on its own

    How Can Surveys Become More Impactful?

    Considering these issues, it would appear that augmenting survey results with system data, collected from engineering systems, could significantly help.

    Powerful insights come when blending qualitative insights from surveys with data and metrics from systems, processes, and workflows, an approach that Google, for one, has used very effectively with its People Analytics, with an average of 90% participation rate in surveys.

    Matthew Runkle, Director of Cloud Engineering at SmartBear, a Faros customer, shared an example. “We’ve always had this vision of correlating developer sentiment with the concrete process and outcome metrics we’re measuring on Faros to understand how the two are linked. For instance, one of the frequent pieces of feedback we got from our surveys was that developers wanted better tests. It was helpful to look at system data and correlate a team’s relative investment in product quality with its members’ satisfaction in this regard.”

    Here’s another example. Below is a chart that correlates survey responses on “goals and alignment” to a team’s ratio of unplanned work. It helps leaders understand whether lower scores on alignment correlate to higher levels of unplanned work. If corroborated, managers can take corrective action faster, by implementing measures to limit or address the amount of unplanned work that floods into the team.

    A scatter plot shows how lower scores in survey responses from the Cloud team correlate with higher levels of unplanned work. The Desktop has lower ratios of unplanned work and thus gave higher scores on questions related to alignment and goals.
    Correlating survey responses on goals and alignment with a team’s unplanned work ratio sheds light on developer feedback

    A Novel Approach to Blended Visibility

    To give engineering organizations the insights they need to monitor and improve the developer experience, we are delighted to introduce our new Developer Experience module.

    What is a module? Modules are prebuilt analytics libraries — inclusive of all the data sources, metrics, dashboards, widgets, and customizations you need — that run on top of the Faros AI platform.

    Infused with domain expertise, benchmarks, and best practices, modules provide rapid insight immediately upon connecting to your data sources. From there, you can build upon the module’s foundation by creating your own custom metrics, views, and reports.

    The Developer Experience module centralizes developer satisfaction survey data in one place and intersects the sentiment data from employee responses with telemetry-based data from engineering operations.

    A venn diagram from Faros AI: Survey Ddata and System Data overlap, creating an overlapping section titled Developer Experience.
    Faros AI provides novel blended visibility into the complete developer experience

    This novel blended visibility into the complete developer experience provides actionable insights that allow engineering leaders and their HR partners to take corrective measures faster and observe their impact on engagement, retention, and operational excellence over time.

    Engineering leaders and their HR partners are now able to ingest survey data from any source into the Faros AI platform and overlay engineering data and metrics on the survey responses around alignment and goals, developer productivity, quality, speed and agility, and more.

    Like everything in Faros, survey data can be analyzed over time and sliced and diced by team or other dimensions of choice.

    How It Works

    Because every organization is unique and each team is different, the Developer Experience module is designed to be completely configurable:

    1. Pull data from any survey tool you work with.
    2. Configure your survey themes or categories based on what makes sense for your teams.
    3. Select the system metrics you want to overlay on survey data, based on your team and organizational goals.

    To get you up and running quickly, you can also leverage pre-packaged survey templates from Faros, that include categories and metrics based on industry benchmarks and best practices. Our Lighthouse AI engine will be running behind the scenes to provide you with actionable insights to help you analyze and act upon survey insights.

    Want to see it in action? Request a demo of Faros AI today.

    Thierry Donneau-Golencer

    Thierry Donneau-Golencer

    Thierry is Head of Product at Faros AI, where he builds solutions to empower teams and drive engineering excellence. His previous roles include AI research (Stanford Research Institute), an AI startup (Tempo AI, acquired by Salesforce), and large-scale business AI (Salesforce Einstein AI).

    Connect
    Back to Blog Posts
    AI Is Everywhere. Impact Isn’t.
    75% of engineers use AI tools—yet most organizations see no measurable performance gains.

    Read the report to uncover what’s holding teams back—and how to fix it fast.
    Read the report
    AI Productivity Paradox Report 2025
    Discover the Engineering Productivity Handbook
    How to build a high-impact program that drives real results.
    ‍
    What to measure and why it matters.

    And the 5 critical practices that turn data into impact.
    Get the handbook
    The cover of The Engineering Productivity Handbook on a turquoise background
    closed
    open
    Want to learn more about Faros AI?

    Fill out this form and an expert will reach out to schedule time to talk.

    Loading calendar...
    An illustration of a lighthouse in the sea

    Thank you!

    A Faros AI expert will reach out to schedule a time to talk.
    P.S. If you don't see it within one business day, please check your spam folder.
    Oops! Something went wrong while submitting the form.

    More articles for you

    Editor's Pick
    AI
    DevEx
    3
    MIN READ

    Claude Code vs Devin: AI Coding Tools Comparison for Developers

    Compare Claude Code vs Devin for daily development work. Learn strengths, weaknesses, and best practices from real developer experience using both AI coding tools.
    June 6, 2025
    Editor's Pick
    DevProd
    DevEx
    Solutions
    4
    MIN READ

    Faros AI Powers the Future of AI-Driven Engineering Excellence with Microsoft Azure

    Supercharging the AI transformation with data-driven insights and orchestration built for the enterprise SDLC. Fully integrated with Microsoft and GitHub products, available on the Marketplace with MACC.
    May 20, 2025
    Editor's Pick
    DevEx
    10
    MIN READ

    Developer Experience Surveys Alone Aren’t Enough: The Case for Triangulating Developer Experience Data

    Uncover the limits of DevEx surveys and learn how to ensure insights are valid, actionable, and grounded in operational reality.
    May 6, 2025

    See what Faros AI can do for you!

    Global enterprises trust Faros AI to accelerate their engineering operations. Give us 30 minutes of your time and see it for yourself.

    Request a demo
    Salespeak