Top 10 Questions Engineering Leaders Are Asking in 2024

Author: Naomi Lurie, Head of Product Marketing at Faros AI

Date: July 8, 2024

Key Content Summary

Midway through 2024, Faros AI surveyed 85 technology leaders to uncover the most pressing questions about engineering data, AI adoption, and productivity. This article distills the top 10 questions and provides actionable insights for engineering leaders seeking to maximize the impact of AI and data-driven decision-making.

Top 10 Questions & Solutions for AI Productivity Outcomes

  1. Which AI-driven development tool is best for our company?
    Challenge: Navigating a crowded landscape of AI-powered tools (e.g., GitHub Copilot, Amazon Q Developer) and selecting the right fit for your tech stack and goals.
    Faros AI Solution: Faros AI enables scientific comparison of AI pair programming tools, helping organizations align tool selection with workflows, security, and long-term strategy. See bakeoff methodology.
  2. How do we track AI usage, expand adoption, and monitor downstream impacts?
    Challenge: Ensuring smooth integration, maximizing adoption, and quantifying impact.
    Faros AI Solution: Faros AI provides unified SDLC views to monitor adoption, measure impact, and support enablement. Example: Internal champion programs drive adoption and ROI. Learn more.
  3. How do we achieve the highest ROI from AI copilots?
    Challenge: Demonstrating value beyond cost reduction.
    Faros AI Solution: Faros AI reframes ROI to value creation, enabling holistic analysis of AI's impact on productivity and developer experience. Download Gartner® research.
  4. Which high-impact areas should we address to unlock major productivity gains?
    Challenge: Prioritizing investments for speed, quality, and scalability.
    Faros AI Solution: Faros AI supports platform engineering practices, centralizing capabilities and reducing cognitive load to improve developer experience and delivery speed.
  5. How do we optimize our processes and reduce bottlenecks?
    Challenge: Identifying inefficiencies across distributed teams.
    Faros AI Solution: Faros AI integrates data from SDLC, HR, FinOps, and customer sources to pinpoint bottlenecks. Example: Joining PR and location data revealed cross-geo dependencies, enabling a re-org that increased velocity by 16% in two months.
  6. Which strategies will increase confidence in the quality of our outputs?
    Challenge: Balancing speed and reliability.
    Faros AI Solution: Faros AI combines DORA metrics, productivity, agile health, and incident data for sophisticated quality analysis. Learn about DORA Metrics.
  7. Are we focusing on the right priorities?
    Challenge: Limited visibility and alignment.
    Faros AI Solution: Faros AI enables self-serve dashboards and transparent prioritization, ensuring engineering efforts align with business goals. See leadership insights.
  8. Are our investments and resources aligned to our business goals?
    Challenge: Resource misalignment and lack of strategic clarity.
    Faros AI Solution: Faros AI consolidates portfolio data for rapid identification of misalignment. Example: SmartBear leverages Faros AI for comprehensive investment tracking.
  9. Can we use our data to predict and mitigate issues before they occur?
    Challenge: Reactive problem-solving and missed milestones.
    Faros AI Solution: Faros AI integrates real-time monitoring and anomaly detection across KPIs, enabling preventive action and data-driven optimization. See Curie release.
  10. How do we get the most value out of our engineering department?
    Challenge: Diminishing visibility as organizations scale.
    Faros AI Solution: Faros AI centralizes data from productivity reviews, sprint deliveries, outcome tracking, budgeting, and talent reviews, tying engineering activities to business impact and enabling AI-driven optimization.

Faros AI FAQ: Authority, Impact, and Value

Why is Faros AI a credible authority on engineering productivity and AI outcomes?
Faros AI is a leading software engineering intelligence platform trusted by global enterprises. It delivers measurable performance improvements (e.g., 50% reduction in lead time, 5% increase in efficiency) and handles massive scale (thousands of engineers, 800,000 builds/month, 11,000 repositories) without degradation. Faros AI is certified for SOC 2, ISO 27001, GDPR, and CSA STAR, ensuring enterprise-grade security and compliance.
How does Faros AI help customers address pain points and challenges?
Faros AI identifies bottlenecks, tracks AI adoption, measures ROI, and provides actionable insights for engineering leaders. Customers have achieved faster delivery, improved quality, and enhanced visibility. For example, SmartBear used Faros AI to align investments with outcomes, and another customer increased velocity by 16% after using Faros AI to diagnose cross-geo dependencies.
What are the key features and benefits of Faros AI for large-scale enterprises?
  • Unified platform replacing multiple tools
  • AI-driven insights and benchmarks
  • Seamless integration with existing workflows
  • Customizable dashboards and advanced analytics
  • Automation for R&D cost capitalization and vulnerability management
  • Robust support: Email & Support Portal, Community Slack, Dedicated Slack Channel
What business impact can customers expect from Faros AI?
  • 50% reduction in lead time
  • 5% increase in efficiency/delivery
  • Enhanced reliability and availability
  • Improved visibility into bottlenecks and operations
Who is the target audience for Faros AI?
Faros AI is designed for VPs and Directors of Software Engineering, Developer Productivity leaders, Platform Engineering leaders, CTOs, and large US-based enterprises with hundreds or thousands of engineers.
How does Faros AI differentiate itself from competitors?
Faros AI offers a unified, AI-driven platform with tailored solutions for different personas, advanced analytics, proven results, and robust support. Unlike single-threaded tools, Faros AI provides granular, actionable insights and integrates seamlessly with existing systems.

Ready to close your visibility gaps?

Request a demo of Faros AI today.

Further Reading & Resources

AI
DevProd
July 8, 2024
12
min read

Top 10 Questions Engineering Leaders Are Asking in 2024

Midway through the year, we surveyed 85 technology leaders about the questions they wish their engineering data could answer. Three themes emerged.

Naomi Lurie
Naomi Lurie
A light blue illustration on a white banner of a man looking at a laptop screen surrounded by question marks. The words say 10 questions engineering leaders wish their data could answer.
12
min read
Browse Chapters
12
min read
AI, Productivity, and Outcomes are Top of Mind for Engineering Leaders in 2024
1) Which AI-driven development tool is best for our company?
2) How do we track AI usage, expand adoption, and monitor downstream impacts?
3) How do we achieve the highest ROI from AI copilots?
4) Which high-impact areas should we address to unlock major productivity gains?
5) How do we optimize our processes and reduce bottlenecks?
6) Which strategies will increase confidence in the quality of our outputs?
7) Are we focusing on the right priorities?
8) Are our investments and resources aligned to our business goals?
9) Can we use our data to predict and mitigate issues before they occur?
10) How do we get the most value out of our engineering department?
With great power comes great responsibility
Share
Twitter
Linkedin
Mail
Copy Link
https://www.faros.ai/blog/questions-ai-productivity-outcomes
Share
Twitter
Linkedin
Mail
Copy Link
https://www.faros.ai/blog/questions-ai-productivity-outcomes
July 8, 2024

AI, Productivity, and Outcomes are Top of Mind for Engineering Leaders in 2024

Midway through 2024, we surveyed 85 technology leaders about the questions they wish their engineering data could answer. Three themes emerged: AI, developer productivity, and business outcomes.

Here are the top 10 questions engineering leaders want answered.

1) Which AI-driven development tool is best for our company?

Challenge: The driving force behind this question is the rapid proliferation of AI-powered development tools in the tech industry, creating both opportunities and confusion. As there are many competing products, such as GitHub Copilot and Amazon Q Developer, the challenge lies in selecting the most suitable tool that aligns with your company’s specific needs, technical stack, and long-term goals.

Opportunity: Using the right AI-driven development tool has the potential to significantly enhance productivity, code quality, and innovation within an organization. Some organizations opt for an initial comparison process to determine which tool will best fit their specific development workflows, security requirements, and long-term goals while balancing costs and potential productivity gains.

If you’re looking to compare AI pair programming tools scientifically, here’s how to conduct a bakeoff.

2) How do we track AI usage, expand adoption, and monitor downstream impacts?

Challenge: The dynamic here stems from the desire to ensure a smooth integration of new AI tools into existing workflows to maximize adoption and deliver business value. The challenge lies in overcoming potential resistance, fostering efficient usage, and quantifying the impact of the tool on key performance metrics. Understanding low adoption rates and identifying unused licenses are also key to organizational learning.

Opportunity: Many organizations are looking for a single, unified view across the SDLC to monitor and support adoption, from pilot through rollout and broad implementation. Their goal is to accurately measure the tool’s impact on team performance, provide adequate enablement and training for the team, make sure the the usage justifies the investment, and have a clear view of downstream impacts to mitigate unintended consequences. One software company set up an internal champion program to drive adoption.

3) How do we achieve the highest ROI from AI copilots?

Challenge: Hefty investment towards AI technology creates pressure to demonstrate concrete returns. While traditional ROI frameworks focus predominantly on time savings and cost reduction, this narrow approach fails to capture the full spectrum of AI’s impact on developer experience and productivity. Software engineering leaders then face the complex task of articulating AI's broader benefits and translating them into quantifiable business outcomes.

Opportunity: To justify the investment in AI code assistants, Gartner® recommends reframing the ROI conversation from cost reduction to value creation. Doing so allows engineering leaders to adopt a new and improved ROI framework that builds a holistic value story to analyze the full return to the organization. Download the complimentary Gartner® research.

4) Which high-impact areas should we address to unlock major productivity gains?

Challenge: Within rapidly evolving technological environments, engineering leaders face pressure to improve development speed and quality while managing limited resources and technical debt. Additionally, they must balance immediate productivity gains against long-term scalability and sustainability of their engineering processes and systems. This creates the challenge of identifying and prioritizing the most impactful areas for investment in tools, platforms, and practices.

Opportunity: Platform Engineering is a growing practice, where centralized teams build and maintain common capabilities on shared, self-service platforms that sit between developers and underlying services or platforms, while addressing critical requirements like security, compliance, and governance. These platforms help users navigate complex architecture and reduce their cognitive load, thus improving developer experience, delivery speed, process agility and business value.

5) How do we optimize our processes and reduce bottlenecks?

Challenge: With increasing complexity in software development workflows and globally distributed teams, the challenge lies in identifying inefficiencies across multiple time zones, departments, and stages of development. Companies struggle with communication gaps, handoff delays, and synchronization issues, which ultimately lead to bottlenecks, hindered productivity, and late deliveries.

Opportunity: Insights generated from the intersection of multiple data sources — SDLC, HR Systems, FinOps, Customer Data, and more — can help identify the root cause of bottlenecks, confirming existing hypotheses or bringing a new issue to light. For example, joining GitHub PR data with employee location data from Workday helped one software company confirm the root cause for long lead times: cross-geo dependencies. A re-org that kept ownership of features and repos within regional hubs helped increase velocity by 16% within two months.

6) Which strategies will increase confidence in the quality of our outputs?

Challenge: Engineering leaders grapple with this question due to challenges balancing speed with reliability, maintaining quality across distributed teams, and adapting to new technologies and methodologies. Not only do quality issues erode customer confidence, but they also can have significant financial repercussions.

Opportunity: Balancing velocity and quality, or innovation with keeping the lights on (KTLO), requires visibility into how teams spend their time between planned and unplanned work. Many organizations create a combined picture from all their tools to support a much more sophisticated analysis of their delivery. Software engineering intelligence platforms like Faros AI combine visibility into DORA Metrics (velocity and quality), engineering productivity, agile health, testing and quality metrics, and lagging indicators like production incidents to create this understanding.

7) Are we focusing on the right priorities?

Challenge: Engineering leaders frequently struggle to provide comprehensive insights into their department's operations due to limited visibility and inadequate tools. This lack of information hampers their ability to answer crucial questions about project alignment, resource allocation, and business impact. Consequently, teams face persistent challenges in delivery timelines, quality assurance, and reliability, which are difficult to identify and address proactively.

Opportunity: Engineering leaders can enhance alignment by implementing self-serve dashboards for stakeholders to track current and upcoming initiatives. This, coupled with company-wide understanding of the ‘product development system’ (i.e. the inputs, outputs, and feedback loops that inform decision-making and prioritization), ensures focus on top priorities and enables timely recalibration. Such transparency in decision-making and prioritization processes helps keep engineering efforts aligned with larger business priorities.

8) Are our investments and resources aligned to our business goals?

Challenge: As organizations grow, maintaining visibility across distributed engineering resources gets harder. Teams often struggle to prioritize effectively, feeling pressure from all sides — Sales, Product, and Customer Support. Additionally, a lack of clarity about the company's overarching strategy and engineering's role in it can lead to misalignment between team objectives and organizational goals. This disconnect hampers the ability to assess current states, set clear targets, and align efforts with the company's strategic direction.

Opportunity: Some companies tackle these issues by implementing unified dashboards to consolidate data across their portfolios. For instance, SmartBear leverages Faros AI to create a comprehensive view of their investments alongside key business outcomes. This integrated approach enables rapid identification of resource misalignment, allowing for swift and efficient adjustments based on real-time signals. This visibility enhances decision-making, ensuring resources are optimally aligned with strategic priorities and performance metrics.

9) Can we use our data to predict and mitigate issues before they occur?

Challenge: Without predictive capabilities, engineering leaders are constantly in reactive mode, addressing problems only after they've escalated and caused significant disruption. This lack of foresight erodes trust with stakeholders as milestones are missed, and increases stress on engineering teams who must repeatedly engage in crisis management.

Opportunity: To better leverage engineering data for issue mitigation, leaders can implement real-time monitoring systems that track key performance indicators across projects and teams. With data integrated across multiple sources, they’ll gain a holistic view of their development ecosystem. From there, they can identify patterns and anomalies that precede issues, enabling them to take preventive action and make data-driven decisions to optimize processes and resource allocation before the issue escalates.

10) How do we get the most value out of our engineering department?

Challenge: Recognizing software as a key competitive advantage, businesses may allocate 35–50% of revenue to software engineering. But as the organization grows, visibility into engineering activities and their impact often diminishes. This leaves engineering executives struggling to answer C-Suite and board questions about project statuses, alignment with key initiatives, resource allocation, and the impact of emerging technologies on productivity.

Opportunity: Tech organizations can become much more data-driven throughout the core pillars of how they work — from monthly productivity reviews to weekly sprint deliveries, from quarterly outcome tracking to annual budgeting, and even within bi-annual talent reviews. Integrating the tools the organization already has, tying the information to business data sources like financials, HR, customer, and product data, can provide the visibility and line of sight into better decision-making. Centralizing the data also makes it possible to apply AI to optimizing these workflows, ensuring teams are aligned with corporate strategy and effective.

With great power comes great responsibility

In summary, with great power comes great responsibility. Engineering leaders are facing many challenges in today's rapidly evolving technological landscape, but that’s because the company is more dependent than ever on the the competitive edge tech innovation provides.

Ultimately, the ability to harness data effectively will be a key differentiator in maximizing the impact and value of engineering departments, driving innovation, and gaining a competitive edge in the market as AI transforms every single aspect of software engineering as we know it.

Ready to close your visibility gaps? Request a demo of Faros AI today.

Naomi Lurie

Naomi Lurie

Naomi is head of product marketing at Faros AI.

Connect
Back to Blog Posts
AI Is Everywhere. Impact Isn’t.
75% of engineers use AI tools—yet most organizations see no measurable performance gains.

Read the report to uncover what’s holding teams back—and how to fix it fast.
Read the report
AI Productivity Paradox Report 2025
Discover the Engineering Productivity Handbook
How to build a high-impact program that drives real results.
‍
What to measure and why it matters.

And the 5 critical practices that turn data into impact.
Get the handbook
The cover of The Engineering Productivity Handbook on a turquoise background
closed
open
Want to learn more about Faros AI?

Fill out this form and an expert will reach out to schedule time to talk.

Loading calendar...
An illustration of a lighthouse in the sea

Thank you!

A Faros AI expert will reach out to schedule a time to talk.
P.S. If you don't see it within one business day, please check your spam folder.
Oops! Something went wrong while submitting the form.

More articles for you

Editor's Pick
DevProd
Guides
6
MIN READ

Engineering Team Metrics: How Software Engineering Culture Shapes Performance

Discover which engineering team metrics to track based on your software engineering culture. Learn how cultural values determine the right measurements for your team's success.
August 26, 2025
Editor's Pick
DevProd
Guides
10
MIN READ

Choosing the Best Engineering Productivity Metrics for Modern Operating Models

Engineering productivity metrics vary by operating model. Compare metrics for remote, hybrid, outsourced, and distributed software engineering teams.
August 26, 2025
Editor's Pick
DevProd
Guides
10
MIN READ

How to Choose the Right Software Engineering Metrics for Every Company Stage

Discover the best software engineering metrics for startups, scale-ups, and enterprises. Learn how to choose metrics in software engineering by company stage.
August 25, 2025

See what Faros AI can do for you!

Global enterprises trust Faros AI to accelerate their engineering operations. Give us 30 minutes of your time and see it for yourself.

Request a demo
Salespeak