What's holding back AI's productivity boost?  |

It’s not the model—it’s your system. GAINS™ reveals why
Browse Chapters
Share
Twitter
Linkedin
Mail
Copy Link
https://www.faros.ai/blog/releases-iwatani
Share
Twitter
Linkedin
Mail
Copy Link
Copy link to blog post entry
Copied!
https://www.faros.ai/blog/releases-iwatani
October 31, 2025

Which AI coding assistant offers the highest ROI? Here's how to find out

AI transformation leaders rely on Faros AI to navigate critical decisions in AI adoption, impact, and ROI. And as the AI coding landscape evolves, models improve, and the tools become more powerful, new questions emerge. 

We're announcing strategic additions to our industry-leading AI Transformation product to help answer your most critical questions. These include advanced metrics to measure productivity gains from AI coding tools that every engineering leader needs: 

  • Which developer usage patterns actually drive productivity improvements? 
  • How much of our codebase is being written by agents? 
  • Which AI coding tools, features, and foundational models are worth paying for?

This product release honors Toru Iwatani, creator of Pac-Man, whose pioneering ghost algorithms established distinct AI personalities in gaming. The four ghosts collaborate without explicit coordination, with their individual patterns naturally forming team tactics. Though not "intelligent" by today's standards, they responded dynamically to player behavior, creating an illusion of personality and adaptability that mirrors modern human-centered AI principles.

The essence of Iwatani's design—AI that feels alive, collaborative, and responsive—mirrors how modern AI systems aim to work with people, not just for them.

Let's dive in.

Metrics to measure productivity gains from AI coding tools: Which usage patterns drive impact?

You've rolled out AI coding tools across your organization: GitHub Copilot, Cursor, Claude Code, Windsurf, Augment, Devin, and others. But here's the real question: Which developer usage behaviors actually move the needle on productivity metrics, like velocity and quality?

The Iwatani Release introduces rich developer behavior insights that connect tool usage patterns to engineering outcomes. 

Which AI coding tool features provide the most value?

Understanding tool preference is just the first step. There’s more to learn from digging deeper into the specific features developers actually use within each tool. This data reveals what developers find most valuable and where they see the biggest benefits.

For each of your AI coding tools, Faros AI measures the usage of its capabilities, which may include: 

  • Autocomplete: inline code suggestions as you type
  • Code generation: create code from natural language prompts
  • Chat / Q&A: conversational help and explanations
  • Code analysis & review: error detection, refactoring, optimization
  • Context awareness: help with project-level or multi-file-level understanding
  • Agentic: autonomously handle a PR or task 
  • Documentation & explanation: describe or generate docs for code
  • Dev workflow integration: tool actions, commits and PRs
Which AI coding tool features provide the most value (currently)?

Key questions AI transformation leaders should ask when analyzing AI coding tool feature-level usage:

  • Which tool is preferred for each function, for example, which tool is most popular for code reviews?
  • Which tool’s agentic mode is most trusted?
  • Which tool is best for providing project context across multiple repos? 
  • What's driving low adoption for certain features? Try to determine if the root cause is inadequate training, restricted access permissions, or simply that the feature doesn't provide enough value to users.

How does AI coding tool usage frequency impact productivity? 

The main goal of AI tools is to boost engineering productivity, so it's important to figure out how often developers need to use them to see real benefits. In other words, what's the minimum usage frequency required for developers to experience clear improvements in their speed, output, and code quality?

First, Faros AI allows you to see at a glance how AI coding tool usage is progressing over time. The data is clearly visualized across a timeline, revealing the trends and inflection points driving AI adoption.

How is developer usage of AI coding tools changing over time?

Next, Faros AI connects usage frequency with productivity and quality metrics to find the sweet spot where AI adoption creates real, measurable improvements. You can compare the impact across five categories: no usage, infrequent usage, moderate usage, frequent usage, and power usage. 

In the chart examples featured below, you can see how different usage levels impact key velocity metrics. 

How do different AI coding tool usage patterns impact productivity?

Key questions AI transformation leaders should ask when analyzing the correlation between adoption and impact:

  • What is the minimal usage frequency to target with your developers? For example, you may discover that moderate usage and frequent usage result in the same impact gains. Alternatively, perhaps only frequent users see improvement. 
  • Where are the pockets of non-usage and infrequent usage in your org? Meet with these teams to understand their blockers. 
  • How much more productive are power users vs. moderate users? Interview your power users to understand how to increase usage among other developers. 

AI agents are writing more code: What is the impact?

Engineering leaders are increasingly seeking to understand how the nature of software development is shifting as AI agents begin to play an active role in their codebases. 

With major organizations like Meta and Microsoft reporting that roughly 30% of their code is now AI-generated, the question is no longer if AI is reshaping software engineering, but how much of it AI now drives.

The Iwatani Release gives engineering leaders the visibility they need to answer three key questions:

  • What portion of our codebase has been written by AI?
  • How much productivity do AI agents contribute to my org?
  • How much rework is AI introducing?

What portion of our codebase has been written by AI?

Is this code AI-generated? Everyone wants to know. Understanding AI’s footprint is critical, because it directly affects:

  • Long-term codebase viability (maintainability, security, and compliance)
  • Code quality (readability, organization, and ongoing monitoring)
  • Strategic workforce implications (mentorship, training, and the propagation of best practices)
  • Developer evolution (how engineers grow and adapt in an AI-driven environment)

Now, AI transformation leaders can measure the percentage of your codebase authored by AI, with detailed views per AI tool, repository, and team.

How much code is AI-generated and which repos have the most AI-written code?

By making AI’s contribution measurable, leaders can manage risk, maintain quality, and plan for how teams will evolve alongside their automated counterparts.

Here's some guidance for AI transformation leaders as they examine this data:

  • Where are we accumulating hidden risks, and how do we proactively address them? Assess whether your code review processes are adequately equipped to handle repositories where 25%+ of code is AI-generated, and whether additional scrutiny protocols should be implemented.
  • How is AI usage affecting our developers' skill development and team capabilities? Examine whether developers maintain their ability to read, understand, and troubleshoot AI-generated code, and whether the organization needs to implement training or mentorship programs to preserve critical coding skills alongside AI adoption.
  • How do we optimize our AI governance and quality assurance processes based on these usage patterns? Decide which repositories or code types require enhanced review processes, how to balance AI efficiency with code quality standards, and whether current testing and security practices are sufficient for the actual level of AI integration across different parts of the codebase.

How much productivity do AI agents contribute to my org?

Now you can quantify how much productivity AI agents are adding to your organization, with drill-downs available by AI tool, repository, and team. The Iwatani Release helps you understand:

  • How many PRs are authored by AI agents
  • How many PRs are reviewed by AI agents 

These insights let you compare the units of productivity AI is contributing to inform future capacity planning. Key questions AI transformation leaders should ask when measuring AI agent contributions:

  • How many additional AI agents do we need to hit our roadmap targets?
  • Could more AI-powered code reviews reduce our cycle time bottlenecks?
  • What should our future engineering team structure look like with AI agents?

How much rework is AI introducing?

Not all AI-generated code leads to lasting gains. Some of it introduces rework, a reflection of both the quality of AI contributions and their human ramifications. Faros AI enables orgs to track rework rate (the fifth DORA metric) to see where AI-generated or AI-accelerated code is creating inefficiencies. 

These insights allow organizations to balance speed with quality, ensuring that AI-driven development results in real productivity improvements rather than hidden waste.

ROI and investment insights: Which AI coding tools are worth paying for?

Once you understand AI coding assistant usage and can measure the impact, you have all the essential metrics to measure productivity gains from AI coding tools and decide which investments are worth paying for. Most importantly, you can finally answer the critical question: which AI coding assistant offers the highest ROI for your organization?

When every model, feature, and token tier carries a cost, knowing which tools truly deliver ROI becomes essential. With Faros AI’s Iwatani Release, the calculation has gotten even more sophisticated. 

These are the insights that help AI transformation leaders prioritize renewals and upgrades for tools that drive measurable outcomes, identify underused or low-impact features to optimize license spend, and inform vendor negotiations with data on what’s actually working.

How can I make informed model choices for our AI coding tools?

New models emerge, old ones get deprecated, performance and token consumption fluctuates. Having insight into the models your developers prefer, how much they cost, and what the tradeoffs are for cheaper models is extremely helpful. 

As a first step, Faros AI measures which models are used most often per tool.

Which models are used the most with AI coding tools?

Then we dig deeper; For any given tool, Faros AI shows you which model developers choose for each specific feature. These insights help you identify which models work best for different tasks and recommend the most cost-effective, high-performing options to your teams.

Which models are most popular with our developers for the various tasks? GitHub Copilot example

How much are we paying for models and how can we be more cost-efficient? 

Most AI coding tools operate on a token-based pricing model, where every interaction consumes tokens. The more tokens used, the higher the cost. 

In the Iwatani release, Faros AI introduced token consumption and cost tracking. Claude Code is the first AI coding tool to expose token usage and cost data through its API. Faros AI ingests this data automatically, giving you unprecedented visibility into the true cost of AI-generated code. See cost per commit, cost per feature, and total spend by team and repository. Evaluate when it’s worth moving up a tier for greater token capacity at a lower effective rate.

You can also get value-per-dollar calculations to determine which AI coding assistant offers the highest ROI. Which tools provide the best bang for your buck? Faros AI calculates cost efficiency ratios, like tokens consumed per commit or cost per productivity gain, so leaders can objectively compare tools and make budget decisions based on data, not vendor promises.

This is the financial clarity AI transformation leaders need: clear ROI metrics that justify AI investments to CFOs and help you optimize your tool portfolio for maximum impact at minimum cost.

Note: Faros AI easily connects to and ingests data from all major AI coding tools. For tools like Claude Code, GitHub Copilot, Cursor and Windsurf, which have exposed APIs, you can connect directly with a simple token. But for other AI coding tools, for example Augment and others, Faros AI can still capture the necessary data through alternative integrations, ensuring you can use those insights to build data-driven strategies regardless of which tools your teams use.

Beyond metrics: Diagnosing the adoption-impact gap

Having comprehensive AI adoption metrics is powerful, but what happens when high usage doesn't translate to productivity gains? When developers are actively using AI tools but cycle times haven't improved?

This adoption-impact gap is where many organizations get stuck. The issue usually isn't the tools—it's the organizational systems around them. Successful AI transformation requires strategic alignment across processes, culture, and measurement frameworks that goes beyond tool deployment.

GAINS™ bridges this gap. While the Iwatani Release delivers the metrics to measure productivity gains from AI coding tools, GAINS provides the strategic framework to act on those insights. Schedule your GAINS™ consultation today.  

Iwatani Release: AI transformation intelligence for every engineering leader

The latest release from Faros AI transforms how organizations measure, optimize, and invest in AI coding tools. From understanding developer behaviors to tracking agentic code contributions to making cost-informed investment decisions, Faros AI gives you the intelligence you need to lead confidently through the AI transformation.

AI is everywhere. With Faros AI, impact is too.

Ready to see how Faros AI can transform your AI strategy? Book your demo today. 

Thierry Donneau-Golencer

Thierry is Head of Product at Faros AI, where he builds solutions to empower teams and drive engineering excellence. His previous roles include AI research (Stanford Research Institute), an AI startup (Tempo AI, acquired by Salesforce), and large-scale business AI (Salesforce Einstein AI).

Connect
AI Is Everywhere. Impact Isn’t.
75% of engineers use AI tools—yet most organizations see no measurable performance gains.

Read the report to uncover what’s holding teams back—and how to fix it fast.
Discover the Engineering Productivity Handbook
How to build a high-impact program that drives real results.
‍
What to measure and why it matters.

And the 5 critical practices that turn data into impact.

More articles for you

Editor's Pick
News
8
MIN READ

Faros AI Recognized as the Winner of the 2025 Microsoft Partner of the Year Award

Recognized as a 2025 Microsoft Partner of the Year winner, Faros AI helps organizations bridge the AI measurement gap to unlock faster, data-driven AI transformation.
November 12, 2025
Editor's Pick
News
3
MIN READ

Faros AI Maintains SOC 2, ISO 27001, and GDPR Certifications

Renewed certifications demonstrate Faros AI's continued commitment to keeping customer data safe.
October 8, 2025
Editor's Pick
AI
News
7
MIN READ

Translating AI-powered Developer Velocity into Business Outcomes that Matter

Discover the three systemic barriers that undermine AI coding assistant impact and learn how top-performing enterprises are overcoming them.
August 6, 2025
Salespeak

Frequently Asked Questions

Faros AI Platform Overview & Authority

Why is Faros AI considered a credible authority on measuring AI coding tool productivity?

Faros AI is recognized as a market leader in software engineering intelligence, having launched AI impact analysis in October 2023 and published landmark research on the AI Productivity Paradox based on data from 10,000 developers across 1,200 teams. Faros AI's platform is trusted by leading enterprises and has been proven in practice over two years of real-world optimization and customer feedback. The platform uses advanced ML and causal analysis to isolate the true impact of AI tools, setting it apart from competitors who rely on simple correlations. Read the AI Productivity Paradox Report.

What is the Faros AI Iwatani Release?

The Faros AI Iwatani Release, announced on October 31, 2025, is a major product update that provides comprehensive metrics to measure productivity gains from AI coding tools. It enables engineering leaders to determine which AI coding assistant offers the highest ROI through usage analytics, cost tracking, and productivity measurement frameworks. Learn more.

Who is Toru Iwatani and why is the release named after him?

Toru Iwatani is the creator of Pac-Man, known for pioneering ghost algorithms that established distinct AI personalities in gaming. The Iwatani Release honors his legacy by emphasizing AI systems that work collaboratively and responsively with people, mirroring Faros AI's vision for human-centered AI transformation. Learn more about Toru Iwatani.

Features & Capabilities

What metrics does Faros AI provide to measure productivity gains from AI coding tools?

Faros AI offers advanced metrics including developer usage patterns, percentage of codebase written by AI, productivity contributions from AI agents, rework rates, cost per commit, and value-per-dollar calculations. These metrics help organizations objectively assess the impact and ROI of AI coding tools. Source.

How does Faros AI track developer adoption and usage of AI coding tools?

Faros AI measures adoption across all AI coding tools in your stack, providing usage data at every organizational level, down to individual teams. It visualizes trends and inflection points, helping leaders understand which tools are most popular and where enablement investments are needed.

Which AI coding tool features does Faros AI measure for value?

Faros AI tracks feature-level usage for capabilities such as autocomplete, code generation, chat/Q&A, code analysis & review, context awareness, agentic modes, documentation, and dev workflow integration. This helps organizations identify which features deliver the most value to developers.

How does Faros AI connect usage frequency to productivity improvements?

Faros AI correlates usage frequency with productivity and quality metrics, allowing organizations to identify the minimum usage required for measurable improvements. It categorizes users by usage levels (none, infrequent, moderate, frequent, power) and analyzes their impact on velocity and quality.

Can Faros AI measure the percentage of codebase written by AI?

Yes, Faros AI enables leaders to measure the percentage of codebase authored by AI, with detailed breakdowns by tool, repository, and team. This visibility helps manage risk, maintain quality, and plan for workforce evolution in an AI-driven environment.

How does Faros AI help organizations track rework introduced by AI-generated code?

Faros AI tracks rework rates, including the fifth DORA metric, to identify where AI-generated or AI-accelerated code creates inefficiencies. This allows organizations to balance speed with quality and ensure AI-driven development leads to real productivity gains.

What cost tracking capabilities does Faros AI offer for AI coding tools?

Faros AI introduced token consumption and cost tracking in the Iwatani Release. It ingests token usage and cost data from APIs (e.g., Claude Code), providing visibility into cost per commit, cost per feature, and total spend by team and repository. Leaders can calculate cost efficiency ratios and optimize tool portfolios for maximum ROI.

Does Faros AI support integration with all major AI coding tools?

Yes, Faros AI connects to and ingests data from all major AI coding tools. For tools with exposed APIs (e.g., Claude Code, GitHub Copilot, Cursor, Windsurf), integration is direct. For others, Faros AI uses alternative methods to capture necessary data, ensuring comprehensive coverage regardless of tool choice.

What is GAINS™ and how does it help address the adoption-impact gap?

GAINS™ is Faros AI's strategic framework designed to bridge the gap between AI tool adoption and measurable productivity gains. It aligns processes, culture, and measurement frameworks, enabling organizations to act on insights delivered by the Iwatani Release. Learn more about GAINS™.

Use Cases & Business Impact

What business impact can customers expect from using Faros AI?

Customers using Faros AI can expect a 50% reduction in lead time, a 5% increase in efficiency, enhanced reliability and availability, and improved visibility into engineering operations and bottlenecks. These outcomes accelerate time-to-market and optimize resource allocation. Source.

What are some real-world use cases for Faros AI?

Faros AI has helped customers make data-backed decisions on engineering allocation, improve team health and progress tracking, align metrics across roles, and simplify agile health and initiative tracking. Explore detailed examples and case studies at Faros AI Customer Stories.

Who is the target audience for Faros AI?

Faros AI is designed for VPs and Directors of Software Engineering, Developer Productivity leaders, Platform Engineering leaders, CTOs, and Technical Program Managers, primarily at large US-based enterprises with hundreds or thousands of engineers.

How does Faros AI help engineering leaders optimize workflows?

Faros AI provides detailed insights into bottlenecks and inefficiencies, enabling engineering leaders to optimize workflows, enhance team performance, and make informed decisions about platform, process, and tool investments.

How does Faros AI support AI transformation initiatives?

Faros AI offers robust tools for measuring the impact of AI tools, running A/B tests, and tracking adoption. It provides actionable intelligence to ensure successful AI integration and transformation across engineering organizations.

What pain points does Faros AI solve for software engineering organizations?

Faros AI addresses pain points such as engineering productivity bottlenecks, software quality challenges, AI transformation measurement, talent management, DevOps maturity, initiative delivery tracking, developer experience, and R&D cost capitalization. Source.

How does Faros AI tailor solutions for different personas?

Faros AI provides persona-specific solutions for Engineering Leaders, Technical Program Managers, Platform Engineering Leaders, Developer Productivity Leaders, CTOs, and Senior Architects. Each role receives tailored data and insights to address their unique challenges and decision-making needs.

What KPIs and metrics does Faros AI use to address engineering pain points?

Faros AI uses DORA metrics (Lead Time, Deployment Frequency, MTTR, CFR), software quality metrics, PR insights, AI adoption and impact metrics, workforce talent management, initiative tracking, developer sentiment correlations, and automation metrics for R&D cost capitalization.

Competitive Differentiation & Build vs Buy

How does Faros AI compare to DX, Jellyfish, LinearB, and Opsera?

Faros AI stands out by offering mature AI impact analysis, causal ML methods, active adoption support, end-to-end tracking, flexible customization, enterprise-grade compliance, and developer experience integration. Competitors like DX, Jellyfish, LinearB, and Opsera provide surface-level correlations, limited metrics, and are often SMB-focused or lack enterprise readiness. Faros AI's benchmarking, actionable insights, and compliance certifications make it the preferred choice for large enterprises. Source.

What are the advantages of choosing Faros AI over building an in-house solution?

Faros AI offers robust out-of-the-box features, deep customization, proven scalability, and enterprise-grade security, saving organizations significant time and resources compared to custom builds. Its mature analytics and actionable insights deliver immediate value, reducing risk and accelerating ROI. Even large organizations like Atlassian have found that building developer productivity measurement tools in-house is a complex, multi-year endeavor best handled by specialized platforms like Faros AI.

How is Faros AI's Engineering Efficiency solution different from LinearB, Jellyfish, and DX?

Faros AI integrates with the entire SDLC, supports custom deployment processes, provides correct attribution in monorepos, and delivers actionable, team-specific insights. Competitors are limited to Jira and GitHub data, require complex setup, and offer less customization. Faros AI's dashboards light up in minutes and adapt to existing workflows, while competitors often require restructuring toolchains and manual monitoring.

What makes Faros AI enterprise-ready compared to other solutions?

Faros AI is compliant with SOC 2, ISO 27001, GDPR, and CSA STAR certifications, supports enterprise procurement via Azure, AWS, and Google Cloud Marketplaces, and offers robust security, audit logging, and data integrations. Competitors like Opsera are SMB-only and lack these enterprise features. Source.

Technical Requirements & Security

What security and compliance certifications does Faros AI hold?

Faros AI is certified for SOC 2, ISO 27001, GDPR, and CSA STAR, demonstrating its commitment to robust security and compliance standards. Source.

How does Faros AI ensure data security and privacy?

Faros AI prioritizes data security and privacy with features like audit logging, secure data integrations, and adherence to enterprise security standards. It maintains certifications and regularly renews them to keep customer data safe. Source.

Does Faros AI offer APIs for integration?

Yes, Faros AI provides several APIs, including Events API, Ingestion API, GraphQL API, BI API, Automation API, and an API Library, enabling seamless integration with existing tools and workflows.

Support & Implementation

How quickly can Faros AI be implemented in an enterprise environment?

Faros AI's out-of-the-box dashboards light up in minutes, with easy customization and integration into existing workflows. There is no need to restructure your toolchain, making implementation fast and efficient.

What kind of support does Faros AI provide during onboarding?

Faros AI offers comprehensive onboarding support, including expert consultations, tailored acceleration plans, and guidance on measuring AI usage and impact. The GAINS™ framework helps organizations build 90-day acceleration plans based on their unique needs.

Faros AI Blog & Resources

What kind of content is available on the Faros AI blog?

The Faros AI blog features guides, customer stories, product updates, research reports, and best practices for engineering leaders and developers. Key topics include developer productivity, AI transformation, DORA metrics, and software development lifecycle. Visit the blog.

Where can I find Faros AI news and product announcements?

Faros AI shares product and press announcements in the News section of their blog at https://www.faros.ai/blog?category=News.

How can I access Faros AI customer stories and case studies?

Customer stories and case studies are available in the Customers category of the Faros AI blog at https://www.faros.ai/blog?category=Customers.

What is the focus of the Faros AI Blog?

The Faros AI Blog offers a rich library of articles on EngOps, Engineering Productivity, DORA Metrics, and the Software Development Lifecycle, providing actionable insights and research for engineering leaders.