TL;DR: A software factory brings AI agents into a repeatable system for building and shipping software. Agents can plan, write, test, review, and troubleshoot code, while harnesses, skills, evals, and human checkpoints keep the work moving. As more implementation shifts to agents, engineering teams focus more on the surrounding system: choosing the right models and workflows, verifying the output, controlling costs, and measuring whether all that AI activity actually results in reliable, shipped software.

What is a software factory?

A software factory is a software development system that uses AI agents, engineering tools, organizational context, automated workflows, verification, and human judgment to move work from an issue or specification to shipped software.

Instead of operating as isolated coding sessions, agents work within a broader system that can coordinate planning, implementation, testing, review, debugging, and other engineering tasks. The factory provides the infrastructure around that work, including the context agents receive, the tools and environments they can access, the workflows that coordinate them, and the checks that determine whether their output can move forward.

Software factories build on several stages of AI-assisted development. Coding assistants brought AI into the developer's editor. Coding agents expanded the scope of work AI could perform by navigating repositories, editing multiple files, executing commands, and running tests. Cloud agents made more of this work asynchronous. Software factories organize these capabilities into repeatable engineering workflows that can span multiple agents, systems, and stages of the SDLC.

Existing CI/CD infrastructure can also become part of the factory. CI/CD automates predefined processes for building, testing, and deploying software, while agentic workflows introduce systems that can reason and act within those processes.

The result is a broader unit of automation. A coding agent can complete an engineering task; a software factory coordinates the system responsible for moving that work toward a verified software outcome.

Why are software factories emerging now?

AI coding agents can increasingly work across larger engineering tasks, from investigating issues and modifying multiple files to running tests and preparing pull requests. As organizations deploy these capabilities across more of the SDLC, the unit of AI-assisted work expands from an individual coding session to an engineering workflow.

A work item can move through a sequence such as:

Issue created → triage → planning → implementation → testing → verification → review → human approval → deployment

Agents can perform work at multiple stages, automation can coordinate handoffs, and human judgment can enter at defined checkpoints. At scale, the engineering challenge becomes one of operating the system that produces software.

How does a software factory work?

Software factory architectures vary, but they generally bring together three capabilities: coordinating engineering work, giving agents the environment and knowledge needed to perform it, and verifying the resulting output.

Intake and orchestration

Work can enter the factory through a developer request, GitHub issue, Jira ticket, CI failure, production incident, security finding, or another engineering event. Some workflows begin with direct human intent, while others can be triggered automatically.

The agent harness coordinates what happens next. It can determine which agent and model handle the task, what context and tools they receive, which actions they can perform, when another agent becomes involved, and where human approval is required.

GitHub describes a similar model for agentic workflows, where repository events can trigger scoped agent tasks while existing controls such as tests, security scans, branch protections, and human review remain part of the delivery workflow.

Knowledge and execution

Agents also need access to the knowledge and environments required to complete engineering work. Context engineering supplies task-relevant information, agent memory can preserve useful state across longer-running work, and agent skills package repeatable instructions and domain knowledge.

The ecosystem is beginning to standardize some of these capabilities. Agent Plugins 1.0.0 provides a vendor-neutral specification for packaging Agent Skills and MCP servers into portable plugins.

Agents then execute work inside controlled environments where they can interact with repositories, dependencies, tests, browsers, and development services using defined credentials and permissions. Cloudflare describes agent infrastructure as spanning execution, communication, orchestration, memory, observability, and security.

Verification and feedback

Before agent-generated work moves forward, the factory needs to determine whether it meets the expected requirements. Tests, security and quality checks, specialized review agents, human review, and other evaluation methods can all contribute to that decision.

The results also create feedback for improving the factory. Recurring failures can reveal where models, context, skills, tools, or workflows need to change. This makes verification part of both the software delivery process and the continuous improvement of the system producing the work.

How Uber built a software factory at scale

Uber offers one of the clearest examples of what a software factory looks like at scale. In August 2026, the company reported that more than 70% of its pull requests were attributed to local or cloud agents, while weekly agentic requests had grown 9.4x between February and August.

Those agents now work across multiple stages of the SDLC. Uber describes AI handling code review, CI remediation, end-to-end pull request creation, visual validation, on-call triage, debugging, and code maintenance. A growing share of agent sessions are also initiated by managed systems, allowing engineering events to trigger work and route it through agentic workflows.

Uber has built a large layer of reusable organizational knowledge around these agents. The company reports more than 3,600 agent skills and 30,000 skill executions per day, capturing repeatable engineering practices that agents can apply across tasks.

Measurement is also part of the system. Uber tracks usage and cost across models, agents, users, requests, tokens, and caching. For managed agents, it goes a step further by measuring cost per engineering outcome, including cost per merged PR, review, alert, and cleanup.

At this scale, the engineering challenge expands beyond deploying coding agents. Teams have to operate and improve the system of agents, skills, infrastructure, verification, and measurement responsible for producing software.

How do software factories verify and improve agent output?

As agents produce more software changes, factories need reliable ways to determine whether that work is ready to progress. Verification can combine automated tests, static analysis, security scanning, specialized review agents, human review, and production signals.

Evals make this verification repeatable and provide the foundation for evaluation-driven development (EDD). Anthropic describes evals as tests of an AI system in which performance on a task is measured using defined grading logic. Teams can use those results to identify failures and improve the models, context, skills, tools, prompts, or harness responsible for them.

Faros's AI model routing evaluation framework provides one example of this approach. We replay real historical engineering tasks, score candidate patches against task-specific weighted rubrics, and use a blinded LLM judge to evaluate each patch against predefined criteria. Here, LLM-as-a-judge is one evaluation technique within the broader EDD process.

The value of this feedback loop extends beyond choosing a model. Our research on intelligent model routing found that the aggregate-winning route was the best choice on only 84 of 211 historical engineering tasks. Performance can depend on the model, harness, context, available tools, and criteria used to determine successful completion. Evals give teams a way to identify these patterns and test whether changes to the factory actually improve performance.

Run engineering tasks → evaluate output → identify failure patterns → improve the system → rerun evals → measure the change

Over time, EDD turns evaluation into a continuous improvement loop for the software factory itself.

How do software factories change the role of engineers?

As agents take on more implementation work, engineers spend more time defining and improving the systems that guide that work. GitHub describes this shift as moving from coder to orchestrator, with developers increasingly responsible for designing how agents operate within the software delivery process.

That can include defining specifications and acceptance criteria, engineering the context agents receive, building reusable skills and evals, setting permissions and human checkpoints, and handling exceptions that require deeper judgment.

The impact can extend beyond a single task. If an agent repeatedly struggles with database migrations, for example, an engineer can capture those failures in evals and improve the relevant skill, context, or workflow. Those changes can then improve how the factory handles future migrations.

Engineering productivity increasingly includes the ability to improve the system performing the work.

How do you measure a software factory?

A software factory can generate enormous amounts of activity without necessarily improving software delivery. Measuring its performance requires following the work from agent activity through agent effectiveness, engineering system performance, and ultimately software and business outcomes.

Start with agent activity

The first layer establishes how much AI-assisted engineering is taking place and what it costs. Organizations can track agent sessions, requests, tokens, tasks attempted, skill usage, model utilization, and AI spend.

These metrics are useful for understanding adoption and operating costs. They also provide limited information about whether the resulting work creates value. A rising number of agent sessions could represent productive automation, repeated failed attempts, or work that later requires substantial human rework.

Measure whether agents complete the work successfully

The next layer evaluates agent effectiveness. Metrics such as eval pass rate, first-pass success, task completion rate, retry rate, human intervention, and cost per successful task help determine which agents and configurations perform well for different types of engineering work.

Uber provides an example of pushing cost measurement closer to completed work. Alongside model and token usage, the company tracks outcome-denominated costs for managed agents, including cost per merged PR, review, alert, and cleanup.

This layer becomes especially useful when comparing models, harnesses, skills, or workflows. Teams can evaluate whether a change to the factory improves success rates, reduces intervention, or lowers the cost of completing a task.

Follow agent output into the engineering system

Agent effectiveness still captures only part of the picture. Software created by agents enters an engineering system where it must be reviewed, integrated, deployed, and operated.

That makes downstream metrics such as review time, cycle time, rework, deployment frequency, bugs, and incidents essential for understanding the factory's broader impact.

The 2026 Faros AI Engineering Report illustrates why. Across telemetry from more than 22,000 developers and 4,000 teams, higher AI adoption was associated with 33.7% more task throughput and 66.2% more epics completed. At the same time, bugs per developer increased 54%, incidents per PR increased 242.7%, and median PR review time increased 441.5%.

More agent output can therefore change the behavior of the entire delivery system. Increased implementation capacity may create new pressure on review, verification, testing, or production reliability. Measuring the factory requires visibility into those downstream effects.

Connect the factory to software and business outcomes

The final layer asks whether the software factory creates meaningful engineering leverage. That can include greater roadmap capacity, faster delivery of valuable software, improved reliability, or a lower cost of producing verified software outcomes.

This is where the hierarchy becomes important. An organization might simultaneously see:

Agent usage ↑ → agent task completion ↑ → PR volume ↑ → review time ↑ → incidents ↑

Looking only at the first two layers would suggest the factory is improving. Following the work through the engineering system reveals where additional output creates constraints or quality costs.

The same measurement system can then feed back into how the factory operates. Teams can identify recurring failures or bottlenecks, adjust models, skills, context, tools, verification, or workflows, and measure whether those changes improve downstream performance.

Agent activity → agent effectiveness → engineering performance → software outcomes → factory improvement

The performance of a software factory is ultimately determined by how effectively it converts AI activity into reliable, verified software outcomes.

What software factories mean for engineering leaders

As software factories mature, engineering leaders need visibility into how effectively the entire system turns AI resources into shipped software. That means understanding where agents create meaningful engineering capacity, where additional output creates constraints, and which configurations deliver the best results for the cost.

Three questions become especially important:

• Which models, agents, and workflows produce the best outcomes for different types of engineering work?
• Where are tokens being spent without producing useful or shipped work?
• Which changes to model routing, context, skills, and workflows improve quality, velocity, and cost?

Answering these questions requires connecting what happens inside AI coding agents with what ultimately happens to the code they produce.

Faros builds that connection across the software factory. Its Engineering World Model links token usage and agent sessions with tickets, commits, pull requests, CI results, and shipped outcomes. The Time Machine then evaluates alternative models, routes, and configurations against an organization's own historical engineering work to identify improvements grounded in its codebase. Faros can also apply policies for model usage, budgets, routing, and risk as agentic work runs.

Together, these capabilities create a closed loop for observing how the factory performs, finding opportunities to improve it, verifying those changes against real engineering work, and governing how they are deployed. As software factories scale, that ability to continuously measure and improve the system will determine how effectively organizations turn AI resources into reliable, shipped software.