TL;DR:
What we were trying to find out: Coming off the back of our recent routing simulation experiment (more on this below), how would GLM 5.2 hold up in real engineering workflows?
Experiment we ran: We implemented a five-day policy that routed direct Claude Code Opus 4.8 requests to GLM 5.2. We wanted to see if GLM 5.2 earned a broader rollout by observing the differences in cost, workflow experience, operational friction, and PR-linked work.
Results: GLM cut direct-route request cost by 78.1%. Among survey respondents with confirmed GLM exposure, median result quality sat at the numerical midpoint between “much worse than usual” and “much better than usual.” A small number of image-reasoning-state sessions required manual recovery, so I paused the broad default pending another monitored validation.
Outcome: It did not earn a broader rollout: image-bearing workflows with Claude Code reasoning state highlighted a compatibility boundary in the route.
GLM 5.2 direct-route cost fell 78.1%, but a compatibility boundary blocked scaling
Across 7,015 attributed direct GLM requests in a July 27–31 rollout, Fireworks recorded $225.04 of provider-rated GLM cost, or $0.03208 per request. A separately defined July 6–10 Opus reference period recorded $0.14620 per direct request in Claude Code OpenTelemetry.
The result was useful but incomplete. In the GLM 5.2 route, I found a compatibility boundary around image input and Claude Code reasoning state. It required manual recovery in the affected workflow, so I paused the broader GLM default rather than treating a lower direct-request cost as sufficient.
Earlier this year, Faros Field CTO Ron Meldiner used our proprietary Time Machine analysis to compare seven model-and-harness routes across 211 historical Faros engineering tasks. The study reported the Claude Code + GLM 5.2 route as competitive on its quality measures while being cheaper and faster in that task cohort. Read Ron's Time Machine study.
That result was a reason to test the route in real engineering work. I led the field follow-up: keep the Claude Code workflow familiar, change the backend behind the Opus selector, and observe what happened to cost, reliability, and developer experience.
I used a five-day GLM-default routing policy for the field-validation window. The pilot had a narrow job: determine whether a lower-cost route had earned a wider, monitored deployment.
Testing whether GLM 5.2 earned a broader rollout
On July 24, before the bounded rollout began, I set one economic rule: GLM had to reduce cost per direct eligible request by at least 40% against a separately defined Opus reference period. Cost alone would not earn a wider rollout. The route also had to work cleanly enough in ordinary Claude Code sessions that its savings were operationally usable.
The 40% threshold was dated and locked on July 24. The compatibility findings were operational release criteria surfaced during the field validation.
Method: eligible Opus 4.8 requests routed to GLM 5.2 via a LiteLLM gateway
The client-facing Claude Code selector was held constant. During the bounded field-validation week, the eligible Claude Opus 4.8 selector defaulted to the same GLM 5.2 backend route for all 28 participants. All other model selectors (Fable 5, Sonnet 5, Haiku 4.5, etc.) continued to route requests to the user-selected model.
I implemented the route through a LiteLLM gateway: developers kept the familiar Claude Code selector while the gateway applied the backend policy and recorded aggregate routing behavior.
Population: All 28 participants
Developer-facing selector: Claude Opus 4.8
Default backend: Fireworks-hosted GLM 5.2
GLM 5.2 does not accept image inputs, so I configured an Opus 4.8 compatibility fallback for those requests. That handled the basic image incompatibility. However, a subset of image requests also carried Claude Code reasoning state that the fallback could not be configured to accept in LiteLLM, so those requests failed after falling back.
The fallback applied only to the image-bearing request. Later requests returned to the GLM default unless they also required image handling. Participants received a private gateway key and installation instructions, were told that Claude Code traffic would route through a LiteLLM gateway, and continued using the familiar Claude selector. The rollout notice did not disclose the per-request backend mapping.
The study included 28 participants: 18 Engineering and 10 field colleagues. The GLM-default routing policy applied to all enrolled participants, while 13 generated at least one attributable direct GLM request during the five-day window. Those 13 are the active population behind the direct-route economics below.
Measuring cost, workflow experience, operational friction, and PR-linked work
The live pilot complements the Time Machine evaluation rather than attempting to reproduce it. Real usage introduces context, tool calls, developer preferences, code review, and operational failures that a historical task replay cannot fully simulate.
I used provider-rated cost for direct GLM economics, a confidential five-day survey for developer-reported experience, route logs for operational friction, and two author-approved PR examples for workflow context. The appendix defines the source boundaries and attribution rules.
Direct GLM 5.2 requests were 78.1% cheaper than the historical Opus 4.8 reference
The comparison below gives the cost result its operating context. It is direct-route unit economics: image requests resolved through the Opus compatibility fallback are reported as route behavior below, not folded into this figure.
Fireworks is the source of truth for GLM dollars and billed-use tokens. I cross-checked those records with routing and session telemetry to validate rollout attribution. The historical Opus reference is sourced from Claude Code OpenTelemetry. This is a historical reference, not a concurrent control: it answers whether the direct GLM route was economical enough to keep testing, not whether GLM caused different engineering outcomes.
Survey sentiment showed GLM 5.2 at comparable quality, but mixed reliability
The survey was sent at the end of Friday, July 31, and asked participants to answer specifically about Claude Code use from Monday, July 27 through Friday, July 31. Sixteen of 28 participants responded. Twelve had at least one successfully resolved GLM 5.2 response, so they are the primary population for interpreting route experience. On the quality scale, 1 meant “much worse than usual” and 7 meant “much better than usual”; confidence and reliability used the analogous “not at all” to “extremely” endpoints.
The confirmed-exposure group’s median quality was 4, the numerical midpoint between “much worse than usual” and “much better than usual.” The distribution below uses only those 12 respondents.
.webp)
Within confirmed exposure, quality concentrated at 4. Confidence was somewhat higher, while reliability had a wider spread. Across all free-text responses, four respondents mentioned gateway or client-integration friction, and four mentioned agent-workflow concerns such as verification loops or difficulty with complex tasks. Those themes overlap. They are why the cost result did not become a broad-default decision.
An image compatibility issue prevented a broader rollout
The gateway route-audit export contained 8,821 routing rows across 175 sessions. Most provider responses resolved to GLM. Thirteen percent resolved through the approved Opus fallback, and 17 sessions used both routes.
The first two rows partition the 8,821 returned provider responses. The last two count developer sessions. Every returned row was marked as a successful provider response, but that confirms transport success, not necessarily an uninterrupted developer workflow. The error logs represented 10 image-input failures across four sessions and 14 clear_thinking failures across six sessions; three sessions contained both signatures.
For the affected developer, the workflow was recoverable but not seamless. Claude Code’s /rewind command could return a session to the point before the incompatible reasoning-state request, allowing work to continue without the image; new sessions worked as usual. Those were manual recovery steps interrupting development work, which is why this issue blocked a broad default rollout, especially considering how integral image input has become to agentic coding.
PR-linked work provided workflow context
I also reviewed two author-approved, commit-confirmed examples: one from the historical Opus period and one from the GLM-default routing policy. They show how the agent participated in normal engineering loops of investigation, human design judgment, testing, review, and merge. They are workflow context, not matched evidence of code quality or productivity.
Historical Opus reference: diagnose, challenge, then verify a performance fix
One historical session began with a reproducible cold-start timeout in a scoring workflow. Claude Code helped measure the bottleneck and propose a different query path, while the developer continued to question the design and test strategy. Before shipping, the developer asked for an adversarial review. That review surfaced two substantive risks in the initial approach: an unbounded memory and transfer path for a limited query, and a correctness edge case for sessions whose summary records lagged. The final PR addressed both, added two targeted tests, and passed the relevant end-to-end and package test suites.
GLM-default routing policy: trace an identity mismatch across operational systems
One GLM-routed session began with a vulnerability-automation failure after a service's deployment identity and container-artifact identity had diverged. The session moved from the immediate lookup failure through the deployment and source-control conventions that produced it. The developer tested the tempting rollback option, then chose the narrower design: preserve the canonical service identity and translate the legacy artifact name in the automation path. The resulting three-file change was reviewed and merged.
Decision: broad GLM 5.2 default paused pending a second monitored validation
The direct GLM route cleared the pre-specified cost threshold by a wide margin. The GLM-default routing policy also exposed real compatibility and workflow friction, particularly around image-capable sessions and Claude Code request state. That is a limitation of this policy, not evidence that GLM is uniformly unreliable.
I am pausing the broad GLM route pending another monitored validation. That validation could perhaps test a capability-aware policy that limits GLM to workflows without image input, or a separately qualified image-capable model. It should retain the same cost measurement, add explicit compatibility release criteria, and observe downstream engineering outcomes for longer.
Ron’s historical study established why local evidence matters. This live follow-up made the next lesson concrete: a lower-cost route earns its place through both the economics it delivers and the workflows it preserves.
Five design considerations for a bounded routing pilot
If you are considering a lower-cost coding route, start with a bounded pilot:
- Keep the developer-facing workflow stable and change one route at a time.
- Set a cost threshold before reviewing the result.
- Exercise capability boundaries such as images, tools, and reasoning state before broadening the audience.
- Measure developer-reported experience alongside cost and route-level behavior.
- Review a small number of attributable PRs to understand what the aggregate metrics cannot show.
Faros can help you easily run your own routing experiment. Contact us for more information.
Appendix: data sources, attribution rules, units, and exclusions
The GLM-default routing window was July 27–31, 2026 PDT; the native-Opus reference window was July 6–10, 2026 PDT. The periods have different task mix, calendar time, and participation, so the historical Opus values provide context for a local routing decision rather than a concurrent causal model comparison.
An earlier internal split-routing dry run validated gateway operation, but uneven usage meant it was not used for the published cost, experience, or workflow findings.
For the primary economic result, GLM cost per eligible request is Fireworks provider-rated GLM cost divided by 7,015 attributable direct-GLM Claude Code OpenTelemetry events. The Opus reference is Claude Code OpenTelemetry–recorded native Opus cost divided by 10,756 eligible native Opus events. These are different source boundaries, which is why the 78.1% result is directional rather than a provider-billed invoice-to-invoice comparison.
Direct GLM economics exclude requests resolved through the image-only Opus fallback; that fallback was a compatibility mechanism, not part of the direct GLM unit-cost calculation. Fallback and mixed-route sessions are reported as route behavior. The Claude Code OpenTelemetry dataset used for economics contains 7,015 eligible direct-GLM events across 91 sessions, all of which were found in the approved gateway-session population. The survey is confidential, aggregate-only self-report data from one five-day workweek; the 12-person confirmed-exposure subset provides the primary route-experience view. PR-linked figures use Claude Code OpenTelemetry-attributed cost for both routes under a commit-confirmed attribution rule and are not provider invoices.
For PR-linked analysis, a candidate required commit-confirmed session-to-PR evidence: 97 of 214 historical Opus workflow sessions and 28 of 74 GLM-only workflow sessions met that rule. One session can be connected to more than one repository-plus-PR identity, yielding 92 historical Opus and 39 GLM-only PR candidates. In the primary attributed-cost distribution, I retained only sessions linked to one commit-confirmed PR: 50 historical Opus PRs had a median OpenTelemetry-attributed direct-route cost of $5.24 (IQR $1.72–$12.23), while 17 GLM-only PRs had a median of $1.14 (IQR $0.54–$2.79). Those samples are not matched work.
.webp)