Opus 4.5 → 4.6: What Changed in Practice?

Executive Summary

What Changes When You Switch

Across 529 statistical tests (overall, per-complexity, and cross-cut strata), 141 survive Bonferroni correction—21 at the overall level. Most describe how the model works, not whether it succeeds. The overall success rates are comparable; what changes is the experience of working alongside it. These are the five most noticeable differences, drawn from one user’s workflow over 2,837 tasks.

1. The model plans before it acts—and you stop steering it

Opus 4.6 uses formal planning mode on 12.3% of tasks vs 1.8% for 4.5, rising to 43% at complex and 65% at major difficulty. It front-loads codebase investigation with a 2.3× longer explore phase, deploying subagents that are 69% read-only researchers (vs 49% for 4.5). The practical effect is a shift from interactive collaboration to delegation: you issue a prompt and return to find completed work rather than course-correcting mid-task. This shows up in the data as fewer user-directed corrections across all complexity levels (§4, §8).

2. It thinks less often, but more carefully

The largest overall effect across all 529 tests is thinking fraction (d=0.64, medium, §3). Opus 4.5 activates extended thinking on 75% of requests regardless of difficulty; 4.6 activates on 59% but averages 4,067 characters when it does (vs 2,578). On trivial tasks, 4.6 often skips thinking entirely. On complex tasks, it thinks deeply. This calibration means compute is allocated where it matters rather than spread uniformly across every interaction.

3. Fewer rewrites, better first-attempt accuracy

Opus 4.6 rewrites its own edits 11.6% of the time vs 18.2%—a 36% reduction. Its self-correction rate is actually higher (3.5% vs 1.8%), meaning it catches its own mistakes rather than having the user point them out. Failure rates drop from 12.0% to 5.4%, and alignment scores improve significantly (p=0.000714, one of 21 overall Bonferroni survivors). The “plan first” approach appears to pay off in execution accuracy (§5, §6).

4. Sessions get longer—you trust it with more

Median task duration rises 46% (62s vs 42s), with fewer ultra-short interactions (34% of tasks under 30 seconds vs 42%). The task mix shifts toward moderate-to-complex work issued in a single instruction, and 4.6 runs more tasks in the background for parallel execution. This isn’t purely a model capability difference—it’s a workflow adaptation. When the model handles larger tasks reliably, the user gives it larger tasks, waits longer, and intervenes less. The 7× increase in planning mode (§4) and 44% more tool calls per task (§7) are partly a consequence of this delegation shift.

5. Cost stays flat where it counts

Despite 2.5× more output tokens and 44% more tool calls per task, 4.6 is 13–37% cheaper at trivial through moderate complexity—the bulk of daily work. The reason is counterintuitive: output tokens account for just 6.7% of per-task cost, while cache operations account for 93–97%. Opus 4.6 writes 29% less to cache (the most expensive token category at $18.75/MTok), more than offsetting its higher output and cache reads. Cost only tips higher at 30+ API requests, where cumulative cache reads compound past the write savings. Overall per-task cost is $2.56 vs $2.44: functionally neutral for a meaningfully different style of work (§2).

The data is consistent with a tentative characterization: Opus 4.5 acts first and adjusts, while Opus 4.6 investigates first and implements in concentrated bursts. The confounded study design means this framing is a hypothesis, not a conclusion. The analysis was iterative—several initial findings were revised or reversed when more direct signals became available (§10).

How This Report Works

This report is itself a Claude Code project. The analysis pipeline, statistical tests, table generation, and report assembly are all automated Python scripts, most written with substantial assistance from Opus 4.6—the same model being evaluated. LLMs are used in two places: task classification (Haiku annotates complexity, sentiment, and task type) and prose drafting. Most quantitative claims—numbers, tables, and statistical tests—are produced by deterministic computation, not LLM generation. All data comes from one user’s real Claude Code sessions during and after the Early Access Program—not synthetic benchmarks or controlled experiments.

A 12-step pipeline transforms raw JSONL session logs into the finished report. A few things worth noting about the approach:

• Edit timeline reconstruction: Rather than relying on LLM self-report, the pipeline builds per-file content ownership maps from every Edit/Write tool call, detecting when later edits overwrite earlier work. This mechanistic signal replaces the sentiment-proxy approach that proved unreliable in early iterations.
• Multi-signal sentiment: Dissatisfaction detection combines keyword patterns, structural edit signals (self-corrections, error recoveries), and LLM judgement with downgrade logic—no single source is trusted alone.
• Expression-bound output: Most numbers in this report resolve from analysis JSON via expression-based template variables ({{expr | format}}), and most tables are generated from spec files. This reduces transcription errors and makes it easier to keep prose in sync with the data as the analysis evolves. The pipeline can be re-run end-to-end to reproduce the report from raw session logs.

What to Trust

Most numbers, tables, and statistical results are computed deterministically from analysis JSON files and are reproducible by re-running the pipeline. The expression system binds prose to data paths, which helps catch drift but is not a guarantee of correctness. Interpretive prose was drafted with LLM assistance and may contain errors or overstatements. Effect sizes and p-values are exact; narrative claims linking those numbers to causal explanations are hypotheses, not conclusions. A sensitivity analysis validates key findings against restricted datasets excluding shared projects. The Methodology section describes every step in full detail.

1. Dataset at a Glance

All data comes from a single user's organic Claude Code sessions between December 2025 and February 2026. The dataset is intentionally asymmetric: Opus 4.5 served as the primary model for two months, while Opus 4.6 entered evaluation in early February. This means Opus 4.5 totals are larger in absolute terms, but per-task and per-session comparisons normalize for this. Where sample size limits statistical power, the report notes it explicitly.

The 13-day concentration of the Opus 4.6 data creates a temporal clustering concern: a productive stretch, a particular project focus, or simply the novelty of a new model could color all 937 tasks simultaneously. The report treats tasks as independent observations, but short collection windows make this assumption weaker for 4.6 than for 4.5’s 70-day span.

With the dataset in view, we turn to what the token data reveals about how each model allocates its computational budget.

2. Token Economy & Cost

Opus 4.6 costs ~4.9% more per task on average ($2.56 vs $2.44), despite producing 2.5× more output tokens and making more API round-trips (9.5 vs 7.4 requests/task). But this aggregate masks a complexity-dependent pattern: at trivial through moderate levels, 4.6 is 13–37% cheaper, driven by superior cache economics—not output efficiency. Output tokens account for less than 7% of per-task cost; cache operations account for ~93%. 4.6 achieves a leaner cache footprint, writing 29% fewer tokens at the most expensive token category. The cost advantage reverses at complex and major tiers, where accumulated cache reads over many requests outweigh the write savings.

Output Verbosity by Task Type

The 2.1× ratio for refactoring is notable: Opus 4.6 produces substantially more output tokens for refactoring tasks, suggesting more thorough changes. For continuation tasks (follow-ups within a session), Opus 4.6 produces 2.2× the output volume of Opus 4.5.

Cost by Complexity

Session-Hour Cost

Normalizing by session hours rather than task count: Opus 4.5 costs $12.57/session-hour ($5,222 over 415.4h) vs $6.74/session-hour for Opus 4.6 ($2,839 over 421.0h). Session hours measure wall-clock time from first to last message, so this metric includes idle time and is not a direct measure of active coding cost.

Per-Request Output

Per-request output survives Bonferroni correction (d=-0.30, small). Opus 4.6 produces more tokens per API round-trip at every complexity level, concentrating work into larger responses rather than many small incremental calls.

Why Output Doesn’t Drive Cost

The 2.5× output difference seems like it should dominate the cost comparison, but output tokens are a minor cost component. Cache operations dwarf everything else:

Three forces offset to produce the $0.12 net difference. 4.6 writes 29% fewer tokens to cache per task (48K vs 67K), saving $0.36 at the most expensive category ($18.75/M—10× the read price). But 4.6 reads 34% more cached context (792K vs 589K), costing $0.38 at the cheapest category ($1.875/M). And the 2.5× output increase costs only $0.10. The write savings nearly cancel the read and output increases: −$0.36 + $0.38 + $0.10 = +$0.12.

Why does 4.6 write less to cache? Per-request analysis reveals two mechanisms. First, 4.6 starts tasks with a leaner cache footprint—its first API request writes 50% fewer cache tokens than 4.5’s (24K vs 48K), suggesting a more compact or reusable context structure. Second, the first-request write penalty is steeper for 4.5—11.4× above its steady-state rate vs 7.2× for 4.6—so 4.5 pays a larger per-task initialization tax. The likely explanation: 4.6’s “investigate then execute” pattern creates more compact, reusable context, while 4.5’s incremental approach builds a larger accumulated context that costs more to establish.

This mechanism produces the complexity-dependent curve above. At trivial through moderate tiers (typically ≤30 API requests), lean initialization dominates and 4.6 is 13–37% cheaper. At major complexity, tasks average 50–70 API requests and cumulative cache reads compound past the initialization savings—4.6 reads 34% more cached context per task overall, and over enough round-trips this gap overwhelms the write savings.

Cross-Cut Detail

The cost difference raises a natural question: does 4.6’s different spending pattern correspond to different thinking strategies? The next section examines thinking calibration—the largest overall effect in the study.

3. Thinking & Calibration

Thinking fraction is the largest overall effect in the study (d=0.64, medium by Cohen’s convention). Opus 4.5 thinks on 75% of tasks but shallowly; Opus 4.6 thinks on 59% of tasks but more deeply when it does (4,067 vs 2,578 chars). The pattern suggests 4.6 has better calibration of when thinking is needed, reserving it for moderate-and-above complexity.

Calibration by Complexity

Thinking Depth by Task Type

Cross-Cut Detail

Thinking calibration is one manifestation of broader behavioral differences between the models. The next section examines other behavioral patterns—subagent deployment, planning adoption, and effort distribution.

4. Behavioral Patterns

Beyond token economics, the models differ in how they approach tasks. Opus 4.6 plans more often (12.3% vs 1.8% of tasks), deploys more subagents, and favors read-only exploration over general-purpose workers. These behavioral differences are among the most visible in the dataset, though the Claude Code platform itself evolved between the two collection periods—some of the shift may reflect SDK changes rather than model decisions.

Subagent & Planning Adoption

Subagent Type Distribution

Planning Adoption

Opus 4.6 enters plan mode on 12.3% of tasks (115 of 937) vs 1.8% for Opus 4.5. Adoption scales steeply with complexity: 42.9% at complex, 65.2% at major. Planned tasks show a modest alignment benefit (+0.17 overall) that diminishes at complex and major tiers.

Effort Distribution

Effort distribution shows Opus 4.6 allocates more tool calls to research (35.1% vs 28.3%) and fewer to implementation (17.5% vs 27.0%), consistent with the research-first approach visible in subagent type preferences.

Cross-Cut Detail

Different behavioral strategies raise the question of whether they lead to different outcomes. The next section examines completion rates, failure rates, and user satisfaction—the quality signals that the behavioral patterns should ultimately serve.

5. Quality & Satisfaction

LLM-annotated alignment scores (1–5 scale) show Opus 4.6 scoring higher on average, an effect that survives Bonferroni correction (p=0.000714, d=-0.13). The failed rate difference is also notable: 5.4% of 4.6 tasks fail vs 12.0% for 4.5 (p=0.000, significant at FDR but not Bonferroni). Both alignment and failure rate are LLM-classified—a Claude Haiku model reads each session transcript and assigns scores. The “LLM quality judgement” approach was abandoned as unreliable (see §10), but alignment scoring proved more robust because it rates user-goal correspondence from observable signals rather than attempting to judge code quality directly.

Two categorical distributions—task completion and communication quality—also survive Bonferroni as chi-square tests, indicating the models differ in how they reach outcomes, not just in outcome rates. Note that the completion distribution test (p=0.000000) survives, while the individual completion rate proportion test (p=0.000) is marginal. All chi-square tests carry a low-expected-cell-count warning due to rare categories in the 20-status taxonomy.

Completion Distribution

Sentiment Distribution

Satisfaction trends higher for 4.6 but does not survive Bonferroni correction. Dissatisfaction rates are essentially tied. Both completion and sentiment are LLM-classified: a Claude Haiku annotator reads the full session transcript for each task, classifying completion status from a 20-category taxonomy and inferring user sentiment from contextual signals (follow-up messages, tone shifts, task abandonment patterns). These classifications were validated through human spot-checks of flagged cases, but no formal inter-rater reliability was computed.

Cross-Cut Detail

Quality metrics paint a consistent-but-modest picture: 4.6 fails less and scores higher on alignment, but effect sizes are small (d=0.13) and the LLM-classification methodology adds a layer of uncertainty. The next section asks whether these quality differences manifest in the editing process itself.

6. Edit Accuracy

Edit timeline analysis tracks every Edit and Write tool call, building per-file content ownership maps to detect when a model later overwrites its own earlier output. Opus 4.5 rewrites 18.2% of its edits vs 11.6% for Opus 4.6. Overlap classification reveals the rewrites are predominantly iterative refinement (64% for 4.5, largest category), not error recovery.

Overlap Breakdown

The overlap composition tells a more nuanced story than the headline rewrite rate. When Opus 4.6 does overlap, a larger share is self-correction (30.4% vs 10.1% for 4.5)—meaning 4.6 catches and fixes its own mistakes more explicitly. Opus 4.5’s overlaps are more heavily iterative refinement (64% vs 59%), suggesting gradual adjustment rather than correction. Error recovery rates are comparable (15.2% vs 10.3%).

Cross-Cut Detail

Edit patterns capture one dimension of how the models work; the next section broadens the lens to overall resource usage and complexity scaling.

7. Complexity & Resource Usage

Task Distribution by Complexity

Opus 4.6 sessions skew toward higher complexity: fewer trivial tasks (37% vs 46%) and proportionally more moderate tasks (26% vs 22%). This makes raw aggregate comparisons misleading—Opus 4.6 is tackling harder work on average.

Resource Usage

Cross-Cut Detail

Tool calls and tools/file are classified under the “behavior” theme in the cross-cut analysis. Their per-complexity, per-task-type, and per-iteration breakdowns appear in §4’s cross-cut detail (Behavioral Findings). Key results: the tool-call gap is largest for significantly-iterated tasks (d=0.53) and trivial complexity (d=0.30), both Bonferroni-significant.

The preceding sections examined behavioral, quality, and resource dimensions. The next section examines temporal patterns—how performance unfolds within and across sessions.

8. Session Dynamics

Task duration survives Bonferroni correction (p=0.000001), though the effect size is negligible (d=0.005)—a case of statistical significance without practical significance, driven by sample size. Opus 4.6 takes longer per task (median 62s vs 42s, a 46% increase). The explore phase runs 2.3× longer at median (71.0s vs 31.3s). Effort distribution shows 4.6 allocates more tool calls to research (35.1% vs 28.3%) and fewer to implementation (17.5% vs 27.0%). Active-time cost is $27.48/hour for 4.6 vs $25.52/hour for 4.5 (5-min idle threshold).

Task Duration

Session Length & Warmup

Active-Time Cost

Context Compaction

Context-window compaction occurs in 9.8% of 4.5 sessions (32/327) and 11.7% of 4.6 sessions (22/188). Pre/post comparisons show improvement after compaction, but a position-adjusted control group—splitting non-compacting sessions at the median compaction position to isolate position effects—reveals the effect is driven by session position, not compaction itself (position-adjusted effect: −0.17 for 4.5, −0.17 for 4.6). Compaction appears to preserve rather than degrade performance.

Cross-Cut Detail

Duration is classified under the “behavior” theme in the cross-cut analysis. Per-task-type and per-iteration breakdowns appear in §4’s cross-cut detail (Behavioral Findings). Key results: the duration gap is largest for significantly-iterated tasks (median 99.0s vs 45.5s) and investigation tasks (median 79.9s vs 41.0s), both Bonferroni-significant. Effect sizes are negligible (d<0.1) despite significance—driven by sample size, not practical magnitude.

Session dynamics reveal a temporal dimension to the behavioral differences. The next section synthesizes all dimensions into overall model profiles.

9. Model Profiles

Observed Patterns by Task Type

10. Methodology

This section documents how each pipeline step works. Each step includes a summary of the approach and a collapsible detail block with thresholds, algorithms, and parameters.

Task Extraction & Classification

Each Claude Code session was segmented into tasks at user-message boundaries. An LLM annotator (Haiku) then classified each task for complexity, type, sentiment, completion status, and alignment score (1–5 scale). Behavioral metrics—subagent usage, planning, parallelization—were extracted directly from tool-call logs.

Cross-Cut Dimensions

All 529 statistical tests are run both at the overall level and stratified across three cross-cut dimensions. Each section’s “Cross-Cut Detail” expansion shows how its metrics behave under each slice.

Edit Timeline Analysis

The edit timeline reconstructs a per-file content ownership history from every Edit/Write tool call across all sessions. When a later edit’s old_string overlaps with content placed by an earlier edit, a rewrite is detected—providing a mechanistic signal for self-correction that doesn’t depend on sentiment classification.

Compaction Analysis

Claude Code compacts conversation context when token limits approach. This analysis measures whether compaction degrades task outcomes or merely correlates with session position.

Statistical Testing

529 tests were conducted across overall and per-complexity strata, using Bonferroni correction—the most conservative standard—to minimize false positives given the observational design.

Sensitivity Analysis

To validate robustness, all overall-level tests were re-run on a restricted dataset excluding overlapping projects. This tests whether findings depend on specific project mix or are stable across the data.

Ranked Findings