After the fix: half the games on paired boards
Corrected search at depth 2 against the builders
95% intervalWhat the defect was
The first cohorts ran with a turn-planning defect. After any non-ending action, the enumerator reused the root's restricting action as its own action list, so it re-applied that action, failed, and explored nothing. A road, a bank exchange, or a purchase was therefore valued at the position right after it, without the opponents' round that every end-turn candidate received, and inner turns collapsed to ending the turn. Measuring the game tree exposed it: inner-level nodes averaged 1.17 end-of-turn positions where a hand with several options should produce many.
The correction limits the restriction to the root step. A second problem then surfaced: with sequences actually explored, the flattened list of end-of-turn positions averaged over the player's own alternatives instead of taking the best, and enumerating every sequence cost seconds. The planner is now a recursion that takes the maximum over own actions and the expectation over chance outcomes, keeps every first action but only the best few continuations of the current turn, follows one greedy line in later turns, and spends the lookahead on the afterstates of the chosen line.
Depth 3 wins more still
256-game engine cohort seated the same lineup with depth 3 under a 1.5 s budget. It won 147 of 256 slot-games (57.4%) against the compared ETA slot's 16.0%; the paired contrast is +0.414 (95% interval +0.327 to +0.502), above depth 2's +0.301 on the same boards. Decisions averaged 360 ms. With the defective search, depth 3 had bought nothing; with the turn tree working, the extra planned turn is worth about eleven more wins per 256.
Corrected search at depth 3 with a 1.5 s budget
95% intervalAgainst the reference it replaces
256-game engine cohort seated the corrected depth-2 search
with the frozen expectimax-v1, an ETA builder and a fast builder. It won
120 of 256 slot-games (46.9%); the reference won 34 (13.3%), ETA 69 and
fast 33. The paired contrast against the reference is +0.336 (95% interval
+0.249 to +0.423). The reference spent 240 ms per decision to the corrected
search's 88 ms.
Corrected search against the frozen reference
95% intervalWhat the corrected search does differently
The per-game means show a builder-like profile: 2.0 settlements, 2.1 cities, 4.7 roads and 2.8 development cards, with 6.2 completed player trades and 9.9 offers per game. The defective search offered less and, in its deep variant, bought more development cards than it built cities.
What each policy built per game
Measured evidenceScroll the chart horizontally to inspect all values.
Per-game means over 256 completed games per slot; the corrected search builds like ETA and trades with players more often.
View data table
Source: Engine-arena run record; per-game means over 256 completed games per slot.
| Series | Action | Per game |
|---|---|---|
| expectimax-v2 depth 2 | settlements | 2.02 |
| expectimax-v2 depth 2 | cities | 2.06 |
| expectimax-v2 depth 2 | roads | 4.71 |
| expectimax-v2 depth 2 | dev cards | 2.76 |
| expectimax-v2 depth 2 | player trades | 6.2 |
| expectimax-v2 depth 2 | bank trades | 4.27 |
| ETA builder | settlements | 2.31 |
| ETA builder | cities | 2.31 |
| ETA builder | roads | 4.75 |
| ETA builder | dev cards | 1.08 |
| ETA builder | player trades | 5.3 |
| ETA builder | bank trades | 5.29 |
| fast builder | settlements | 1.46 |
| fast builder | cities | 2.79 |
| fast builder | roads | 3.11 |
| fast builder | dev cards | 2.06 |
| fast builder | player trades | 5.34 |
| fast builder | bank trades | 5.44 |
Methods and reproduction
Registered experiments registered protocol (run
256-game engine cohort, depth 2) and
registered protocol (run
256-game engine cohort, depth 3 with "time_budget_ms":1500)
seat the search against eta, fast, eta; experiment
registered protocol (run
256-game engine cohort) seats depth 2 against v1, eta,
fast. The depth-2 configuration is
v2:{"depth":2,"samples":4,"scenarios":8,"inner_scenarios":4}; every cohort
uses seeds 0 to 63 with all four rotations. The paired
contrast is the per-seed mean of slot-0 minus slot-1 wins with a
normal-approximation interval over 64 seeds. Reproduce with
just engine-run EXPERIMENT after just build-arena; the manifest records the
tournament binary hash and both repositories' Git state (server commit
59433b7 carries the fix). Attribution: Claude Fable 5.1 (claude-fable-5-1)
through Claude Code.
Detailed result and cohort context
On 64 boards played once per seat rotation, expectimax-v2 at depth 2 won
129 of 256 slot-games (50.4%); the ETA builder in the compared slot won
52 (20.3%), the other ETA slot 38 and the fast builder 37. The paired seed
contrast against ETA is +0.301 wins per game (95% interval +0.190 to
+0.412). Decisions averaged 87 ms. This is engine-arena evidence; a protocol
cohort for the frozen candidate is registered.