Reading the next build
The promising development result did not survive fresh boards
Measured evidenceScroll the chart horizontally to inspect all values.
All three fresh-board intervals include no change in win rate.
View data table
Source: Source: reports/build-prediction.mdx, How the cohorts were run; analysis/build_prediction.py. Twelve registered cohorts, all 3,072 games complete. Each estimate combines both seatings on 64 boards with four rotations per half (512 games per arm and stage); 95% intervals over boards.
| Series | Comparison | Change in wins per game | Low | High |
|---|---|---|---|---|
| Development: 64 boards | Deny predicted sites | 0.061 | 0.009 | 0.112 |
| Development: 64 boards | Predicted opponents | 0.043 | -0.007 | 0.093 |
| Development: 64 boards | Predicted threat | 0.039 | -0.006 | 0.084 |
| Fresh boards: 64 boards | Deny predicted sites | -0.025 | -0.071 | 0.02 |
| Fresh boards: 64 boards | Predicted opponents | -0.039 | -0.1 | 0.022 |
| Fresh boards: 64 boards | Predicted threat | 0.031 | -0.024 | 0.087 |
The mechanism
expectimax-v2 models other seats one way: inside the lookahead they follow a
fixed builder on sampled hands, and outside it the bargaining threats read points and
a shared frontier scan. The study added one predictor and three switches, each off
by default, in crates/expectimax/src/v2/predict.rs:
| Switch | What it does |
|---|---|
weights.denial | Discounts a leaf by the production value of an open site another seat's predicted line is heading for, divided by one plus that seat's turns to afford the whole line |
predict.opponents | The lookahead's opponent model completes the predicted piece when affordable, trades toward it with surplus, or saves; the builder's detours to dev cards and off-target pieces are gone |
predict.threat | The bargaining space threat keys on the other seat's predicted target site instead of scanning this seat's best three sites for reachability |
The predictor itself runs the builders' own greedy target choice on a hand and the public board: the next piece the line completes (the first road toward the expansion site, the site settlement, a city, or a development card), the site that line ends up taking, and the turns until the piece and the whole line are affordable. Inside the search the hand is the seat's sampled hand, so every prediction is an expectation over the searcher's belief; the arena diagnostic replaces it with a public-knowledge estimate. The deployable policy never reads a hidden hand: its samples come from its own observation and event history.
How the cohorts were run
Each arm ran as a mirrored pair of registered deterministic cohorts, 64 seeds and four rotations (256 games) each: treatment slot 0 against the unchanged depth-2 control in slot 1, and the same two searches with the slots swapped. The average of the two order-paired per-seed contrasts is the mechanism estimate, because slot 1 always sits immediately after slot 0 in engine seat order and that order alone moves wins. An ETA and a fast builder held the other two seats. All 3,072 games completed; there was no censoring or invalid game.
| Arm | Development, seeds 0–63 | Confirmation, seeds 64–127 |
|---|---|---|
Denial leaf term (weights.denial 0.5) | +0.061 (+0.009 to +0.112) | −0.025 (−0.071 to +0.020) |
| Predicted opponent model | +0.043 (−0.007 to +0.093) | −0.039 (−0.100 to +0.022) |
| Predicted space threat | +0.039 (−0.006 to +0.084) | +0.031 (−0.024 to +0.087) |
Only the denial term met its development rule, and its fresh-seed reading refutes the development estimate rather than confirming it: over both seed sets the pooled estimate is about +0.018, indistinguishable from zero at this scale. The predicted opponent model and the predicted threat stay inconclusive in both directions. No default changes; the switches remain research knobs. The depth-2 hand-written leaf, the fixed builders, and the single lineup bound every number here, and the twin search in slots 0 and 1 is the strongest rival at the table, which is exactly the opponent the predictor reads worst.
Does the prediction read opponents?
The arena measured the predictor honestly from one observer seat's public knowledge:
at each opponent's first main-phase decision of a turn it recorded the predicted next
build and compared it with the first build that seat actually completed in the same
turn, over 64 games (seeds 0 to 15, --predict-diag 0).
| Opponent | Predictions matching the next completed build |
|---|---|
| fast builder | 281 of 401 (70.1%) |
| eta builder | 246 of 467 (52.7%) |
| depth-2 search | 310 of 1,089 (28.5%) |
The predictor reads builder-like opponents well and searches badly, which explains the result: the seat that wins these tables is the other search, whose next build depends on trades, development cards, and blocking choices no greedy line predicts. Builder-directed terms can only matter against the builders, who are the weakest seats at this table.
One mechanism signal did move. The denial seat lost about 0.1 fewer frontier sites
per game than its control in all four denial cohorts (1.75 against 1.85 or so, from
the new sites_lost counter, which counts opponent settlements built on a vertex the
seat could reach within three roads). The term changes the board in the direction it
claims without changing who wins measurably.
The slot-order asymmetry
A control cohort of two identical depth-2 searches, ETA, and fast on seeds 0 to 63 split 129 to 85 for slot 0: +0.172 wins per game (95% interval +0.083 to +0.261) for the seat that always acts immediately before its twin, with identical policies. On seeds 64 to 79 the same pair split 30 to 29. The asymmetry is a property of the board lottery on the development seed set, not of any policy: the four rotations seat slot 1 immediately after slot 0 in engine seat order in every one of them.
This matters beyond this study. The single-order ablation contrasts reported earlier, all on seeds 0 to 63 with the same lineup shape, carry the same slot bonus; a treatment that did nothing would have shown about +0.17 there. The near-zero ablation arms were likely slightly negative, and the decisive levers stand well above the bonus. Future paired contrasts should either run mirrored pairs, as here, or report the twin-seed slot baseline alongside. This cohort is an unregistered diagnostic; the registered evidence in this report is the twelve mirrored cohorts.
Methods and reproduction
Twelve registered deterministic cohorts under adaptation/build-prediction,
each 64 seeds, four rotations, 256 games, depth 2 with the hand-written leaf and
an ETA and fast builder in slots 2 and 3, eight threads per tournament.
Development seeds 0 to 63: denial registered protocol (run
256-game engine cohort) and mirror registered protocol
(256-game engine cohort); predicted opponents
registered protocol (256-game engine cohort)
and mirror registered protocol (run
256-game engine cohort); predicted threat
registered protocol (256-game engine cohort)
and mirror registered protocol (run
256-game engine cohort). Confirmation seeds 64 to 127: denial
registered protocol (256-game engine cohort)
and mirror registered protocol (run
256-game engine cohort); predicted opponents
registered protocol (256-game engine cohort)
and mirror registered protocol (run
256-game engine cohort); predicted threat
registered protocol (run
256-game engine cohort) and mirror
registered protocol (256-game engine cohort).
Every cohort completed 256 of 256 games. The mechanism estimate for each arm is the
per-seed average of the two order-paired contrasts; analysis/build_prediction.py
computes it, the slot-order effect, sites_lost, and the prediction match rates from
the retained games.jsonl. Screens (seeds 0 to 15, denial 0.25, 0.5, and 1.0, both
other switches, control pairs on 0 to 15, 64 to 79, and 0 to 63, and the
--predict-diag run) are unregistered diagnostics under
runs/screens-build-prediction/. The switches are documented in the server's
docs/expectimax.md; cargo test -p settlers-expectimax covers the predictor, the
denial term, the guided opponent, and the predicted threat. The slot-order cohort
used v2:{"depth":2} in slots 0 and 1 with eta and fast; its interval is a
normal approximation over 64 paired seeds. Reproduce any cohort with
just engine-run EXPERIMENT. Attribution: GLM-5.3 (baseten/zai-org/GLM-5.3) through
opencode.
Detailed result and cohort context
A denial leaf term that discounts open sites opponents are predicted to take
reached +0.061 wins per game on development seeds 0 to 63 (95% interval
+0.009 to +0.112) and fell to −0.025 (−0.071 to +0.020) on fresh seeds 64 to
127. The predicted opponent model and the predicted space threat never separated
from zero in either cohort. No switch becomes a default. The same cohorts measured
a slot-order asymmetry in the standard paired ablation lineup: on seeds 0 to 63 the
seat that always acts immediately before its twin won +0.172 of the identical
twin pair's games (95% interval +0.083 to +0.261), so single-order contrasts on those
seeds carry roughly a sixth of a win per game of slot advantage.