A draft-aware opening planner, calibrated to the leaf
The calibrated planner after correcting seating
Measured evidenceScroll the chart horizontally to inspect all values.
The calibrated planner and search placement have no measured strength difference.
View data table
Source: Source: reports/opening-planner.mdx, swapped-pair results; runs run record, run record, run record, run record. Every 64-board stage plays both seatings with four rotations (512 games); pooled estimate combines 128 boards. Reported 95% intervals.
| Series | Comparison | Change in wins per game | Low | High |
|---|---|---|---|---|
| Planner minus search placement | Boards 0–63 | -0.006 | -0.087 | 0.075 |
| Planner minus search placement | Boards 64–127 | -0.027 | -0.111 | 0.057 |
| Planner minus search placement | Pooled 128 boards | -0.017 | -0.075 | 0.042 |
What changed
The tunable search used to rank setup sites by the static leaf, keep six, and
value each through a two-turn lookahead that includes the first rolls. That
ranking is dominated by raw production, so the pairs it chose covered the
island badly: eleven brick pips and no grain on one traced board, no ore on two
others. The planner in crates/expectimax/src/v2/opening.rs replaces it. It
scores a pair of sites as a portfolio with seven terms: production per
resource with a scarcity markup, coverage of distinct dice numbers, a penalty
for resources left thin, the time to afford a settlement with a road and a
city with surplus converting at bank and port ratios, ports, expansion within
two or three roads, and the value lost to the draft. A first placement is
valued by the best pair it still leaves after the seats ahead in the snake
draft have picked, predicted with the same scoring; the last seat plans both
placements at once; roads head for the best site that survives the remaining
picks. Every candidate carries its terms, the strategy that values it, and the
expected partner site, so a report explains a placement rather than only
choosing it.
The calibration
Each row is one unregistered screen of 256 games (boards 0 to 63, four rotations) with the planner-decided seat against the previous placement at depth 2, plus an ETA and a fast builder.
| Setting | Planner | Search | Contrast |
|---|---|---|---|
| pip weights, expansion 0.5 (first draft) | 23.4% | 53.5% | −0.301 |
| leaf weights, specialize 0.5, scarcity 0.5, expansion 0.15 | 28.5% | 44.9% | −0.164 |
| calibrated, specialize 0, scarcity 0 | 47.3% | 37.9% | +0.094 |
| calibrated, specialize 1, scarcity 0 | 31.6% | 47.7% | −0.161 |
| calibrated, specialize 0, scarcity 1 | 46.9% | 35.2% | +0.117 |
| calibrated, specialize 0.5, scarcity 0.5 | 42.2% | 42.6% | −0.004 |
| calibrated, coverage 0.4, balance 0.5 | 46.1% | 37.1% | +0.090 |
Two lessons. First, the leaf's preference for ore and grain, which the search
had learned to win with against builders, is right in this arena, and a
valuation that rewards spreading production across all five resources is
wrong by a wide margin. Second, the five discrete strategies (coverage,
cities, roads, ports, cards), blended in by specialize, hurt at every
strength because taking the best specialized valuation inflates concentrated
pairs of the wrong kind. The strategies remain in the code as a research knob
with the default at zero.
Registered cohorts
The registered protocol, run 256-game engine cohort, boards 64 to 127, four rotations, all 256 games complete: planner in slot 0, 110 wins (43.0%); previous placement in slot 1, 94 (36.7%); paired contrast +0.062 (−0.036 to +0.161). The registered rule asked for an interval excluding zero and was not met.
The same day the bargaining arms found that two identical searches in slots 0 and 1 differ by seating alone (arms report), so the comparison was registered again as swapped pairs on the same boards, planner in slot 0 and then the previous placement in slot 0, with the effect taken as half the difference of the two contrasts and the seating term as half their sum.
| Boards | Planner in slot 0 | Previous placement in slot 0 | Planner effect | Seating term |
|---|---|---|---|---|
| 0 to 63 | +0.051 | +0.062 | −0.006 (−0.087 to +0.075) | +0.057 |
| 64 to 127 | +0.043 | +0.098 | −0.027 (−0.111 to +0.057) | +0.070 |
| pooled, 128 boards | −0.017 (−0.075 to +0.042) | +0.063 (+0.008 to +0.119) |
Every single-seating number above, including the calibration sweep, carried that seating term. The calibration ordering still stands (the arms were run in the same seating, and the pip-only draft lost by far more than any seating term), but the claim that the calibrated planner beats the search does not.
What could still be wrong
Every game here is against fixed builders that never block and rarely trade, which is exactly the setting where an ore-and-grain start is safest. Against opponents who contest ground the coverage and expansion terms may deserve more weight than this calibration gives them; the knobs exist for that study. The draft prediction assumes opponents place by the same planner, which is exact for the builders and the search but not for a person.
Methods and reproduction
Screens were run with cargo run --release -p settlers-arena --bin tournament -- --seat 'v2:{"depth":2,"opening_weights":{...}}' --seat 'v2:{"depth":2,"opening":"search"}' --seat eta --seat fast --seeds 0-63 --rotations 4 --deterministic and are not filed as records. The confirmation
is filed under 256-game engine cohort; the
swapped pairs are runs 256-game engine cohort,
256-game engine cohort, 256-game engine cohort,
and 256-game engine cohort under the same study. The
planner's terms and settings are documented in the server's
docs/expectimax.md under Opening placement.
Detailed result and cohort context
On 64 deterministic boards played once per seat rotation, a planner that valued setup sites in pips with a player's sense of balance lost to the search's own placement by −0.301 wins per game (95% interval −0.414 to −0.188). Calibrated to the leaf's resource values, with its structural terms shrunk to tie-breakers and its discrete strategies switched off, it appeared to win by +0.09 to +0.12 in single-seating screens and by +0.062 on fresh boards. Measured in both seatings on 128 boards, that gain is seating: the planner effect is −0.017 wins per game (−0.075 to +0.042) and the seating term +0.063 (+0.008 to +0.119). The planner is as strong as the search's placement, and unlike it every placement is balanced and explained.