Settlers / Research

75 pages · Search titles and descriptions

↑ ↓ to navigate · Enter to open · Esc to closeLocal search
Play the game

A draft-aware opening planner, calibrated to the leaf

The calibrated planner after correcting seating

Measured evidence
-0.13-0.0725-0.0150.04250.1Boards 0–63Boards 64–127Pooled 128 boardsComparisonChange in wins per gameNo difference

Scroll the chart horizontally to inspect all values.

Planner minus search placement95% intervalNo difference

The calibrated planner and search placement have no measured strength difference.

View data table

Source: Source: reports/opening-planner.mdx, swapped-pair results; runs run record, run record, run record, run record. Every 64-board stage plays both seatings with four rotations (512 games); pooled estimate combines 128 boards. Reported 95% intervals.

SeriesComparisonChange in wins per gameLowHigh
Planner minus search placementBoards 0–63-0.006-0.0870.075
Planner minus search placementBoards 64–127-0.027-0.1110.057
Planner minus search placementPooled 128 boards-0.017-0.0750.042

What changed

The tunable search used to rank setup sites by the static leaf, keep six, and value each through a two-turn lookahead that includes the first rolls. That ranking is dominated by raw production, so the pairs it chose covered the island badly: eleven brick pips and no grain on one traced board, no ore on two others. The planner in crates/expectimax/src/v2/opening.rs replaces it. It scores a pair of sites as a portfolio with seven terms: production per resource with a scarcity markup, coverage of distinct dice numbers, a penalty for resources left thin, the time to afford a settlement with a road and a city with surplus converting at bank and port ratios, ports, expansion within two or three roads, and the value lost to the draft. A first placement is valued by the best pair it still leaves after the seats ahead in the snake draft have picked, predicted with the same scoring; the last seat plans both placements at once; roads head for the best site that survives the remaining picks. Every candidate carries its terms, the strategy that values it, and the expected partner site, so a report explains a placement rather than only choosing it.

The calibration

Each row is one unregistered screen of 256 games (boards 0 to 63, four rotations) with the planner-decided seat against the previous placement at depth 2, plus an ETA and a fast builder.

SettingPlannerSearchContrast
pip weights, expansion 0.5 (first draft)23.4%53.5%−0.301
leaf weights, specialize 0.5, scarcity 0.5, expansion 0.1528.5%44.9%−0.164
calibrated, specialize 0, scarcity 047.3%37.9%+0.094
calibrated, specialize 1, scarcity 031.6%47.7%−0.161
calibrated, specialize 0, scarcity 146.9%35.2%+0.117
calibrated, specialize 0.5, scarcity 0.542.2%42.6%−0.004
calibrated, coverage 0.4, balance 0.546.1%37.1%+0.090

Two lessons. First, the leaf's preference for ore and grain, which the search had learned to win with against builders, is right in this arena, and a valuation that rewards spreading production across all five resources is wrong by a wide margin. Second, the five discrete strategies (coverage, cities, roads, ports, cards), blended in by specialize, hurt at every strength because taking the best specialized valuation inflates concentrated pairs of the wrong kind. The strategies remain in the code as a research knob with the default at zero.

Registered cohorts

The registered protocol, run 256-game engine cohort, boards 64 to 127, four rotations, all 256 games complete: planner in slot 0, 110 wins (43.0%); previous placement in slot 1, 94 (36.7%); paired contrast +0.062 (−0.036 to +0.161). The registered rule asked for an interval excluding zero and was not met.

The same day the bargaining arms found that two identical searches in slots 0 and 1 differ by seating alone (arms report), so the comparison was registered again as swapped pairs on the same boards, planner in slot 0 and then the previous placement in slot 0, with the effect taken as half the difference of the two contrasts and the seating term as half their sum.

BoardsPlanner in slot 0Previous placement in slot 0Planner effectSeating term
0 to 63+0.051+0.062−0.006 (−0.087 to +0.075)+0.057
64 to 127+0.043+0.098−0.027 (−0.111 to +0.057)+0.070
pooled, 128 boards−0.017 (−0.075 to +0.042)+0.063 (+0.008 to +0.119)

Every single-seating number above, including the calibration sweep, carried that seating term. The calibration ordering still stands (the arms were run in the same seating, and the pip-only draft lost by far more than any seating term), but the claim that the calibrated planner beats the search does not.

What could still be wrong

Every game here is against fixed builders that never block and rarely trade, which is exactly the setting where an ore-and-grain start is safest. Against opponents who contest ground the coverage and expansion terms may deserve more weight than this calibration gives them; the knobs exist for that study. The draft prediction assumes opponents place by the same planner, which is exact for the builders and the search but not for a person.

Methods and reproduction

Screens were run with cargo run --release -p settlers-arena --bin tournament -- --seat 'v2:{"depth":2,"opening_weights":{...}}' --seat 'v2:{"depth":2,"opening":"search"}' --seat eta --seat fast --seeds 0-63 --rotations 4 --deterministic and are not filed as records. The confirmation is filed under 256-game engine cohort; the swapped pairs are runs 256-game engine cohort, 256-game engine cohort, 256-game engine cohort, and 256-game engine cohort under the same study. The planner's terms and settings are documented in the server's docs/expectimax.md under Opening placement.

Detailed result and cohort context

On 64 deterministic boards played once per seat rotation, a planner that valued setup sites in pips with a player's sense of balance lost to the search's own placement by −0.301 wins per game (95% interval −0.414 to −0.188). Calibrated to the leaf's resource values, with its structural terms shrunk to tie-breakers and its discrete strategies switched off, it appeared to win by +0.09 to +0.12 in single-seating screens and by +0.062 on fresh boards. Measured in both seatings on 128 boards, that gain is seating: the planner effect is −0.017 wins per game (−0.075 to +0.042) and the seating term +0.063 (+0.008 to +0.119). The planner is as strong as the search's placement, and unlike it every placement is balanced and explained.