brackt/docs/agents/simulators.md
Chris Parsons e5295812f6
Formalize simulator system with manifest, input-policy, runner, and admin UI (#409)
Introduces three new schema tables (simulator_profiles,
sports_season_simulator_configs, season_participant_simulator_inputs),
a central model layer (app/models/simulator.ts), and a single runner
entry point so every simulator run follows the same prepare → simulate
→ persist → snapshot → recalculate flow.

Key additions:
- manifest.ts: per-simulator display names, default configs, required/
  optional inputs, derivable-input declarations, and setup sections
- input-policy.ts: resolves sourceElo from projectedWins,
  projectedTablePoints, or sourceOdds; resolves ratings from sourceOdds;
  supports block / fallbackElo / averageKnown / worstKnownMinus strategies
- runner.ts: single entry point for admin simulation runs; materialises
  derived inputs, normalises result columns, zeroes omitted participants,
  snapshots EVs, and recalculates linked fantasy standings
- /admin/simulators: inventory page with per-season readiness and bulk run
- /admin/sports-seasons/:id/simulator: per-season setup page with readiness
  summary, input-policy editor, raw JSON config override, and CSV bulk input
- NCAAM/NCAAW simulators now read ratings from season_participant_simulator_inputs,
  falling back to the hardcoded name-keyed maps while DB data is being populated
- Clone flow copies simulator config by default; volatile inputs (odds, Elo)
  only copied when explicitly requested

Code-review fixes included in this commit:
- source field in compatibility bridge checked with !== null instead of !== undefined
- sourceEloRequirementLabel no longer appends "configured fallback" when the
  participant is already excluded from all resolved sources
- Duplicate inline label maps in input-policy.ts replaced with simulatorInputLabel
- save-config preserves existing inputPolicy when the submitted JSON omits it
- Input table truncation label added (Showing 20 of N)
- CSV description notes values must not contain commas
- N+1 comment added to listSportsSeasonSimulatorSummaries
- assertRegistrySchemaDriftFree called in manifest tests
- Runner test suite added covering happy path, already-running guard,
  readiness failure, empty results, and error recovery with status reset

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 21:09:53 -07:00

6.4 KiB

Simulator Guide

Brackt EV simulators project real-world sports outcomes into the standard top-eight probability shape used by fantasy expected values.

Architecture

  • app/services/simulations/registry.ts maps each simulatorType to the algorithm implementation.
  • app/services/simulations/manifest.ts describes each simulator for admin/UI use: display name, default config, required inputs, optional inputs, and setup sections.
  • app/services/simulations/input-policy.ts resolves acceptable alternate inputs, such as projected wins or futures odds, into simulator-ready Elo values.
  • app/models/simulator.ts owns simulator profile/config/input queries. Routes should use this model layer and must not query Drizzle directly.
  • app/services/simulations/runner.ts is the only supported way for admin routes to run a sports-season simulation.
  • season_participant_expected_values is an output table. New mutable simulator inputs belong in season_participant_simulator_inputs.

Data Flow

  1. Admin configures the sport's default simulatorType on the sport.
  2. Each sports_seasons row gets its own simulator config through sports_season_simulator_configs.
  3. Admin/import flows save participant inputs to season_participant_simulator_inputs.
  4. The runner validates readiness, runs the registered simulator, normalizes output columns, persists EVs, snapshots probabilities, zeroes omitted participants, and recalculates linked fantasy standings.
  5. /admin/simulators inventories all sports-season simulators and can run ready/idle seasons.

Before running older Elo-based simulators, the runner materializes resolved Elo values back through season_participant_simulator_inputs and the legacy EV compatibility bridge. This lets older simulator algorithms keep reading sourceElo while admin setup can use direct Elo, projected wins/table points, futures odds, or an explicit fallback policy.

The runner also materializes derived rating values for rating-based simulators that declare that support, such as preseason NCAAM/NCAAW using futures odds before KenPom/Barttorvik-style ratings are available.

Season Scoping

Never store mutable simulator state on sports. A sport can have multiple active seasons at once, such as NHL playoffs for one season and preseason drafting for the next. Each sports season needs independent:

  • Participants
  • Simulator config overrides
  • Ratings, rankings, odds, seeds, and metadata
  • Standings/events/brackets
  • EV outputs and snapshots

When cloning a sports season, copy simulator structure/config by default. Do not copy volatile inputs such as odds, Elo ratings, rankings, or standings unless an admin explicitly opts in.

Adding A Simulator

  1. Add the simulator type to database/schema.ts and generate a Drizzle migration with npm run db:generate.
  2. Implement Simulator in app/services/simulations/.
  3. Register the simulator in app/services/simulations/registry.ts.
  4. Add a manifest profile in app/services/simulations/manifest.ts.
  5. Add readiness and setup expectations through required/optional inputs and setup sections.
  6. If a required input can be derived, add derivableInputs. For example, an Elo simulator with projected wins support should declare derivableInputs: { sourceElo: ["projectedWins"] }.
  7. Add unit tests for algorithm helpers and a readiness/manifest regression test.
  8. Confirm /admin/simulators shows the season and the per-season setup page reports actionable missing inputs.

Hardcoding Rules

Do not hardcode refreshable season data in simulator files:

  • No current-season Elo tables
  • No KenPom/Barttorvik/SRS maps as production inputs
  • No bookmaker odds
  • No team rankings/seeds that change per season
  • No participant-specific metadata that admins may need to change

It is acceptable to keep stable algorithmic rules in code:

  • Bracket advancement mechanics
  • Swiss-stage mechanics
  • Best-of series math
  • Plackett-Luce, Elo, Log5, and Harville formulas
  • Sport format rules that are intentionally not admin-editable yet

If a hardcoded fallback is temporarily retained for backwards compatibility, admin readiness should still require the corresponding DB-managed input before normal admin-triggered runs.

Admin Setup

Use /admin/sports-seasons/:id/simulator for per-season setup. The page should show readiness, missing inputs, config JSON, links to sport-specific setup pages, and a generic CSV importer for common inputs:

name,sourceElo,sourceOdds,worldRanking,rating,projectedWins,projectedTablePoints,seed,region
Boston Celtics,1699,+450,1,0.95,58,,1,East

Keep specialized pages when they provide real workflow value, such as Golf Skills, Surface Elo, CS2 setup, regular standings, and bracket setup. Those pages should still feed readiness through the simulator model layer.

Input Policies

Direct ratings are always preferred. If a simulator declares derived inputs, readiness may also pass with those alternatives:

  • projectedWins can become Elo using seasonGames and parityFactor from season config.
  • projectedTablePoints can become Elo using seasonGames, maxTablePoints, and parityFactor.
  • sourceOdds can become Elo through the shared futures-to-Elo conversion.
  • sourceOdds can become a generic rating when the simulator declares derivableInputs: { rating: ["sourceOdds"] }.

Missing tail participants must remain blocked unless the season config explicitly chooses an inputPolicy.missingEloStrategy:

{
  "inputPolicy": {
    "missingEloStrategy": "worstKnownMinus",
    "fallbackElo": 1400,
    "fallbackEloDelta": 25,
    "eloMin": 1100,
    "eloMax": 1900
  }
}

Supported strategies are block, fallbackElo, averageKnown, and worstKnownMinus. Use fallbacks only when the tail participants have genuinely tiny EV impact; the setup page will warn when derived or fallback Elo is being used.

For rating conversions, configure inputPolicy.ratingMin and inputPolicy.ratingMax to match the target model scale. NCAAM uses a KenPom-like net rating range; NCAAW uses a Barthag-like 0-1 range. Futures-derived ratings are a preseason approximation and should be replaced by real ratings when available.

Required Checks

After simulator changes, run:

npm run typecheck
npm run lint
npm run test:run

At minimum, add or update tests for:

  • Manifest/schema/registry drift
  • Readiness validation
  • Input persistence and legacy compatibility bridge
  • Shared runner success/failure behavior
  • Any simulator whose required inputs changed