brackt/docs/agents/simulators.md

124 lines
6.4 KiB
Markdown
Raw Normal View History

Formalize simulator system with manifest, input-policy, runner, and admin UI (#409) Introduces three new schema tables (simulator_profiles, sports_season_simulator_configs, season_participant_simulator_inputs), a central model layer (app/models/simulator.ts), and a single runner entry point so every simulator run follows the same prepare → simulate → persist → snapshot → recalculate flow. Key additions: - manifest.ts: per-simulator display names, default configs, required/ optional inputs, derivable-input declarations, and setup sections - input-policy.ts: resolves sourceElo from projectedWins, projectedTablePoints, or sourceOdds; resolves ratings from sourceOdds; supports block / fallbackElo / averageKnown / worstKnownMinus strategies - runner.ts: single entry point for admin simulation runs; materialises derived inputs, normalises result columns, zeroes omitted participants, snapshots EVs, and recalculates linked fantasy standings - /admin/simulators: inventory page with per-season readiness and bulk run - /admin/sports-seasons/:id/simulator: per-season setup page with readiness summary, input-policy editor, raw JSON config override, and CSV bulk input - NCAAM/NCAAW simulators now read ratings from season_participant_simulator_inputs, falling back to the hardcoded name-keyed maps while DB data is being populated - Clone flow copies simulator config by default; volatile inputs (odds, Elo) only copied when explicitly requested Code-review fixes included in this commit: - source field in compatibility bridge checked with !== null instead of !== undefined - sourceEloRequirementLabel no longer appends "configured fallback" when the participant is already excluded from all resolved sources - Duplicate inline label maps in input-policy.ts replaced with simulatorInputLabel - save-config preserves existing inputPolicy when the submitted JSON omits it - Input table truncation label added (Showing 20 of N) - CSV description notes values must not contain commas - N+1 comment added to listSportsSeasonSimulatorSummaries - assertRegistrySchemaDriftFree called in manifest tests - Runner test suite added covering happy path, already-running guard, readiness failure, empty results, and error recovery with status reset Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 21:09:53 -07:00
# Simulator Guide
Brackt EV simulators project real-world sports outcomes into the standard top-eight probability shape used by fantasy expected values.
## Architecture
- `app/services/simulations/registry.ts` maps each `simulatorType` to the algorithm implementation.
- `app/services/simulations/manifest.ts` describes each simulator for admin/UI use: display name, default config, required inputs, optional inputs, and setup sections.
- `app/services/simulations/input-policy.ts` resolves acceptable alternate inputs, such as projected wins or futures odds, into simulator-ready Elo values.
- `app/models/simulator.ts` owns simulator profile/config/input queries. Routes should use this model layer and must not query Drizzle directly.
- `app/services/simulations/runner.ts` is the only supported way for admin routes to run a sports-season simulation.
- `season_participant_expected_values` is an output table. New mutable simulator inputs belong in `season_participant_simulator_inputs`.
## Data Flow
1. Admin configures the sport's default `simulatorType` on the sport.
2. Each `sports_seasons` row gets its own simulator config through `sports_season_simulator_configs`.
3. Admin/import flows save participant inputs to `season_participant_simulator_inputs`.
4. The runner validates readiness, runs the registered simulator, normalizes output columns, persists EVs, snapshots probabilities, zeroes omitted participants, and recalculates linked fantasy standings.
5. `/admin/simulators` inventories all sports-season simulators and can run ready/idle seasons.
Before running older Elo-based simulators, the runner materializes resolved Elo values back through `season_participant_simulator_inputs` and the legacy EV compatibility bridge. This lets older simulator algorithms keep reading `sourceElo` while admin setup can use direct Elo, projected wins/table points, futures odds, or an explicit fallback policy.
The runner also materializes derived `rating` values for rating-based simulators that declare that support, such as preseason NCAAM/NCAAW using futures odds before KenPom/Barttorvik-style ratings are available.
## Season Scoping
Never store mutable simulator state on `sports`. A sport can have multiple active seasons at once, such as NHL playoffs for one season and preseason drafting for the next. Each sports season needs independent:
- Participants
- Simulator config overrides
- Ratings, rankings, odds, seeds, and metadata
- Standings/events/brackets
- EV outputs and snapshots
When cloning a sports season, copy simulator structure/config by default. Do not copy volatile inputs such as odds, Elo ratings, rankings, or standings unless an admin explicitly opts in.
## Adding A Simulator
1. Add the simulator type to `database/schema.ts` and generate a Drizzle migration with `npm run db:generate`.
2. Implement `Simulator` in `app/services/simulations/`.
3. Register the simulator in `app/services/simulations/registry.ts`.
4. Add a manifest profile in `app/services/simulations/manifest.ts`.
5. Add readiness and setup expectations through required/optional inputs and setup sections.
6. If a required input can be derived, add `derivableInputs`. For example, an Elo simulator with projected wins support should declare `derivableInputs: { sourceElo: ["projectedWins"] }`.
7. Add unit tests for algorithm helpers and a readiness/manifest regression test.
8. Confirm `/admin/simulators` shows the season and the per-season setup page reports actionable missing inputs.
## Hardcoding Rules
Do not hardcode refreshable season data in simulator files:
- No current-season Elo tables
- No KenPom/Barttorvik/SRS maps as production inputs
- No bookmaker odds
- No team rankings/seeds that change per season
- No participant-specific metadata that admins may need to change
It is acceptable to keep stable algorithmic rules in code:
- Bracket advancement mechanics
- Swiss-stage mechanics
- Best-of series math
- Plackett-Luce, Elo, Log5, and Harville formulas
- Sport format rules that are intentionally not admin-editable yet
If a hardcoded fallback is temporarily retained for backwards compatibility, admin readiness should still require the corresponding DB-managed input before normal admin-triggered runs.
## Admin Setup
Use `/admin/sports-seasons/:id/simulator` for per-season setup. The page should show readiness, missing inputs, config JSON, links to sport-specific setup pages, and a generic CSV importer for common inputs:
```csv
name,sourceElo,sourceOdds,worldRanking,rating,projectedWins,projectedTablePoints,seed,region
Boston Celtics,1699,+450,1,0.95,58,,1,East
```
Keep specialized pages when they provide real workflow value, such as Golf Skills, Surface Elo, CS2 setup, regular standings, and bracket setup. Those pages should still feed readiness through the simulator model layer.
## Input Policies
Direct ratings are always preferred. If a simulator declares derived inputs, readiness may also pass with those alternatives:
- `projectedWins` can become Elo using `seasonGames` and `parityFactor` from season config.
- `projectedTablePoints` can become Elo using `seasonGames`, `maxTablePoints`, and `parityFactor`.
- `sourceOdds` can become Elo through the shared futures-to-Elo conversion.
- `sourceOdds` can become a generic `rating` when the simulator declares `derivableInputs: { rating: ["sourceOdds"] }`.
Missing tail participants must remain blocked unless the season config explicitly chooses an `inputPolicy.missingEloStrategy`:
```json
{
"inputPolicy": {
"missingEloStrategy": "worstKnownMinus",
"fallbackElo": 1400,
"fallbackEloDelta": 25,
"eloMin": 1100,
"eloMax": 1900
}
}
```
Supported strategies are `block`, `fallbackElo`, `averageKnown`, and `worstKnownMinus`. Use fallbacks only when the tail participants have genuinely tiny EV impact; the setup page will warn when derived or fallback Elo is being used.
For rating conversions, configure `inputPolicy.ratingMin` and `inputPolicy.ratingMax` to match the target model scale. NCAAM uses a KenPom-like net rating range; NCAAW uses a Barthag-like 0-1 range. Futures-derived ratings are a preseason approximation and should be replaced by real ratings when available.
## Required Checks
After simulator changes, run:
```bash
npm run typecheck
npm run lint
npm run test:run
```
At minimum, add or update tests for:
- Manifest/schema/registry drift
- Readiness validation
- Input persistence and legacy compatibility bridge
- Shared runner success/failure behavior
- Any simulator whose required inputs changed