brackt/app/services/simulations/__tests__/manifest.test.ts

65 lines
2.9 KiB
TypeScript
Raw Normal View History

Formalize simulator system with manifest, input-policy, runner, and admin UI (#409) Introduces three new schema tables (simulator_profiles, sports_season_simulator_configs, season_participant_simulator_inputs), a central model layer (app/models/simulator.ts), and a single runner entry point so every simulator run follows the same prepare → simulate → persist → snapshot → recalculate flow. Key additions: - manifest.ts: per-simulator display names, default configs, required/ optional inputs, derivable-input declarations, and setup sections - input-policy.ts: resolves sourceElo from projectedWins, projectedTablePoints, or sourceOdds; resolves ratings from sourceOdds; supports block / fallbackElo / averageKnown / worstKnownMinus strategies - runner.ts: single entry point for admin simulation runs; materialises derived inputs, normalises result columns, zeroes omitted participants, snapshots EVs, and recalculates linked fantasy standings - /admin/simulators: inventory page with per-season readiness and bulk run - /admin/sports-seasons/:id/simulator: per-season setup page with readiness summary, input-policy editor, raw JSON config override, and CSV bulk input - NCAAM/NCAAW simulators now read ratings from season_participant_simulator_inputs, falling back to the hardcoded name-keyed maps while DB data is being populated - Clone flow copies simulator config by default; volatile inputs (odds, Elo) only copied when explicitly requested Code-review fixes included in this commit: - source field in compatibility bridge checked with !== null instead of !== undefined - sourceEloRequirementLabel no longer appends "configured fallback" when the participant is already excluded from all resolved sources - Duplicate inline label maps in input-policy.ts replaced with simulatorInputLabel - save-config preserves existing inputPolicy when the submitted JSON omits it - Input table truncation label added (Showing 20 of N) - CSV description notes values must not contain commas - N+1 comment added to listSportsSeasonSimulatorSummaries - assertRegistrySchemaDriftFree called in manifest tests - Runner test suite added covering happy path, already-running guard, readiness failure, empty results, and error recovery with status reset Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 21:09:53 -07:00
import { describe, expect, it } from "vitest";
import * as schema from "~/database/schema";
import { SIMULATOR_MANIFEST } from "../manifest";
import { SIMULATOR_TYPES } from "../registry";
Unify simulator strength on a single blended Elo Replace the two parallel odds/Elo mechanisms with one model: every strength source (raw Elo, projected wins, projected table points, futures odds) converts to an Elo, those blend by weight into a single Elo, and that one Elo feeds every simulator. This supersedes the earlier source-priority + per-match probability blend approach. Resolution (app/services/simulations/input-policy.ts): - SimulatorInputPolicy gains baseEloPriority (order among the substitutable base sources: raw Elo / projected wins / projected table points) and oddsWeight (0–1). resolveSourceElos/resolveRatings now compute baseElo, derive an odds Elo via convertFuturesToElo, and blend: 0 = base only, 1 = futures override, between = weighted blend (method "blend"). - Drops the odds-inclusive sourceEloPriority and the prefersFuturesOdds helper. Simulators consume the single resolved Elo: - UCL, World Cup, NCAA Football, MLB drop their separate normalized-odds signal, convertFuturesToElo calls, and per-match probability blend; they read the resolved sourceElo (preserving each sim's hardcoded fallback Elo table). The optional SimulationContext oddsWeight plumbing (types.ts/runner.ts) is removed. - UCL is routed through the central blend (requiredInputs sourceElo, derivable from sourceOdds) so any entered Elo and futures blend uniformly. - College hockey already blends odds into Elo internally (and uses NPI rank the central resolver can't), so its central oddsWeight is set to 0 to avoid double-counting; the simulator is unchanged. Manifest: per-profile oddsWeight defaults (World Cup/UCL/MLB 0.3, NCAA FB 0.4, college hockey 0; global default 0.3). UI: the Input Policy card exposes one "Futures vs. Elo — Odds Blend Weight" control; the /admin/simulators inventory badge shows the effective blend ("Elo only" / "NN% blend" / "overrides Elo") with the odds participant count. Tests: input-policy blend math (0/0.5/1) for Elo and ratings, baseEloPriority and oddsWeight parsing/clamping, manifest per-profile weights; obsolete source-priority and oddsWeight-context tests removed/replaced. Note: this intentionally shifts the calibrated EV outputs of the four sims that previously blended at the probability level (accepted in design discussion). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QhNeB7gN6VKene7sdbBVQT
2026-06-26 01:39:12 +00:00
import { getSimulatorInputPolicy } from "../input-policy";
Formalize simulator system with manifest, input-policy, runner, and admin UI (#409) Introduces three new schema tables (simulator_profiles, sports_season_simulator_configs, season_participant_simulator_inputs), a central model layer (app/models/simulator.ts), and a single runner entry point so every simulator run follows the same prepare → simulate → persist → snapshot → recalculate flow. Key additions: - manifest.ts: per-simulator display names, default configs, required/ optional inputs, derivable-input declarations, and setup sections - input-policy.ts: resolves sourceElo from projectedWins, projectedTablePoints, or sourceOdds; resolves ratings from sourceOdds; supports block / fallbackElo / averageKnown / worstKnownMinus strategies - runner.ts: single entry point for admin simulation runs; materialises derived inputs, normalises result columns, zeroes omitted participants, snapshots EVs, and recalculates linked fantasy standings - /admin/simulators: inventory page with per-season readiness and bulk run - /admin/sports-seasons/:id/simulator: per-season setup page with readiness summary, input-policy editor, raw JSON config override, and CSV bulk input - NCAAM/NCAAW simulators now read ratings from season_participant_simulator_inputs, falling back to the hardcoded name-keyed maps while DB data is being populated - Clone flow copies simulator config by default; volatile inputs (odds, Elo) only copied when explicitly requested Code-review fixes included in this commit: - source field in compatibility bridge checked with !== null instead of !== undefined - sourceEloRequirementLabel no longer appends "configured fallback" when the participant is already excluded from all resolved sources - Duplicate inline label maps in input-policy.ts replaced with simulatorInputLabel - save-config preserves existing inputPolicy when the submitted JSON omits it - Input table truncation label added (Showing 20 of N) - CSV description notes values must not contain commas - N+1 comment added to listSportsSeasonSimulatorSummaries - assertRegistrySchemaDriftFree called in manifest tests - Runner test suite added covering happy path, already-running guard, readiness failure, empty results, and error recovery with status reset Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 21:09:53 -07:00
import { assertRegistrySchemaDriftFree } from "~/models/simulator";
describe("simulator manifest", () => {
it("has a manifest profile for every registered simulator type", () => {
expect(Object.keys(SIMULATOR_MANIFEST).toSorted()).toEqual([...SIMULATOR_TYPES].toSorted());
});
it("keeps the registry and database enum in sync", () => {
expect([...schema.simulatorTypeEnum.enumValues].toSorted()).toEqual([...SIMULATOR_TYPES].toSorted());
});
it("assertRegistrySchemaDriftFree does not throw when registry and schema are aligned", async () => {
await expect(assertRegistrySchemaDriftFree()).resolves.toBeUndefined();
});
it("documents admin setup requirements for all profiles", () => {
for (const simulatorType of SIMULATOR_TYPES) {
const profile = SIMULATOR_MANIFEST[simulatorType];
expect(profile.displayName).toBeTruthy();
expect(profile.description).toBeTruthy();
expect(Array.isArray(profile.requiredInputs)).toBe(true);
expect(Array.isArray(profile.optionalInputs)).toBe(true);
expect(profile.setupSections.length).toBeGreaterThan(0);
expect(profile.defaultConfig).toBeTypeOf("object");
}
});
it("only derives inputs from declared optional inputs", () => {
for (const simulatorType of SIMULATOR_TYPES) {
const profile = SIMULATOR_MANIFEST[simulatorType];
for (const alternatives of Object.values(profile.derivableInputs ?? {})) {
for (const alternative of alternatives ?? []) {
expect(profile.optionalInputs).toContain(alternative);
}
}
}
});
2026-05-13 15:02:23 -07:00
Unify simulator strength on a single blended Elo Replace the two parallel odds/Elo mechanisms with one model: every strength source (raw Elo, projected wins, projected table points, futures odds) converts to an Elo, those blend by weight into a single Elo, and that one Elo feeds every simulator. This supersedes the earlier source-priority + per-match probability blend approach. Resolution (app/services/simulations/input-policy.ts): - SimulatorInputPolicy gains baseEloPriority (order among the substitutable base sources: raw Elo / projected wins / projected table points) and oddsWeight (0–1). resolveSourceElos/resolveRatings now compute baseElo, derive an odds Elo via convertFuturesToElo, and blend: 0 = base only, 1 = futures override, between = weighted blend (method "blend"). - Drops the odds-inclusive sourceEloPriority and the prefersFuturesOdds helper. Simulators consume the single resolved Elo: - UCL, World Cup, NCAA Football, MLB drop their separate normalized-odds signal, convertFuturesToElo calls, and per-match probability blend; they read the resolved sourceElo (preserving each sim's hardcoded fallback Elo table). The optional SimulationContext oddsWeight plumbing (types.ts/runner.ts) is removed. - UCL is routed through the central blend (requiredInputs sourceElo, derivable from sourceOdds) so any entered Elo and futures blend uniformly. - College hockey already blends odds into Elo internally (and uses NPI rank the central resolver can't), so its central oddsWeight is set to 0 to avoid double-counting; the simulator is unchanged. Manifest: per-profile oddsWeight defaults (World Cup/UCL/MLB 0.3, NCAA FB 0.4, college hockey 0; global default 0.3). UI: the Input Policy card exposes one "Futures vs. Elo — Odds Blend Weight" control; the /admin/simulators inventory badge shows the effective blend ("Elo only" / "NN% blend" / "overrides Elo") with the odds participant count. Tests: input-policy blend math (0/0.5/1) for Elo and ratings, baseEloPriority and oddsWeight parsing/clamping, manifest per-profile weights; obsolete source-priority and oddsWeight-context tests removed/replaced. Note: this intentionally shifts the calibrated EV outputs of the four sims that previously blended at the probability level (accepted in design discussion). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QhNeB7gN6VKene7sdbBVQT
2026-06-26 01:39:12 +00:00
it("exposes the per-profile futures blend weight through the input policy", () => {
Unify futures odds with the simulation system Make the futures-vs-Elo relationship an explicit, configurable rule and fix futures odds failing to override stored Elo. Root causes addressed: - resolveSourceElos/resolveRatings used a hardcoded precedence (direct Elo -> projectedWins -> projectedTablePoints -> odds). The prior fix nulled sourceElo on futures entry but not projections, which still outranked odds. - The override relied on a destructive null-on-save hack that also wiped manually entered Elo. - Blended simulators buried their Elo/odds weight in module constants. - Futures odds were not surfaced on the /admin/simulators inventory. Changes: - Add a configurable source policy: sourceEloPriority (ordered) and oddsWeight, parsed/clamped in getSimulatorInputPolicy. resolveSourceElos/resolveRatings now resolve each participant by the configured priority instead of a fixed order. Futures-centric simulators (ncaam, ncaaw, world_cup, ncaa_football, college_hockey) default to a futures-override priority. - Drop the destructive nulling in batchSaveFuturesOddsForSimulator and batchSaveSourceOdds; override is now governed by policy, preserving stored Elo. - Thread an optional SimulationContext (oddsWeight) through the Simulator interface so blended sims (UCL, World Cup, NCAA FB, MLB) read the blend weight from the season policy; defaults preserve prior calibration when no context is passed. - Add a "Futures vs. Elo" strategy control and Odds Blend Weight input to the Simulator Setup input-policy card, persisted via save-input-policy. - Surface futures on /admin/simulators: a source badge and a Futures Odds quick link; extend listSportsSeasonSimulatorSummaries with odds source info. - Tests: configurable priority override (Elo/projections/rating), oddsWeight parsing/clamping, prefersFuturesOdds, manifest defaults, an NCAA Football context-blend behavioral test, and an updated non-destructive save test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QhNeB7gN6VKene7sdbBVQT
2026-06-25 22:11:32 +00:00
expect(getSimulatorInputPolicy(SIMULATOR_MANIFEST.ncaa_football_bracket.defaultConfig).oddsWeight).toBe(0.4);
expect(getSimulatorInputPolicy(SIMULATOR_MANIFEST.world_cup.defaultConfig).oddsWeight).toBe(0.3);
Unify simulator strength on a single blended Elo Replace the two parallel odds/Elo mechanisms with one model: every strength source (raw Elo, projected wins, projected table points, futures odds) converts to an Elo, those blend by weight into a single Elo, and that one Elo feeds every simulator. This supersedes the earlier source-priority + per-match probability blend approach. Resolution (app/services/simulations/input-policy.ts): - SimulatorInputPolicy gains baseEloPriority (order among the substitutable base sources: raw Elo / projected wins / projected table points) and oddsWeight (0–1). resolveSourceElos/resolveRatings now compute baseElo, derive an odds Elo via convertFuturesToElo, and blend: 0 = base only, 1 = futures override, between = weighted blend (method "blend"). - Drops the odds-inclusive sourceEloPriority and the prefersFuturesOdds helper. Simulators consume the single resolved Elo: - UCL, World Cup, NCAA Football, MLB drop their separate normalized-odds signal, convertFuturesToElo calls, and per-match probability blend; they read the resolved sourceElo (preserving each sim's hardcoded fallback Elo table). The optional SimulationContext oddsWeight plumbing (types.ts/runner.ts) is removed. - UCL is routed through the central blend (requiredInputs sourceElo, derivable from sourceOdds) so any entered Elo and futures blend uniformly. - College hockey already blends odds into Elo internally (and uses NPI rank the central resolver can't), so its central oddsWeight is set to 0 to avoid double-counting; the simulator is unchanged. Manifest: per-profile oddsWeight defaults (World Cup/UCL/MLB 0.3, NCAA FB 0.4, college hockey 0; global default 0.3). UI: the Input Policy card exposes one "Futures vs. Elo — Odds Blend Weight" control; the /admin/simulators inventory badge shows the effective blend ("Elo only" / "NN% blend" / "overrides Elo") with the odds participant count. Tests: input-policy blend math (0/0.5/1) for Elo and ratings, baseEloPriority and oddsWeight parsing/clamping, manifest per-profile weights; obsolete source-priority and oddsWeight-context tests removed/replaced. Note: this intentionally shifts the calibrated EV outputs of the four sims that previously blended at the probability level (accepted in design discussion). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QhNeB7gN6VKene7sdbBVQT
2026-06-26 01:39:12 +00:00
expect(getSimulatorInputPolicy(SIMULATOR_MANIFEST.ucl_bracket.defaultConfig).oddsWeight).toBe(0.3);
// College hockey blends odds internally, so the central blend is disabled.
expect(getSimulatorInputPolicy(SIMULATOR_MANIFEST.college_hockey_bracket.defaultConfig).oddsWeight).toBe(0);
// Every profile resolves to the default base-Elo priority unless overridden.
expect(getSimulatorInputPolicy(SIMULATOR_MANIFEST.nba_bracket.defaultConfig).baseEloPriority)
.toEqual(["sourceElo", "projectedWins", "projectedTablePoints"]);
Unify futures odds with the simulation system Make the futures-vs-Elo relationship an explicit, configurable rule and fix futures odds failing to override stored Elo. Root causes addressed: - resolveSourceElos/resolveRatings used a hardcoded precedence (direct Elo -> projectedWins -> projectedTablePoints -> odds). The prior fix nulled sourceElo on futures entry but not projections, which still outranked odds. - The override relied on a destructive null-on-save hack that also wiped manually entered Elo. - Blended simulators buried their Elo/odds weight in module constants. - Futures odds were not surfaced on the /admin/simulators inventory. Changes: - Add a configurable source policy: sourceEloPriority (ordered) and oddsWeight, parsed/clamped in getSimulatorInputPolicy. resolveSourceElos/resolveRatings now resolve each participant by the configured priority instead of a fixed order. Futures-centric simulators (ncaam, ncaaw, world_cup, ncaa_football, college_hockey) default to a futures-override priority. - Drop the destructive nulling in batchSaveFuturesOddsForSimulator and batchSaveSourceOdds; override is now governed by policy, preserving stored Elo. - Thread an optional SimulationContext (oddsWeight) through the Simulator interface so blended sims (UCL, World Cup, NCAA FB, MLB) read the blend weight from the season policy; defaults preserve prior calibration when no context is passed. - Add a "Futures vs. Elo" strategy control and Odds Blend Weight input to the Simulator Setup input-policy card, persisted via save-input-policy. - Surface futures on /admin/simulators: a source badge and a Futures Odds quick link; extend listSportsSeasonSimulatorSummaries with odds source info. - Tests: configurable priority override (Elo/projections/rating), oddsWeight parsing/clamping, prefersFuturesOdds, manifest defaults, an NCAA Football context-blend behavioral test, and an updated non-destructive save test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QhNeB7gN6VKene7sdbBVQT
2026-06-25 22:11:32 +00:00
});
2026-05-13 15:02:23 -07:00
it("keeps EPL projection and match parity as separate config knobs", () => {
expect(SIMULATOR_MANIFEST.epl_standings.defaultConfig).toMatchObject({
parityFactor: 400,
matchParityFactor: 400,
averageOpponentElo: 1500,
baseDrawRate: 0.26,
drawDecay: 0.002,
});
});
Formalize simulator system with manifest, input-policy, runner, and admin UI (#409) Introduces three new schema tables (simulator_profiles, sports_season_simulator_configs, season_participant_simulator_inputs), a central model layer (app/models/simulator.ts), and a single runner entry point so every simulator run follows the same prepare → simulate → persist → snapshot → recalculate flow. Key additions: - manifest.ts: per-simulator display names, default configs, required/ optional inputs, derivable-input declarations, and setup sections - input-policy.ts: resolves sourceElo from projectedWins, projectedTablePoints, or sourceOdds; resolves ratings from sourceOdds; supports block / fallbackElo / averageKnown / worstKnownMinus strategies - runner.ts: single entry point for admin simulation runs; materialises derived inputs, normalises result columns, zeroes omitted participants, snapshots EVs, and recalculates linked fantasy standings - /admin/simulators: inventory page with per-season readiness and bulk run - /admin/sports-seasons/:id/simulator: per-season setup page with readiness summary, input-policy editor, raw JSON config override, and CSV bulk input - NCAAM/NCAAW simulators now read ratings from season_participant_simulator_inputs, falling back to the hardcoded name-keyed maps while DB data is being populated - Clone flow copies simulator config by default; volatile inputs (odds, Elo) only copied when explicitly requested Code-review fixes included in this commit: - source field in compatibility bridge checked with !== null instead of !== undefined - sourceEloRequirementLabel no longer appends "configured fallback" when the participant is already excluded from all resolved sources - Duplicate inline label maps in input-policy.ts replaced with simulatorInputLabel - save-config preserves existing inputPolicy when the submitted JSON omits it - Input table truncation label added (Showing 20 of N) - CSV description notes values must not contain commas - N+1 comment added to listSportsSeasonSimulatorSummaries - assertRegistrySchemaDriftFree called in manifest tests - Runner test suite added covering happy path, already-running guard, readiness failure, empty results, and error recovery with status reset Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 21:09:53 -07:00
});