brackt/app/services/simulations/ncaa-football-simulator.ts
Chris Parsons 2848231235
Canonical tournament layer: schema + backfill (1/2) (#365)
* refactor(schema): rename per-window tables to season_* prefix

Renames participants, participant_expected_values, participant_qualifying_totals,
participant_results, participant_surface_elos to season_* prefixed names.
Renames event_results.participant_id to season_participant_id.
Phase 1a of canonical tournament layer migration.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor: rename participant.ts model file to season-participant.ts

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(models): update model layer to use renamed schema exports

Updated all model files to use the renamed schema exports from Task 1:
- participants → seasonParticipants
- participantExpectedValues → seasonParticipantExpectedValues
- participantQualifyingTotals → seasonParticipantQualifyingTotals
- participantResults → seasonParticipantResults
- participantSurfaceElos → seasonParticipantSurfaceElos
- eventResults.participantId → eventResults.seasonParticipantId
- db.query relation accessors updated
- Relation field .participant → .seasonParticipant where applicable
- Import paths updated: ./participant → ./season-participant

Files updated (14 model files + 3 test files):
- draft-pick.ts
- draft-utils.ts
- event-result.ts
- group-stage-match.ts
- participant-result.ts
- qualifying-points.ts
- scoring-calculator.ts
- scoring-event.ts
- sports-season.ts
- surface-elo.ts
- team-score-events.ts
- cs2-major-stage.ts
- golf-skills.ts
- participant-expected-value.ts
- __tests__/sports-season.clone.test.ts
- __tests__/auto-pick.test.ts
- __tests__/executeAutoPick.timer.test.ts

Typecheck errors decreased: 779 → 499 (280 fewer)
All model file errors related to renamed schemas resolved.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(routes): update route layer to use renamed schema exports

- Update model import from ~/models/participant to ~/models/season-participant
- Rename schema.participants to schema.seasonParticipants
- Rename schema.participantResults to schema.seasonParticipantResults
- Rename db.query.participants to db.query.seasonParticipants
- Update 9 route files and 1 test file

Affected files:
- admin.sports-seasons.$id.events.$eventId.bracket.server.ts
- admin.sports-seasons.$id.participants.tsx
- api/draft.force-manual-pick.ts
- api/draft.make-pick.ts
- api/draft.replace-pick.ts
- api/seasons.$seasonId.draft.ts
- leagues/$leagueId.draft-board.$seasonId.tsx
- leagues/$leagueId.sports-seasons.$sportsSeasonId.server.ts
- admin/__tests__/sports-seasons-participants.test.ts

Error count reduced from 499 to 453 (46 errors fixed).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(routes): update route files for schema rename

Update route imports from ~/models/participant to ~/models/season-participant
and fix references to .participant/.participantId on event results to use
.seasonParticipant/.seasonParticipantId after schema rename.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(services): update simulators and services for renamed schema

Update all simulators, services, and server files to use renamed schema tables:
- participants → seasonParticipants
- participantExpectedValues → seasonParticipantExpectedValues
- participantResults → seasonParticipantResults
- eventResults.participantId → eventResults.seasonParticipantId

Files updated:
- 20 sport simulators (NBA, NHL, NFL, MLB, etc.)
- probability-updater.ts
- standings-sync/index.ts
- sports-data-sync.server.ts
- server/socket.ts

Typecheck errors reduced from 365 to 0.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* migration: rename per-window tables to season_* prefix

* fix(tests): update mock query keys after participants table rename

Change mock db.query.participants to db.query.seasonParticipants in test
files to match the schema rename from commit 66145a9. This fixes
"Cannot read properties of undefined (reading 'findFirst'/'findMany')"
errors that occurred when production code queries db.query.seasonParticipants
but test mocks only defined the old participants key.

Files updated:
- app/services/simulations/__tests__/world-cup-simulator.test.ts
- app/routes/api/__tests__/draft.force-manual-pick.test.ts
- app/routes/api/__tests__/draft.force-manual-pick.timer-mode.test.ts
- app/routes/api/__tests__/draft.make-pick.timer-mode.test.ts
- server/__tests__/timer-autodraft.test.ts
- app/models/__tests__/team-score-events.test.ts

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(tests): update remaining mock paths and keys after schema rename

* fix(tests): final two mock stragglers after schema rename

- draft-pick.test.ts: assertion on db.query.participantQualifyingTotals
- process-match-result.test.ts: mock key participants → seasonParticipants

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore: add post-phase1a baseline capture (temp, for diff verification)

* chore: capture pre-migration baselines

* chore: remove post-phase1a capture helper after verification

* schema: add canonical tournament & participant tables

Adds tournaments, participants (canonical), tournament_results, and
participant_surface_elos (canonical). Adds nullable tournament_id to
scoring_events and nullable participant_id to season_participants.
Phase 1b of canonical tournament layer migration.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(models): add canonical tournament, participant, result, surface-elo models

Adds CRUD modules for the canonical tables created in commit 775b905.
Each module mirrors existing app/models conventions (database() from
~/database/context, schema from ~/database/schema, mock-based tests).

Key implementation notes:
- participant.ts exports use "Canonical" prefix (CanonicalParticipant,
  createCanonicalParticipant, etc.) to avoid collision with existing
  season-participant.ts exports
- All four models include comprehensive unit tests following the
  audit-log.test.ts pattern
- Tests use mocked db responses (no real database access)
- Upsert functions use onConflictDoUpdate for appropriate unique constraints

Part of Phase 1b of canonical tournament layer migration.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* migration: create canonical tables, add nullable FKs

* scripts: add extractTournamentIdentity helper for backfill

Pure function that derives canonical (name, year) identity from a
scoring_events row, stripping trailing 4-digit years from the name or
falling back to eventDate. Used by the Phase 2 backfill to group
per-window events into canonical tournaments.

* scripts: add backfill orchestrator for canonical layer

Populates canonical tournaments, participants, tournament_results, and
participant_surface_elos from per-window data for qualifying-points
sports. Skips already-linked rows, is idempotent, and supports dry-run
mode.

Critical invariants enforced by the implementation:
- qualifying_points_awarded is never copied to tournament_results
- season_participant_qualifying_totals is never touched
- conflicting surface-Elo values between windows raise a loud error
  (recorded in report.errors) rather than overwriting

* scripts: add backfill CLI with dry-run default

Wires backfill-canonical-layer.ts to a CLI entry point exposed as
`npm run backfill:canonical`. Defaults to --dry-run; requires --apply
to actually write. Supports --sport=<uuid> to limit to a single sport.
Exits 2 if the backfill reports errors (e.g., surface-Elo conflicts).

* fix(backfill-cli): wrap runBackfill in DatabaseContext.run

The orchestrator uses database() from ~/database/context, which requires
AsyncLocalStorage to be populated. Wrap the CLI invocation with
DatabaseContext.run(db, ...) using server/db's cached connection pool.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(backfill-cli): exit 0 on success so pg pool doesn't block

The cached postgres connection pool keeps the Node event loop open after
main() returns. Explicit process.exit(0) on success mirrors the pattern
in scripts/capture-baseline.ts.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Chris Parsons <chrisp@extrahop.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 20:13:18 -07:00

350 lines
15 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

/**
* NCAA Football CFP Simulator
*
* Monte Carlo simulation of the College Football Playoff (12-team format, 2024present).
*
* Algorithm:
* 1. Load all participants for the sports season from DB
* 2. Load Elo/FPI ratings from participantExpectedValues.sourceElo
* (entered via Admin → Elo Ratings page; use FPI, S&P+, or any Elo-scale rating)
* 3. If sourceOdds (American format) are also stored, build a normalized selection
* weight from implied championship probability (used for field selection in step 4)
* and blend into per-game win probability (ELO_WEIGHT=0.6 / ODDS_WEIGHT=0.4).
* 4. Per simulation, select 12 teams for the CFP field:
* - If the pool has exactly 12 teams: use all of them (post-bracket mode).
* - If the pool has >12 teams: weighted sample without replacement using each
* team's selection weight — odds-derived if available, Elo-based otherwise.
* Teams with stronger championship odds are sampled more often, naturally
* encoding both selection probability and bracket strength into one signal.
* 5. Seed the 12 selected teams by Elo (best Elo = seed 1).
* 6. Simulate the CFP bracket:
* First Round (not scoring): 5v12, 6v11, 7v10, 8v9
* Quarterfinals (scoring): 1 vs 8/9w, 4 vs 5/12w, 3 vs 6/11w, 2 vs 7/10w
* Semifinals (scoring): QF1w vs QF2w, QF3w vs QF4w
* National Championship: SF1w vs SF2w
* 7. Track placement counts per scoring tier across all simulations.
* 8. Convert counts to probability distributions.
*
* Pre-bracket vs post-bracket mode:
* Pre-bracket (>12 participants): probabilities reflect both selection uncertainty
* and bracket performance. A bubble team might appear in only 40% of simulated
* fields, so its champion probability accounts for that.
* Post-bracket (exactly 12 participants): deterministic field, bracket-only sim.
*
* Win probability (per game): eloWinProbability() from probability-engine (400-divisor).
* Blended win probability: ELO_WEIGHT * eloProb + ODDS_WEIGHT * oddsProb (when odds present).
*
* Selection weight (pre-bracket mode):
* With sourceOdds: normalized implied championship probability (vig removed, sums to 1).
* Without sourceOdds: softmax on Elo with temperature SELECTION_TEMP (sharply favors
* higher-rated teams — a 200-point Elo gap yields ~7× selection weight difference).
*
* Placement tiers → SimulationProbabilities mapping:
* probFirst = National Champion (1 per sim)
* probSecond = Championship game loser (1 per sim)
* probThird / probFourth = Semifinal losers (2 per sim — split evenly)
* probFifthprobEighth = Quarterfinal losers (4 per sim — split evenly)
* First Round losers → all 0 (score 0 fantasy points)
* Teams not selected → all 0 (not in field for that sim)
*
* Admin setup:
* 1. Create a Sport with simulatorType = "ncaa_football_bracket"
* 2. Create a Sports Season and add all contender participants (12 or more)
* 3. Enter FPI ratings via Admin → Elo Ratings (stored as sourceElo)
* 4. Optionally enter championship futures odds via Admin → Futures Odds (sourceOdds)
* — strongly recommended for pre-bracket mode; drives both selection and bracket strength
* 5. Run simulation via Admin → Simulate
*/
import { database } from "~/database/context";
import { eq } from "drizzle-orm";
import * as schema from "~/database/schema";
import {
convertAmericanOddsToProbability,
convertFuturesToElo,
eloWinProbability,
} from "~/services/probability-engine";
import type { Simulator, SimulationResult } from "./types";
// ─── Simulation parameters ────────────────────────────────────────────────────
const NUM_SIMULATIONS = 50_000;
const BRACKET_SIZE = 12;
/**
* Blend weights for per-game win probability when sourceOdds are present.
* Lower Elo weight than other sports (0.7) gives more influence to Vegas
* championship odds, which are highly informative in college football.
*/
const ELO_WEIGHT = 0.6;
const ODDS_WEIGHT = 1 - ELO_WEIGHT;
/**
* Softmax temperature for Elo-based selection weights (pre-bracket mode, no odds).
* At T=100, a 200-point Elo gap produces ~7× weight difference — enough to strongly
* favour the top teams while still giving bubble teams meaningful selection probability.
*/
const SELECTION_TEMP = 100;
// ─── Types ────────────────────────────────────────────────────────────────────
interface Team {
participantId: string;
elo: number;
/** Normalized futures win probability (01). Used for blending per-game win prob. */
oddsProb: number;
/**
* Weight used for probabilistic CFP field selection (pre-bracket mode only).
* Derived from oddsProb when available; otherwise softmax on Elo.
*/
selectionWeight: number;
}
// ─── Helpers ─────────────────────────────────────────────────────────────────
/**
* Blended win probability for team1 vs team2.
* Falls back to pure Elo when no futures data is present.
*/
function blendedWinProb(team1: Team, team2: Team): number {
const eloProbValue = eloWinProbability(team1.elo, team2.elo);
if (team1.oddsProb === 0 && team2.oddsProb === 0) {
return eloProbValue;
}
const oddsSum = team1.oddsProb + team2.oddsProb;
const oddsProbValue = oddsSum > 0 ? team1.oddsProb / oddsSum : 0.5;
return ELO_WEIGHT * eloProbValue + ODDS_WEIGHT * oddsProbValue;
}
function simGame(team1: Team, team2: Team): { winner: Team; loser: Team } {
const p1Wins = Math.random() < blendedWinProb(team1, team2);
return p1Wins
? { winner: team1, loser: team2 }
: { winner: team2, loser: team1 };
}
/**
* Weighted sample without replacement — selects `n` teams from `pool` where each
* team's probability of being drawn is proportional to its selectionWeight.
* Returns the selected teams sorted by Elo descending (seed 1 = best Elo).
*/
function sampleBracketField(pool: Team[], n: number): Team[] {
const remaining = [...pool];
const selected: Team[] = [];
for (let i = 0; i < n; i++) {
const totalWeight = remaining.reduce((sum, t) => sum + t.selectionWeight, 0);
let r = Math.random() * totalWeight;
let j = 0;
for (; j < remaining.length - 1; j++) {
r -= remaining[j].selectionWeight;
if (r <= 0) break;
}
selected.push(remaining[j]);
remaining.splice(j, 1);
}
// Seed by Elo so that the best team in the sampled field is always seed 1.
return selected.toSorted((a, b) => b.elo - a.elo);
}
// ─── Bracket simulation ───────────────────────────────────────────────────────
interface PlacementCounts {
champion: number;
finalist: number;
sfLoser: number;
qfLoser: number;
}
/**
* Simulate one full 12-team CFP bracket.
*
* Seeding (teams sorted best→worst Elo, index 0 = seed 1):
* First Round: [4]v[11], [5]v[10], [6]v[9], [7]v[8]
* Quarterfinals: [0] vs fr4w, [3] vs fr1w, [2] vs fr2w, [1] vs fr3w
* Semifinals: qf1w vs qf2w, qf3w vs qf4w
* Championship: sf1w vs sf2w
*/
function simulateBracket(teams: Team[], counts: Map<string, PlacementCounts>): void {
// ── First Round (seeds 512) ───────────────────────────────────────────────
const fr1 = simGame(teams[4], teams[11]); // 5 vs 12
const fr2 = simGame(teams[5], teams[10]); // 6 vs 11
const fr3 = simGame(teams[6], teams[9]); // 7 vs 10
const fr4 = simGame(teams[7], teams[8]); // 8 vs 9
// ── Quarterfinals (seeds 14 get byes) ────────────────────────────────────
const qf1 = simGame(teams[0], fr4.winner); // 1 vs 8/9 winner
const qf2 = simGame(teams[3], fr1.winner); // 4 vs 5/12 winner
const qf3 = simGame(teams[2], fr2.winner); // 3 vs 6/11 winner
const qf4 = simGame(teams[1], fr3.winner); // 2 vs 7/10 winner
const bump = (id: string, key: keyof PlacementCounts) => {
const entry = counts.get(id);
if (entry) entry[key]++;
};
bump(qf1.loser.participantId, "qfLoser");
bump(qf2.loser.participantId, "qfLoser");
bump(qf3.loser.participantId, "qfLoser");
bump(qf4.loser.participantId, "qfLoser");
// ── Semifinals ────────────────────────────────────────────────────────────
const sf1 = simGame(qf1.winner, qf2.winner);
const sf2 = simGame(qf3.winner, qf4.winner);
bump(sf1.loser.participantId, "sfLoser");
bump(sf2.loser.participantId, "sfLoser");
// ── National Championship ─────────────────────────────────────────────────
const final = simGame(sf1.winner, sf2.winner);
bump(final.winner.participantId, "champion");
bump(final.loser.participantId, "finalist");
}
// ─── Simulator ────────────────────────────────────────────────────────────────
export class NCAAFootballSimulator implements Simulator {
async simulate(sportsSeasonId: string): Promise<SimulationResult[]> {
const db = database();
// 1. Load all participants for this sports season.
const participants = await db
.select({ id: schema.seasonParticipants.id })
.from(schema.seasonParticipants)
.where(eq(schema.seasonParticipants.sportsSeasonId, sportsSeasonId));
if (participants.length < BRACKET_SIZE) {
throw new Error(
`CFP simulator requires at least ${BRACKET_SIZE} participants, ` +
`found ${participants.length}. Add all contender teams to the sports season.`
);
}
// 2. Load Elo/FPI ratings and optional futures odds in a single query.
const evRows = await db
.select({
participantId: schema.seasonParticipantExpectedValues.participantId,
sourceElo: schema.seasonParticipantExpectedValues.sourceElo,
sourceOdds: schema.seasonParticipantExpectedValues.sourceOdds,
})
.from(schema.seasonParticipantExpectedValues)
.where(eq(schema.seasonParticipantExpectedValues.sportsSeasonId, sportsSeasonId));
// Build Elo and raw odds maps in a single pass.
const eloFromDb = new Map<string, number>();
const rawOddsProbs = new Map<string, number>();
for (const row of evRows) {
if (row.sourceElo !== null) {
eloFromDb.set(row.participantId, row.sourceElo);
}
if (row.sourceOdds !== null) {
rawOddsProbs.set(row.participantId, convertAmericanOddsToProbability(row.sourceOdds));
}
}
// 3. Build normalized odds probability map (vig removed).
const normalizedOddsMap = new Map<string, number>();
if (rawOddsProbs.size > 0) {
const rawSum = [...rawOddsProbs.values()].reduce((a, b) => a + b, 0);
for (const [id, prob] of rawOddsProbs) {
normalizedOddsMap.set(id, rawSum > 0 ? prob / rawSum : 0);
}
// Backfill Elo from futures for any team missing sourceElo.
if (eloFromDb.size < participants.length) {
const oddsInput = evRows
.filter((r) => r.sourceOdds !== null && !eloFromDb.has(r.participantId))
.map((r) => ({ participantId: r.participantId, odds: r.sourceOdds ?? 0 }));
if (oddsInput.length > 0) {
const oddsEloMap = convertFuturesToElo(oddsInput, "american");
for (const [id, elo] of oddsEloMap) {
eloFromDb.set(id, elo);
}
}
}
}
// 4. Build team list with Elo, oddsProb, and selectionWeight.
const hasOdds = normalizedOddsMap.size > 0;
const allTeams: Team[] = participants.map((p) => ({
participantId: p.id,
elo: eloFromDb.get(p.id) ?? 1500,
oddsProb: normalizedOddsMap.get(p.id) ?? 0,
selectionWeight: 0, // computed below
}));
if (hasOdds) {
// Selection weight = normalized championship implied probability.
// This encodes both "probability of making the field" and "strength once there."
for (const team of allTeams) {
team.selectionWeight = normalizedOddsMap.get(team.participantId) ?? 0;
}
} else {
// No odds: softmax on Elo so top-rated teams are strongly favoured.
const eloValues = allTeams.map((t) => t.elo);
const maxElo = Math.max(...eloValues);
// Subtract max for numerical stability before exp().
const expWeights = allTeams.map((t) => Math.exp((t.elo - maxElo) / SELECTION_TEMP));
const expSum = expWeights.reduce((a, b) => a + b, 0);
for (let i = 0; i < allTeams.length; i++) {
allTeams[i].selectionWeight = expWeights[i] / expSum;
}
}
const preBracketMode = participants.length > BRACKET_SIZE;
// In post-bracket mode (exactly 12), sort once and reuse the same field every sim.
const deterministicField = preBracketMode
? null
: [...allTeams].toSorted((a, b) => b.elo - a.elo);
// 5. Initialise placement count accumulators for all participants.
const allParticipantIds = participants.map((p) => p.id);
const counts = new Map<string, PlacementCounts>(
allParticipantIds.map((id) => [id, { champion: 0, finalist: 0, sfLoser: 0, qfLoser: 0 }])
);
// 6. Run Monte Carlo simulations.
for (let s = 0; s < NUM_SIMULATIONS; s++) {
const field = preBracketMode
? sampleBracketField(allTeams, BRACKET_SIZE)
: (deterministicField ?? []);
simulateBracket(field, counts);
}
// 7. Convert counts to probability distributions.
// SF losers: 2 per sim → each team's share = sfLoser / (2 * N).
// QF losers: 4 per sim → each team's share = qfLoser / (4 * N).
const sfDivisor = 2 * NUM_SIMULATIONS;
const qfDivisor = 4 * NUM_SIMULATIONS;
return allParticipantIds.map((id) => {
const c = counts.get(id) ?? { champion: 0, finalist: 0, sfLoser: 0, qfLoser: 0 };
const sfProb = c.sfLoser / sfDivisor;
const qfProb = c.qfLoser / qfDivisor;
return {
participantId: id,
probabilities: {
probFirst: c.champion / NUM_SIMULATIONS,
probSecond: c.finalist / NUM_SIMULATIONS,
probThird: sfProb,
probFourth: sfProb,
probFifth: qfProb,
probSixth: qfProb,
probSeventh: qfProb,
probEighth: qfProb,
},
source: "cfp_monte_carlo",
};
});
}
}