Compare commits

...

3 commits

Author SHA1 Message Date
3c7272392e Merge pull request 'claude/llws-simulator-logic-0aqbjb' (#142) from claude/llws-simulator-logic-0aqbjb into main
All checks were successful
🚀 Deploy / ʦ🔍 Typecheck & Lint (push) Successful in 1m19s
🚀 Deploy / 🧪 Test (push) Successful in 3m11s
🚀 Deploy / 🐳 Build (push) Successful in 1m10s
🚀 Deploy / 🚀 Deploy (push) Successful in 20s
Reviewed-on: #142
2026-08-23 01:57:37 +00:00
Claude
3fae78c521
Address review findings in LLWS simulator
All checks were successful
🚀 Deploy / 🧪 Test (pull_request) Successful in 3m48s
🚀 Deploy / ʦ🔍 Typecheck & Lint (pull_request) Successful in 1m20s
🚀 Deploy / 🐳 Build (pull_request) Has been skipped
🚀 Deploy / 🚀 Deploy (pull_request) Has been skipped
Four fixes from code review of the previous commit.

1. Preserve the futures board's dispersion (llws-simulator.ts).
   convertFuturesToElo finishes by rescaling any field onto a fixed
   1250-1750 Elo span, discarding how spread out the board actually is: a
   board with a 22%-priced favorite and one with a 6%-priced favorite both
   came out 500 Elo wide. On a tight board that inflated the favorite from
   6% to 13% -- worse than the raw-futures model it replaced (RMSE 0.025 vs
   0.005), so the previous commit was a regression in that regime.

   buildLLWSElos now maps decompressed strengths by their log-ratio to the
   field's geometric mean, so Elo span tracks real dispersion. Re-swept the
   parity factor across three board shapes rather than one: 550 minimizes
   total error. Simulated vs priced favorite, with RMSE:
     wide       21.8% -> 20.7%  (0.0055), elo span 1355-1682
     top-heavy  28.2% -> 25.6%  (0.0085), elo span 1354-1733
     tight       6.0% ->  5.8%  (0.0026), elo span 1484-1518

2. Rate an unpriced team at the field's median, not 1500.
   DEFAULT_ELO is the midpoint of the Elo output range, not of the field;
   on a typical board it ranked an unpriced team ~6th of 20, so blanking a
   team's odds promoted it. buildLLWSElos now returns the priced field's
   median alongside the ratings (ranks 11th of 20 on the same board).

3. Pick the bracket event deterministically.
   scoringEvents.findFirst with no ordering returned an arbitrary row when a
   season had more than one llws_20 playoff event; landing on a stale one
   silently reverted to a randomized draw that ignored all recorded results.
   Now takes the most recent, matching world-cup-simulator.ts.

4. Fail on a partially seeded bracket instead of discarding it.
   readBracketSlots returned null on any single missing participant, throwing
   away the draw and every completed result with no warning. Since
   playoff_matches participant columns are ON DELETE SET NULL, removing and
   re-adding one participant mid-tournament was enough to put eliminated
   teams back in contention. It now distinguishes "generated but not seeded"
   (0 slots filled -> randomized draw) from "partially seeded" (throws).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5vQZMPeokzfMqHQjq1RDZ
2026-08-21 22:08:04 +00:00
Claude
d4df0b65fb
Make LLWS simulator bracket-aware and calibrate its futures model
The LLWS simulator was overestimating favorites and ignoring games that had
already been played. Two separate causes:

1. Championship futures were used directly as single-game strength
   (p1 / (p1 + p2)). A future already compounds the ~6 wins needed to take
   the title, so this made every individual game as lopsided as the whole
   tournament and re-compounded that edge round after round. Against a
   representative 20-team board the favorite priced at 21.8% simulated at
   44.9%, and the longest shot fell to ~0%.

   Futures are now decompressed to single-game Elo via convertFuturesToElo,
   the same pipeline the other bracket simulators use, and games are played
   with eloWinProbabilityWithParity. The parity factor was calibrated by
   sweeping it until a randomized-draw simulation reproduces the board it
   was fed: at 1000 the favorite simulates at 21.8% and field-wide RMSE
   drops from 0.062 to 0.003. It is overridable per season via config.

2. The simulator never read playoff_matches, so it re-ran the tournament
   from an empty bracket every time and shuffled the draw at random each
   iteration. A recorded loss changed nothing.

   It now loads the seeded llws_20 bracket, places teams in their real
   slots, and replays completed games from their recorded result instead of
   re-simulating them, so an eliminated team correctly drops to zero. When
   no bracket exists (or it has no participants seeded) it falls back to the
   previous randomized-draw behavior, and a seeded bracket is authoritative
   about which side a team is on, so externalId is only required on the
   pre-bracket path.

Guards: a recorded result is only honored when its two participants are the
ones the simulation routed into that game, so a corrupt or out-of-order row
cannot desynchronize the rest of the bracket; brackets seeding an unknown or
duplicated participant now fail loudly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5vQZMPeokzfMqHQjq1RDZ
2026-08-21 21:09:47 +00:00
4 changed files with 1004 additions and 96 deletions

View file

@ -1,5 +1,12 @@
import { describe, it, expect, vi, beforeEach, type MockInstance } from "vitest";
import { LLWSSimulator } from "../llws-simulator";
import {
LLWSSimulator,
makePlayGame,
playCrossoverGame,
readBracketSlots,
} from "../llws-simulator";
import { convertAmericanOddsToProbability } from "~/services/probability-engine";
import type { SimulationResult } from "../types";
vi.mock("~/database/context", () => ({
database: vi.fn(),
@ -27,28 +34,168 @@ function makeEvRows(ids: string[], opts: { includeOdds?: boolean } = {}) {
}));
}
// ─── Bracket fixtures ─────────────────────────────────────────────────────────
/** The subset of playoff_matches columns the simulator reads. */
type PlayoffMatchRow = {
round: string;
matchNumber: number;
participant1Id: string | null;
participant2Id: string | null;
winnerId: string | null;
loserId: string | null;
isComplete: boolean;
};
const EMPTY_MATCH: PlayoffMatchRow = {
round: "",
matchNumber: 0,
participant1Id: null,
participant2Id: null,
winnerId: null,
loserId: null,
isComplete: false,
};
/**
* A freshly generated, fully seeded llws_20 bracket with no results recorded.
*
* Mirrors generateLLWS20Bracket: U.S. matches take the low match numbers
* (Opening Round 14, Winners Round 2 12), International the high ones
* (Opening Round 58, Winners Round 2 34). Byes sit at participant1 of
* Winners Round 2. Slot order per side is ids[0..7] opening, ids[8..9] byes.
*/
function seededBracket(): PlayoffMatchRow[] {
const matches: PlayoffMatchRow[] = [];
const sides = [
{ ids: US_IDS, openingOffset: 0, wr2Offset: 0 },
{ ids: INTL_IDS, openingOffset: 4, wr2Offset: 2 },
];
for (const { ids, openingOffset, wr2Offset } of sides) {
for (let local = 1; local <= 4; local++) {
matches.push({
...EMPTY_MATCH,
round: "Opening Round",
matchNumber: local + openingOffset,
participant1Id: ids[(local - 1) * 2],
participant2Id: ids[(local - 1) * 2 + 1],
});
}
for (let local = 1; local <= 2; local++) {
matches.push({
...EMPTY_MATCH,
round: "Winners Round 2",
matchNumber: local + wr2Offset,
participant1Id: ids[8 + (local - 1)],
});
}
}
return matches;
}
/**
* Mark a bracket match complete, the way the scoring flow would once the game is
* played. `loserId` is passed explicitly for matches whose second slot is filled by
* advancement rather than by the initial seeding.
*/
function completeMatch(
matches: PlayoffMatchRow[],
round: string,
matchNumber: number,
winnerId: string,
loserId: string
): PlayoffMatchRow[] {
const existing = matches.find((m) => m.round === round && m.matchNumber === matchNumber);
const filled: PlayoffMatchRow = {
...(existing ?? { ...EMPTY_MATCH, round, matchNumber }),
participant1Id: existing?.participant1Id ?? winnerId,
participant2Id: existing?.participant2Id ?? loserId,
winnerId,
loserId,
isComplete: true,
};
return [...matches.filter((m) => m !== existing), filled];
}
/** Normalized (vig-removed) market probability for each team in an odds board. */
function marketProbabilities(odds: number[]): number[] {
const raw = odds.map(convertAmericanOddsToProbability);
const sum = raw.reduce((a, b) => a + b, 0);
return raw.map((p) => p / sum);
}
/** Look up one participant's simulated probabilities, failing loudly if absent. */
function probsFor(results: SimulationResult[], participantId: string) {
const match = results.find((r) => r.participantId === participantId);
if (!match) throw new Error(`No simulation result for ${participantId}`);
return match.probabilities;
}
/** Equal-strength Team records for direct (non-Monte-Carlo) helper tests. */
const TEST_TEAMS = new Map(
ALL_IDS.map((id) => [
id,
{
participantId: id,
side: id.startsWith("us") ? ("US" as const) : ("Intl" as const),
elo: 1500,
},
])
);
function team(participantId: string) {
const found = TEST_TEAMS.get(participantId);
if (!found) throw new Error(`No test team for ${participantId}`);
return found;
}
// ─── Tests ────────────────────────────────────────────────────────────────────
describe("LLWSSimulator", () => {
let mockDb: { select: MockInstance };
let mockDb: {
select: MockInstance;
query: {
scoringEvents: { findMany: MockInstance };
playoffMatches: { findMany: MockInstance };
};
};
let selectCallCount: number;
beforeEach(async () => {
selectCallCount = 0;
const { database } = await import("~/database/context");
mockDb = { select: vi.fn() };
mockDb = {
select: vi.fn(),
query: {
scoringEvents: { findMany: vi.fn().mockResolvedValue([]) },
playoffMatches: { findMany: vi.fn().mockResolvedValue([]) },
},
};
(database as unknown as MockInstance).mockReturnValue(mockDb);
});
function setupMockDb(
participants: { id: string; name?: string; externalId: string | null }[],
evRows: { participantId: string; sourceOdds: number | null }[]
evRows: { participantId: string; sourceOdds: number | null }[],
bracketMatches?: Partial<PlayoffMatchRow>[]
) {
selectCallCount = 0;
mockDb.select.mockImplementation(() => {
const callIndex = selectCallCount++;
const data = callIndex === 0 ? participants : evRows;
return { from: vi.fn().mockReturnValue({ where: vi.fn().mockResolvedValue(data) }) };
});
if (bracketMatches) {
mockDb.query.scoringEvents.findMany.mockResolvedValue([
{ id: "event-1", createdAt: new Date("2026-08-01") },
]);
mockDb.query.playoffMatches.findMany.mockResolvedValue(
bracketMatches.map((m) => ({ ...EMPTY_MATCH, ...m }))
);
}
}
function defaultParticipants(mode: "randomized" | "fixed" = "randomized") {
@ -311,4 +458,424 @@ describe("LLWSSimulator", () => {
await expect(new LLWSSimulator(1_000).simulate("season-1")).rejects.toThrow(/10 US teams/);
});
});
// ── Futures calibration ───────────────────────────────────────────────────
//
// A championship future already contains the ~6 wins needed to lift the trophy.
// Feeding it straight into a single game (p1 / (p1 + p2)) makes every game as
// lopsided as the whole tournament and compounds the favorite's edge round after
// round, which inflated favorites badly. The simulator decompresses futures to Elo
// first, so re-simulating a random draw should hand back roughly the prices it was
// given rather than a much more extreme distribution.
describe("futures calibration", () => {
// A representative LLWS board: a clear favorite, a long tail.
const BOARD = [
200, 750, 900, 1200, 1600, 2000, 2500, 3000, 4000, 6000,
350, 800, 1000, 1400, 1800, 2200, 2800, 3500, 5000, 8000,
];
function boardEvRows() {
return ALL_IDS.map((participantId, i) => ({ participantId, sourceOdds: BOARD[i] }));
}
it("reproduces the favorite's championship price instead of inflating it", async () => {
setupMockDb(defaultParticipants(), boardEvRows());
const results = await new LLWSSimulator(20_000).simulate("season-1");
const market = marketProbabilities(BOARD);
const simulated = probsFor(results, "us-1").probFirst;
// The favorite prices around 22%. The old raw-futures model simulated ~45%.
expect(simulated).toBeCloseTo(market[0], 1);
expect(simulated).toBeLessThan(market[0] + 0.06);
});
it("keeps the whole field close to its priced championship probability", async () => {
setupMockDb(defaultParticipants(), boardEvRows());
const results = await new LLWSSimulator(20_000).simulate("season-1");
const market = marketProbabilities(BOARD);
const errors = ALL_IDS.map((id, i) => probsFor(results, id).probFirst - market[i]);
const rmse = Math.sqrt(errors.reduce((s, e) => s + e * e, 0) / errors.length);
// Calibrated RMSE is ~0.003; the old model sat around 0.06.
expect(rmse).toBeLessThan(0.02);
});
// Regression: the previous mapping rescaled every field onto a fixed 12501750
// Elo span, which discarded how spread out the board actually was and pulled a
// nearly flat field apart into contenders and no-hopers the market never implied.
const TIGHT_BOARD = Array.from({ length: 20 }, (_, i) => 1500 + i * 35);
it("does not inflate the favorite on a tightly priced board", async () => {
setupMockDb(
defaultParticipants(),
ALL_IDS.map((participantId, i) => ({ participantId, sourceOdds: TIGHT_BOARD[i] }))
);
const results = await new LLWSSimulator(20_000).simulate("season-1");
const market = marketProbabilities(TIGHT_BOARD);
const simulated = probsFor(results, "us-1").probFirst;
// The favorite prices near 6%. A fixed-span mapping simulated it around 13%,
// so the band is wide enough for Monte Carlo noise but nowhere near that.
expect(Math.abs(simulated - market[0])).toBeLessThan(0.015);
});
it("keeps a tightly priced field tight", async () => {
setupMockDb(
defaultParticipants(),
ALL_IDS.map((participantId, i) => ({ participantId, sourceOdds: TIGHT_BOARD[i] }))
);
const results = await new LLWSSimulator(20_000).simulate("season-1");
const probs = ALL_IDS.map((id) => probsFor(results, id).probFirst);
// Every team prices between roughly 4% and 6%, so nobody should run away with
// it and nobody should be written off.
expect(Math.max(...probs)).toBeLessThan(0.09);
expect(Math.min(...probs)).toBeGreaterThan(0.02);
});
it("rates a team with no odds entered around the middle of the field", async () => {
// us-5 is priced mid-board; blanking its odds should not move it far. The old
// 1500 fallback was the centre of the Elo scale rather than of the field, which
// promoted an unpriced team to roughly 6th of 20.
const priced = ALL_IDS.map((participantId, i) => ({ participantId, sourceOdds: BOARD[i] }));
setupMockDb(defaultParticipants(), priced);
const withOdds = probsFor(
await new LLWSSimulator(20_000).simulate("season-1"), "us-5"
).probFirst;
const blanked = priced.map((row) =>
row.participantId === "us-5" ? { ...row, sourceOdds: null } : row
);
setupMockDb(defaultParticipants(), blanked);
const withoutOdds = probsFor(
await new LLWSSimulator(20_000).simulate("season-1"), "us-5"
).probFirst;
// Priced 5th of 20, so the median rating should land it in the same territory.
expect(withoutOdds).toBeGreaterThan(withOdds / 2);
expect(withoutOdds).toBeLessThan(withOdds * 2);
});
it("does not starve longshots of championship probability", async () => {
setupMockDb(defaultParticipants(), boardEvRows());
const results = await new LLWSSimulator(20_000).simulate("season-1");
// The longest shot on the board prices near 0.8%. Compounding raw futures drove
// teams like this to essentially zero.
const longshot = probsFor(results, "intl-10").probFirst;
expect(longshot).toBeGreaterThan(0.002);
});
});
// ── Bracket-aware mode ────────────────────────────────────────────────────
describe("bracket-aware mode", () => {
it("uses the real draw rather than shuffling when a bracket is seeded", async () => {
// With no odds every team is equally strong, so the only edge is structural:
// the two bye teams skip the Opening Round. Under a randomized draw every team
// gets a bye equally often and this difference disappears.
setupMockDb(defaultParticipants(), makeEvRows(ALL_IDS), seededBracket());
const results = await new LLWSSimulator(20_000).simulate("season-1");
const byeTeam = probsFor(results, "us-9").probFirst;
const openingTeam = probsFor(results, "us-1").probFirst;
expect(byeTeam).toBeGreaterThan(openingTeam);
});
it("still returns a full, normalized distribution in bracket mode", async () => {
setupMockDb(defaultParticipants(), makeEvRows(ALL_IDS), seededBracket());
const results = await new LLWSSimulator(5_000).simulate("season-1");
expect(results).toHaveLength(20);
expect(results.reduce((s, r) => s + r.probabilities.probFirst, 0)).toBeCloseTo(1.0, 1);
expect(results.reduce((s, r) => s + r.probabilities.probThird, 0)).toBeCloseTo(1.0, 1);
});
it("falls back to a randomized draw when the bracket has no participants seeded", async () => {
const unseeded = seededBracket().map((m) => ({
...m,
participant1Id: null,
participant2Id: null,
}));
setupMockDb(defaultParticipants(), makeEvRows(ALL_IDS), unseeded);
const results = await new LLWSSimulator(5_000).simulate("season-1");
expect(results).toHaveLength(20);
expect(results.reduce((s, r) => s + r.probabilities.probFirst, 0)).toBeCloseTo(1.0, 1);
});
it("uses the most recent bracket event when several exist", async () => {
// A stale event's matches would carry no draw, silently reverting to a
// randomized one and discarding every recorded result.
setupMockDb(defaultParticipants(), makeEvRows(ALL_IDS), seededBracket());
mockDb.query.scoringEvents.findMany.mockResolvedValue([
{ id: "stale-event", createdAt: new Date("2026-07-01") },
{ id: "event-1", createdAt: new Date("2026-08-01") },
]);
const results = await new LLWSSimulator(20_000).simulate("season-1");
// Bracket mode is in force, so the fixed bye slots still show their advantage.
expect(probsFor(results, "us-9").probFirst).toBeGreaterThan(
probsFor(results, "us-1").probFirst
);
});
it("throws when the bracket is only partially seeded", async () => {
// participant1Id/participant2Id are ON DELETE SET NULL, so removing and
// re-adding one participant mid-tournament empties a single slot. Falling back
// to a randomized draw there would put eliminated teams back in contention.
const holed = seededBracket().map((m) =>
m.round === "Opening Round" && m.matchNumber === 3
? { ...m, participant2Id: null }
: m
);
setupMockDb(defaultParticipants(), makeEvRows(ALL_IDS), holed);
await expect(new LLWSSimulator(100).simulate("season-1")).rejects.toThrow(
/partially seeded \(19 of 20/
);
});
it("throws when the bracket seeds the same team into two slots", async () => {
const duplicated = seededBracket().map((m) =>
m.round === "Opening Round" && m.matchNumber === 2
? { ...m, participant1Id: "us-1" } // us-1 already opens match 1
: m
);
setupMockDb(defaultParticipants(), makeEvRows(ALL_IDS), duplicated);
await expect(new LLWSSimulator(100).simulate("season-1")).rejects.toThrow(
/more than one slot/
);
});
it("takes sides from the bracket, not externalId, once a bracket is seeded", async () => {
// The bracket is authoritative about the draw, so an externalId the pre-bracket
// path would reject must not block a season that already has a real bracket.
const participants = [
...US_IDS.map((id) => ({ id, name: `US Team ${id}`, externalId: "US" })),
...INTL_IDS.slice(0, 9).map((id) => ({ id, name: `Team ${id}`, externalId: "Intl" })),
{ id: "intl-10", name: "Team intl-10", externalId: "CANADA" },
];
setupMockDb(participants, makeEvRows(ALL_IDS), seededBracket());
const results = await new LLWSSimulator(5_000).simulate("season-1");
expect(results).toHaveLength(20);
expect(results.reduce((s, r) => s + r.probabilities.probFirst, 0)).toBeCloseTo(1.0, 1);
});
it("throws when the bracket is seeded with a participant outside the season", async () => {
const foreign = seededBracket().map((m) =>
m.round === "Opening Round" && m.matchNumber === 1
? { ...m, participant1Id: "stranger-1" }
: m
);
setupMockDb(defaultParticipants(), makeEvRows(ALL_IDS), foreign);
await expect(new LLWSSimulator(100).simulate("season-1")).rejects.toThrow(
/not in this sports season/
);
});
});
// ── Completed results ─────────────────────────────────────────────────────
//
// The core of the fix: games already played must stick across every iteration
// instead of being re-simulated from scratch.
describe("completed results", () => {
// us-1 is a strong favorite, so a recorded loss should visibly move its number.
const favouredEvRows = ALL_IDS.map((participantId, i) => ({
participantId,
sourceOdds: participantId === "us-1" ? 200 : 1000 + i * 200,
}));
async function probFirstFor(id: string, matches: PlayoffMatchRow[]): Promise<number> {
setupMockDb(defaultParticipants(), favouredEvRows, matches);
const results = await new LLWSSimulator(20_000).simulate("season-1");
return probsFor(results, id).probFirst;
}
it("drops a favorite's championship probability after a recorded loss", async () => {
const before = await probFirstFor("us-1", seededBracket());
// us-1 loses its Opening Round game. In double elimination that is not an
// elimination — it drops to the elimination bracket — but it now needs a much
// longer path, so its title probability must fall.
const afterLoss = completeMatch(seededBracket(), "Opening Round", 1, "us-2", "us-1");
const after = await probFirstFor("us-1", afterLoss);
expect(after).toBeLessThan(before);
// Not merely noise: a first-round loss is a real blow to a favorite.
expect(after).toBeLessThan(before * 0.8);
// But not elimination either — the elimination bracket still reaches the final.
expect(after).toBeGreaterThan(0);
});
it("raises the opponent's championship probability after that same win", async () => {
const before = await probFirstFor("us-2", seededBracket());
const afterWin = completeMatch(seededBracket(), "Opening Round", 1, "us-2", "us-1");
const after = await probFirstFor("us-2", afterWin);
expect(after).toBeGreaterThan(before);
});
it("zeroes out a team that has been eliminated (two recorded losses)", async () => {
// Fill the elimination-bracket game the way advancement would: the Opening
// Round 1 and Opening Round 4 losers meet in Elimination Round 1 match 2.
let matches = seededBracket();
matches = completeMatch(matches, "Opening Round", 1, "us-2", "us-1");
matches = completeMatch(matches, "Opening Round", 4, "us-7", "us-8");
matches = completeMatch(matches, "Elimination Round 1", 2, "us-8", "us-1");
setupMockDb(defaultParticipants(), favouredEvRows, matches);
const results = await new LLWSSimulator(5_000).simulate("season-1");
const eliminated = probsFor(results, "us-1");
// A second loss is final — every placement tier must be exactly zero.
for (const value of Object.values(eliminated)) {
expect(value).toBe(0);
}
});
it("keeps the distribution normalized once results have been recorded", async () => {
let matches = seededBracket();
matches = completeMatch(matches, "Opening Round", 1, "us-2", "us-1");
matches = completeMatch(matches, "Opening Round", 5, "intl-2", "intl-1");
setupMockDb(defaultParticipants(), favouredEvRows, matches);
const results = await new LLWSSimulator(5_000).simulate("season-1");
expect(results.reduce((s, r) => s + r.probabilities.probFirst, 0)).toBeCloseTo(1.0, 1);
expect(results.reduce((s, r) => s + r.probabilities.probSecond, 0)).toBeCloseTo(1.0, 1);
expect(results.reduce((s, r) => s + r.probabilities.probFifth, 0)).toBeCloseTo(1.0, 1);
});
it("ignores a completed result whose participants never reach that game", async () => {
// A corrupt row: Elimination Round 1 match 2 takes the Opening Round 1 and 4
// losers, so a team from Opening Round 3 can never appear there. The game must
// be simulated instead of desynchronising the rest of the bracket.
const matches = completeMatch(
seededBracket(), "Elimination Round 1", 2, "us-5", "us-6"
);
setupMockDb(defaultParticipants(), favouredEvRows, matches);
const results = await new LLWSSimulator(5_000).simulate("season-1");
expect(results).toHaveLength(20);
expect(results.reduce((s, r) => s + r.probabilities.probFirst, 0)).toBeCloseTo(1.0, 1);
});
});
// ── Result-honoring rules ─────────────────────────────────────────────────
//
// Tested directly rather than through the Monte Carlo output: the aggregate only
// shows these effects diluted by how often a given pairing occurs, which is too
// noisy to assert on.
describe("result-honoring rules", () => {
function bracketOf(matches: PlayoffMatchRow[]) {
const bracket = readBracketSlots(matches, TEST_TEAMS);
if (!bracket) throw new Error("Expected the seeded bracket to be readable");
return bracket;
}
const us1 = team("us-1");
const us2 = team("us-2");
const us5 = team("us-5");
it("replays a completed game from its recorded result", () => {
const bracket = bracketOf(
completeMatch(seededBracket(), "Opening Round", 1, "us-2", "us-1")
);
const play = makePlayGame(0, bracket, 1_000);
// Deterministic across repeats — no coin flip is involved any more.
for (let i = 0; i < 25; i++) {
const result = play("Opening Round", 1, us1, us2);
expect(result.winner.participantId).toBe("us-2");
expect(result.loser.participantId).toBe("us-1");
}
});
it("returns the recorded winner regardless of which slot it arrives in", () => {
const bracket = bracketOf(
completeMatch(seededBracket(), "Opening Round", 1, "us-1", "us-2")
);
const play = makePlayGame(0, bracket, 1_000);
// Same game, arguments swapped.
expect(play("Opening Round", 1, us2, us1).winner.participantId).toBe("us-1");
});
it("simulates a game that has not been played yet", () => {
const play = makePlayGame(0, bracketOf(seededBracket()), 1_000);
const winners = new Set(
Array.from({ length: 200 }, () => play("Opening Round", 1, us1, us2).winner.participantId)
);
// Equal Elo, so both outcomes must show up.
expect(winners).toEqual(new Set(["us-1", "us-2"]));
});
it("ignores a recorded result between teams that did not arrive at the game", () => {
const bracket = bracketOf(
completeMatch(seededBracket(), "Opening Round", 1, "us-5", "us-2")
);
const play = makePlayGame(0, bracket, 1_000);
// us-5 belongs to a different Opening Round game, so this row cannot apply to
// the us-1 v us-2 pairing — it must be simulated instead.
const winners = new Set(
Array.from({ length: 200 }, () => play("Opening Round", 1, us1, us2).winner.participantId)
);
expect(winners).toEqual(new Set(["us-1", "us-2"]));
});
it("reads U.S. and International games from their own match numbers", () => {
// The same side-local game number maps to different global matches per side:
// U.S. Opening Round 1 is match 1, International Opening Round 1 is match 5.
const bracket = bracketOf(
completeMatch(seededBracket(), "Opening Round", 5, "intl-2", "intl-1")
);
const intl1 = team("intl-1");
const intl2 = team("intl-2");
expect(makePlayGame(1, bracket, 1_000)("Opening Round", 1, intl1, intl2).winner.participantId)
.toBe("intl-2");
// The U.S. side's Opening Round 1 is untouched by that result.
const usWinners = new Set(
Array.from({ length: 200 }, () =>
makePlayGame(0, bracket, 1_000)("Opening Round", 1, us1, us2).winner.participantId
)
);
expect(usWinners).toEqual(new Set(["us-1", "us-2"]));
});
it("honors a completed World Championship", () => {
// The two crossover games are single shared matches, numbered 1.
const bracket = bracketOf(
completeMatch(seededBracket(), "World Championship", 1, "us-3", "intl-4")
);
const us3 = team("us-3");
const intl4 = team("intl-4");
const result = playCrossoverGame("World Championship", bracket, 1_000, us3, intl4);
expect(result.winner.participantId).toBe("us-3");
expect(result.loser.participantId).toBe("intl-4");
});
it("simulates the crossover game when different finalists arrive", () => {
const bracket = bracketOf(
completeMatch(seededBracket(), "World Championship", 1, "us-3", "intl-4")
);
const intl5 = team("intl-5");
const winners = new Set(
Array.from({ length: 200 }, () =>
playCrossoverGame("World Championship", bracket, 1_000, us5, intl5).winner.participantId
)
);
expect(winners).toEqual(new Set(["us-5", "intl-5"]));
});
});
});

View file

@ -9,28 +9,48 @@
* pool play. This mirrors the llws_20 bracket template so simulated placements line
* up with the bracket admins actually score.
*
* Two modes:
* 1. Pre-bracket mode: no llws_20 bracket exists yet (or it has no participants
* seeded). Each side is shuffled into the 10 bracket slots every iteration, so
* the draw is modelled as random.
* 2. Bracket-aware mode: a seeded llws_20 bracket exists. Teams sit in their real
* slots and completed match results are honored rather than re-simulated, so a
* team that has already lost carries that loss into every iteration.
*
* Algorithm:
* 1. Load all 20 participants for the sports season from DB
* (must be exactly 10 US + 10 International, identified by externalId)
* 2. Load championship futures odds from participantExpectedValues.sourceOdds
* 2. Load the llws_20 playoff bracket, if one exists, to get the real draw and
* whatever results have been recorded so far
* 3. Load championship futures odds from participantExpectedValues.sourceOdds
* (entered via Admin Futures Odds; American format)
* 3. Convert odds to normalized championship probabilities (vig removed).
* These drive per-game win probability: p1 / (p1 + p2). Falls back to 50/50.
* 4. Per simulation:
* a. Shuffle each side's 10 teams into the 10 bracket slots (8 opening-round
* teams + 2 byes). The draw is modelled as random a specific known draw
* is not yet expressible in participant config.
* b. Simulate the 10-team double-elimination bracket for each side
* (see simulateSideBracket for the exact game-by-game structure)
* 4. Convert those futures to Elo via the shared probability engine, then drive
* each game with the Elo win probability (see "Why Elo" below)
* 5. Per simulation:
* a. Place each side's 10 teams into the bracket slots (real draw when known,
* otherwise shuffled)
* b. Simulate the 10-team double-elimination bracket for each side, replaying
* completed games from their recorded result (see simulateSideBracket)
* c. Consolation game: US side loser vs Intl side loser 3rd / 4th
* d. World Championship: US champion vs Intl champion 1st / 2nd
* 5. Track placement counts across all simulations.
* 6. Convert counts to probability distributions.
* 6. Track placement counts across all simulations
* 7. Convert counts to probability distributions
*
* Why Elo rather than raw futures:
* A championship future already bakes in the ~6 wins needed to lift the trophy, so
* using it directly as a single-game strength (p1 / (p1 + p2)) makes every
* individual game as lopsided as the whole tournament and compounds the favorite's
* edge over and over. buildLLWSElos undoes that compression first (the empirically
* calibrated cube-root step in decompressProbability) before mapping to an Elo
* scale, and LLWS_PARITY_FACTOR then widens the Elo curve to reflect how much
* single-game variance there is in six-inning Little League baseball. Unlike the
* shared convertFuturesToElo helper, the mapping preserves how spread out the board
* actually is see buildLLWSElos for why that matters.
*
* Side assignment (externalId): "US" or "Intl". The legacy pool suffixes
* ("US:A", "US:B", "Intl:A", "Intl:B") are still accepted and read as the side
* alone, so seasons configured for the old pool-play format keep working pools
* no longer exist, so the suffix has no effect.
* no longer exist, so the suffix has no effect. When a seeded bracket exists the
* bracket's own slots decide the sides and externalId is not consulted.
*
* Placement tiers SimulationProbabilities mapping (matches llws_20's scoring):
* probFirst = World Championship winner (1 per sim)
@ -45,15 +65,21 @@
* 1. Create a Sport with simulatorType = "llws_bracket"
* 2. Create a Sports Season and add exactly 20 participants (10 US, 10 International)
* 3. Set externalId on each participant via Admin Manage Participants to "US" or
* "Intl" (optional names starting with "US " infer US, all others infer Intl)
* "Intl" (optional names starting with "US " infer US, all others infer Intl).
* Once the bracket is generated and seeded this is no longer used.
* 4. Enter championship futures odds via Admin Futures Odds (sourceOdds)
* 5. Run simulation via Admin Simulate
*/
import { database } from "~/database/context";
import { eq } from "drizzle-orm";
import { and, eq } from "drizzle-orm";
import * as schema from "~/database/schema";
import { convertAmericanOddsToProbability } from "~/services/probability-engine";
import {
convertAmericanOddsToProbability,
decompressProbability,
eloWinProbabilityWithParity,
} from "~/services/probability-engine";
import { llwsMatchNumber } from "~/lib/bracket-templates";
import type { Simulator, SimulationResult } from "./types";
import { positiveConfigNumber } from "./config-access";
@ -62,16 +88,61 @@ import { positiveConfigNumber } from "./config-access";
const NUM_SIMULATIONS = 50_000;
const US_TEAM_COUNT = 10;
const INTL_TEAM_COUNT = 10;
const DEFAULT_ELO = 1500;
const LLWS_TEMPLATE_ID = "llws_20";
/**
* Elo scaling for a single LLWS game.
*
* Higher than the 400-point standard because a six-inning Little League game between
* 12-year-olds is far closer to a coin flip than a pro game: one pitcher, one big
* inning, and the mercy rule all compress the gap.
*
* Calibrated by sweeping this value until a randomized-draw simulation reproduces the
* championship futures it was fed, across boards of different shape (see
* LLWS_ELO_SPREAD for why the shape matters). Total RMSE over a wide board, a
* top-heavy board, and a nearly flat one:
* parity 450 0.028
* parity 550 0.016 chosen
* parity 750 0.040
* parity 1000 0.061
* Overridable per season via the `parityFactor` simulator config.
*/
const LLWS_PARITY_FACTOR = 550;
/**
* Elo points per natural-log unit of relative team strength.
*
* Only the ratio LLWS_ELO_SPREAD / parityFactor affects the simulation, so this fixes
* the readable scale of the ratings and LLWS_PARITY_FACTOR does the calibrating. 300
* puts a typical 20-team board in the familiar ~13501700 range.
*/
const LLWS_ELO_SPREAD = 300;
/**
* Power transform undoing the compounding baked into a championship future.
* Matches DEFAULT_CALIBRATION.exponent in the probability engine.
*/
const LLWS_DECOMPRESSION_EXPONENT = 0.33;
// ─── Types ────────────────────────────────────────────────────────────────────
type Side = "US" | "Intl";
/** Bracket-template side index: U.S. matches take the low match numbers. */
const SIDE_INDEX: Record<Side, 0 | 1> = { US: 0, Intl: 1 };
/** The playoff_matches columns the simulator actually reads. */
export type BracketMatch = Pick<
typeof schema.playoffMatches.$inferSelect,
"round" | "matchNumber" | "participant1Id" | "participant2Id" | "winnerId" | "loserId" | "isComplete"
>;
interface Team {
participantId: string;
side: Side;
/** Normalized championship win probability (01, vig removed). */
oddsProb: number;
/** Single-game strength on an Elo scale, decompressed from championship futures. */
elo: number;
}
interface PlacementCounts {
@ -85,6 +156,25 @@ interface PlacementCounts {
elimRound4Loser: number;
}
/**
* Plays one bracket game. `round`/`localMatch` identify the game within its side so a
* completed result can be looked up; `t1`/`t2` are the teams the simulation has
* routed into it.
*/
type PlayGame = (
round: string,
localMatch: number,
t1: Team,
t2: Team
) => { winner: Team; loser: Team };
interface LoadedBracket {
/** Each side's 10 teams in bracket slot order (8 opening-round, then 2 byes). */
slots: Record<Side, Team[]>;
/** All bracket matches, keyed by `${round}#${globalMatchNumber}`. */
matches: Map<string, BracketMatch>;
}
// ─── Helpers ─────────────────────────────────────────────────────────────────
function zeroCounts(): PlacementCounts {
@ -94,16 +184,12 @@ function zeroCounts(): PlacementCounts {
};
}
function simGame(t1: Team, t2: Team): { winner: Team; loser: Team } {
// If either team has no odds entered, treat the game as a coin flip.
// The 50/50 fallback must cover the one-sided case (one team known, one not)
// because oddsProb=0 would otherwise give the unknown team a 0% win rate.
let p1Win: number;
if (t1.oddsProb === 0 || t2.oddsProb === 0) {
p1Win = 0.5;
} else {
p1Win = t1.oddsProb / (t1.oddsProb + t2.oddsProb);
}
function matchKey(round: string, matchNumber: number): string {
return `${round}#${matchNumber}`;
}
function simGame(t1: Team, t2: Team, parityFactor: number): { winner: Team; loser: Team } {
const p1Win = eloWinProbabilityWithParity(t1.elo, t2.elo, parityFactor);
return Math.random() < p1Win ? { winner: t1, loser: t2 } : { winner: t2, loser: t1 };
}
@ -118,6 +204,76 @@ function shuffle<T>(arr: T[]): T[] {
return arr;
}
/**
* The recorded loser of a completed match. loserId is written by the scoring flow,
* but fall back to "whichever slot isn't the winner" for older rows.
*/
function completedLoser(match: BracketMatch): string | null {
if (match.loserId) return match.loserId;
if (match.participant1Id === match.winnerId && match.participant2Id) return match.participant2Id;
if (match.participant2Id === match.winnerId && match.participant1Id) return match.participant1Id;
return null;
}
/**
* Build the game-playing function for one side.
*
* When the bracket has a completed result for a game AND that result is between the
* two teams the simulation routed into it, the recorded winner is used verbatim
* that is what makes an already-played loss stick across all iterations. Anything
* else is simulated. The pair check keeps a corrupt or out-of-order row from
* desynchronising the rest of the bracket.
*/
export function makePlayGame(
sideIndex: 0 | 1,
bracket: LoadedBracket | null,
parityFactor: number
): PlayGame {
if (!bracket) {
return (_round, _localMatch, t1, t2) => simGame(t1, t2, parityFactor);
}
return (round, localMatch, t1, t2) => {
const match = bracket.matches.get(
matchKey(round, llwsMatchNumber(round, sideIndex, localMatch))
);
if (match?.isComplete && match.winnerId) {
const loserId = completedLoser(match);
const arrived = [t1.participantId, t2.participantId];
if (loserId && arrived.includes(match.winnerId) && arrived.includes(loserId)) {
return match.winnerId === t1.participantId
? { winner: t1, loser: t2 }
: { winner: t2, loser: t1 };
}
}
return simGame(t1, t2, parityFactor);
};
}
/**
* Play one of the two cross-side games (Consolation, World Championship). Both are a
* single shared match numbered 1, so they don't go through the side-local mapping.
*/
export function playCrossoverGame(
round: string,
bracket: LoadedBracket | null,
parityFactor: number,
t1: Team,
t2: Team
): { winner: Team; loser: Team } {
const match = bracket?.matches.get(matchKey(round, 1));
if (match?.isComplete && match.winnerId) {
const loserId = completedLoser(match);
const arrived = [t1.participantId, t2.participantId];
if (loserId && arrived.includes(match.winnerId) && arrived.includes(loserId)) {
return match.winnerId === t1.participantId
? { winner: t1, loser: t2 }
: { winner: t2, loser: t1 };
}
}
return simGame(t1, t2, parityFactor);
}
/**
* Simulate one side's 10-team double-elimination bracket.
*
@ -125,7 +281,7 @@ function shuffle<T>(arr: T[]): T[] {
* layout: slots[0..7] are the four opening-round games (two teams each) and
* slots[8], slots[9] are the two bye teams entering Winners Round 2.
*
* Structure (side-local, mirroring LLWS_ADVANCEMENT in models/playoff-match):
* Structure (side-local, mirroring LLWS_ADVANCEMENT in lib/llws-bracket):
* Winners bracket
* OP1 s0 v s1 OP2 s2 v s3 OP3 s4 v s5 OP4 s6 v s7
* WR2-1 s8 v OP1w WR2-2 s9 v OP2w
@ -143,47 +299,51 @@ function shuffle<T>(arr: T[]): T[] {
* Elimination Final. There is no "if necessary" game, so the side championship is
* decided in one game.
*
* The team order passed to `play` matches each match's participant1 / participant2
* slots in the generated bracket, so recorded results line up game for game.
*
* Returns { sideChampion, sideLoser }; the two scoring elimination losers are
* bumped into the counts directly.
*/
function simulateSideBracket(
slots: Team[],
bump: (id: string, key: keyof PlacementCounts) => void
bump: (id: string, key: keyof PlacementCounts) => void,
play: PlayGame
): { sideChampion: Team; sideLoser: Team } {
// ── Winners bracket ────────────────────────────────────────────────────────
const op1 = simGame(slots[0], slots[1]);
const op2 = simGame(slots[2], slots[3]);
const op3 = simGame(slots[4], slots[5]);
const op4 = simGame(slots[6], slots[7]);
const op1 = play("Opening Round", 1, slots[0], slots[1]);
const op2 = play("Opening Round", 2, slots[2], slots[3]);
const op3 = play("Opening Round", 3, slots[4], slots[5]);
const op4 = play("Opening Round", 4, slots[6], slots[7]);
const wr21 = simGame(slots[8], op1.winner);
const wr22 = simGame(slots[9], op2.winner);
const wr21 = play("Winners Round 2", 1, slots[8], op1.winner);
const wr22 = play("Winners Round 2", 2, slots[9], op2.winner);
const wsf1 = simGame(op3.winner, wr21.winner);
const wsf2 = simGame(wr22.winner, op4.winner);
const wsf1 = play("Winners Semifinals", 1, op3.winner, wr21.winner);
const wsf2 = play("Winners Semifinals", 2, wr22.winner, op4.winner);
const wf = simGame(wsf1.winner, wsf2.winner);
const wf = play("Winners Final", 1, wsf1.winner, wsf2.winner);
// ── Elimination bracket ────────────────────────────────────────────────────
const er11 = simGame(op2.loser, op3.loser);
const er12 = simGame(op1.loser, op4.loser);
const er11 = play("Elimination Round 1", 1, op2.loser, op3.loser);
const er12 = play("Elimination Round 1", 2, op1.loser, op4.loser);
const er21 = simGame(wr21.loser, er11.winner);
const er22 = simGame(wr22.loser, er12.winner);
const er21 = play("Elimination Round 2", 1, wr21.loser, er11.winner);
const er22 = play("Elimination Round 2", 2, wr22.loser, er12.winner);
// Cross-over: each semifinal loser meets the winner from the opposite half.
const er31 = simGame(wsf1.loser, er22.winner);
const er32 = simGame(wsf2.loser, er21.winner);
const er31 = play("Elimination Round 3", 1, wsf1.loser, er22.winner);
const er32 = play("Elimination Round 3", 2, wsf2.loser, er21.winner);
const er4 = simGame(er32.winner, er31.winner);
const er4 = play("Elimination Round 4", 1, er32.winner, er31.winner);
bump(er4.loser.participantId, "elimRound4Loser"); // 7th8th tier
// The Winners Final loser gets its second chance here.
const ef = simGame(wf.loser, er4.winner);
const ef = play("Elimination Final", 1, wf.loser, er4.winner);
bump(ef.loser.participantId, "elimFinalLoser"); // 5th6th tier
// ── Side championship ──────────────────────────────────────────────────────
const sideChampionship = simGame(wf.winner, ef.winner);
const sideChampionship = play("Bracket Championship", 1, wf.winner, ef.winner);
return { sideChampion: sideChampionship.winner, sideLoser: sideChampionship.loser };
}
@ -209,13 +369,157 @@ function parseExternalId(raw: string | null): { side: Side } | null {
* Infer an externalId from a participant name when none is stored.
* Teams whose name is exactly "US" or starts with "US " (case-insensitive)
* are assigned to the US side; all others are assigned to Intl.
* The inferred value never has a pool suffix, so pools will be randomized.
*/
function inferExternalIdFromName(name: string): string {
const upper = name.trim().toUpperCase();
return upper === "US" || upper.startsWith("US ") ? "US" : "Intl";
}
// ─── Elo construction ─────────────────────────────────────────────────────────
/**
* Map participants to single-game Elo ratings from their championship futures.
*
* Deliberately NOT convertFuturesToElo. That helper finishes by rescaling the field
* onto a fixed 12501750 span (mapToElo), which throws away how spread out the board
* actually is: a board whose favorite is priced at 22% and one whose favorite is
* priced at 6% both come out 500 Elo wide, so the tight board's field gets pulled
* apart into contenders and no-hopers that the market never implied. On such a board
* that inflated the favorite from 6% to 13%.
*
* Instead the decompressed strengths are mapped by their log-ratio to the field's
* geometric mean, which preserves dispersion: a tight board yields a narrow Elo span
* and a top-heavy one a wide span, both centred on DEFAULT_ELO.
*
* Returns the ratings alongside the rating to use for a team with no odds entered
* the median of the priced field, so leaving odds blank neither promotes nor buries a
* team. (DEFAULT_ELO is the centre of the scale, but futures fields are skewed, so on
* a typical board it would rank a team around 6th of 20.)
*/
export function buildLLWSElos(
evRows: Array<{ participantId: string; sourceOdds: number | null }>
): { elos: Map<string, number>; unpricedElo: number } {
const priced = evRows.filter((row) => row.sourceOdds !== null);
// A single priced team carries no information about the rest of the field, so
// there is nothing to normalise against — treat the season as unpriced.
if (priced.length < 2) return { elos: new Map(), unpricedElo: DEFAULT_ELO };
const rawProbs = priced.map((row) => convertAmericanOddsToProbability(row.sourceOdds ?? 0));
const rawSum = rawProbs.reduce((a, b) => a + b, 0);
if (rawSum <= 0) return { elos: new Map(), unpricedElo: DEFAULT_ELO };
// Vig-removed championship probability → single-game strength.
const logStrengths = rawProbs.map((prob) =>
Math.log(
Math.max(decompressProbability(prob / rawSum, LLWS_DECOMPRESSION_EXPONENT), Number.MIN_VALUE)
)
);
const meanLog = logStrengths.reduce((a, b) => a + b, 0) / logStrengths.length;
const elos = new Map<string, number>(
priced.map((row, i) => [
row.participantId,
DEFAULT_ELO + LLWS_ELO_SPREAD * (logStrengths[i] - meanLog),
])
);
return { elos, unpricedElo: median([...elos.values()]) };
}
function median(values: number[]): number {
if (values.length === 0) return DEFAULT_ELO;
const sorted = values.toSorted((a, b) => a - b);
const mid = Math.floor(sorted.length / 2);
return sorted.length % 2 === 0 ? (sorted[mid - 1] + sorted[mid]) / 2 : sorted[mid];
}
// ─── Bracket loading ──────────────────────────────────────────────────────────
/**
* Read the seeded llws_20 bracket for this season, if there is one.
*
* Returns null only when the bracket carries no draw at all no matches, or a
* freshly generated bracket with every slot still empty in which case the caller
* falls back to a randomized draw.
*
* A *partially* seeded bracket is an error rather than a fallback. Silently falling
* back there would throw away the real draw and every recorded result along with it,
* putting eliminated teams back in contention; and it is reachable in practice,
* because playoff_matches.participant1Id/participant2Id are ON DELETE SET NULL, so
* removing and re-adding a single participant mid-tournament empties a slot.
* Likewise, a bracket seeded with unknown or duplicated participants fails loudly.
*/
export function readBracketSlots(
matches: BracketMatch[],
teamsById: Map<string, Team>
): LoadedBracket | null {
if (matches.length === 0) return null;
const byKey = new Map(matches.map((m) => [matchKey(m.round, m.matchNumber), m]));
// Collect both sides' draws before deciding, so "nothing seeded" is judged over the
// whole bracket rather than one side at a time.
const draw: Record<Side, (string | null)[]> = { US: [], Intl: [] };
for (const side of ["US", "Intl"] as const) {
const sideIndex = SIDE_INDEX[side];
for (let local = 1; local <= 4; local++) {
const match = byKey.get(
matchKey("Opening Round", llwsMatchNumber("Opening Round", sideIndex, local))
);
draw[side].push(match?.participant1Id ?? null, match?.participant2Id ?? null);
}
for (let local = 1; local <= 2; local++) {
const match = byKey.get(
matchKey("Winners Round 2", llwsMatchNumber("Winners Round 2", sideIndex, local))
);
draw[side].push(match?.participant1Id ?? null);
}
}
const allSlots = [...draw.US, ...draw.Intl];
const seededCount = allSlots.filter((id) => id !== null).length;
// Generated but not yet filled in — no draw to honor.
if (seededCount === 0) return null;
if (seededCount < allSlots.length) {
throw new Error(
`LLWS bracket is only partially seeded (${seededCount} of ${allSlots.length} slots ` +
`filled). Re-seed the bracket in Admin → Bracket before simulating; simulating ` +
`around the gap would discard the draw and every recorded result.`
);
}
const slots: Record<Side, Team[]> = { US: [], Intl: [] };
const seen = new Set<string>();
for (const side of ["US", "Intl"] as const) {
for (const id of draw[side]) {
const participantId = id as string;
if (seen.has(participantId)) {
throw new Error(
`LLWS bracket seeds participant ${participantId} into more than one slot.`
);
}
seen.add(participantId);
const team = teamsById.get(participantId);
if (!team) {
throw new Error(
`LLWS bracket references participant ${participantId}, which is not in this sports season.`
);
}
// The bracket is authoritative about which side a team is on.
slots[side].push({ ...team, side });
}
}
return { slots, matches: byKey };
}
// ─── Simulator ────────────────────────────────────────────────────────────────
export class LLWSSimulator implements Simulator {
@ -223,6 +527,7 @@ export class LLWSSimulator implements Simulator {
async simulate(sportsSeasonId: string, config: Record<string, unknown> = {}): Promise<SimulationResult[]> {
const numSimulations = Math.round(positiveConfigNumber(config, "iterations", this.numSimulations));
const parityFactor = positiveConfigNumber(config, "parityFactor", LLWS_PARITY_FACTOR);
const db = database();
// 1. Load all participants.
@ -247,41 +552,62 @@ export class LLWSSimulator implements Simulator {
.from(schema.seasonParticipantExpectedValues)
.where(eq(schema.seasonParticipantExpectedValues.sportsSeasonId, sportsSeasonId));
const rawOddsMap = new Map<string, number>();
for (const row of evRows) {
if (row.sourceOdds !== null) {
rawOddsMap.set(row.participantId, convertAmericanOddsToProbability(row.sourceOdds));
}
}
// 3. Normalize odds (remove vig) to get championship probability per team.
const normalizedOddsMap = new Map<string, number>();
if (rawOddsMap.size > 0) {
const rawSum = [...rawOddsMap.values()].reduce((a, b) => a + b, 0);
for (const [id, prob] of rawOddsMap) {
normalizedOddsMap.set(id, rawSum > 0 ? prob / rawSum : 0);
}
}
// 3. Decompress the futures into single-game Elo ratings.
const { elos, unpricedElo } = buildLLWSElos(evRows);
// 4. Parse externalId for each participant to determine which side they're on.
// A seeded bracket overrides this below, but the field still has to be a legal
// 10/10 split before we know whether a bracket exists.
const teams: Team[] = [];
const unparseableSides: Array<{ id: string; externalId: string | null }> = [];
for (const p of participants) {
const raw = p.externalId ?? inferExternalIdFromName(p.name);
const parsed = parseExternalId(raw);
if (!parsed) {
throw new Error(
`Participant ${p.id} has invalid externalId "${p.externalId}". ` +
`Expected: "US" or "Intl".`
);
}
if (!parsed) unparseableSides.push({ id: p.id, externalId: p.externalId });
teams.push({
// Provisional: a seeded bracket overwrites this below.
participantId: p.id,
side: parsed.side,
oddsProb: normalizedOddsMap.get(p.id) ?? 0,
side: parsed?.side ?? "Intl",
elo: elos.get(p.id) ?? unpricedElo,
});
}
// Validate team counts per side.
const teamsById = new Map(teams.map((t) => [t.participantId, t]));
// 5. Load the real bracket (draw + results so far), if one has been generated.
// If several llws_20 playoff events exist, take the most recent so a re-created
// event wins over a stale one — landing on the stale row would silently discard
// the real draw and every recorded result.
const playoffEvents = await db.query.scoringEvents.findMany({
where: and(
eq(schema.scoringEvents.sportsSeasonId, sportsSeasonId),
eq(schema.scoringEvents.eventType, "playoff_game"),
eq(schema.scoringEvents.bracketTemplateId, LLWS_TEMPLATE_ID)
),
});
const bracketEvent = playoffEvents.toSorted(
(a, b) => (b.createdAt?.getTime() ?? 0) - (a.createdAt?.getTime() ?? 0)
)[0];
const bracketMatches = bracketEvent
? await db.query.playoffMatches.findMany({
where: eq(schema.playoffMatches.scoringEventId, bracketEvent.id),
})
: [];
const bracket = readBracketSlots(bracketMatches, teamsById);
// Validate sides. A seeded bracket already fixes the draw and an even 10/10 split,
// so externalId only has to be usable on the randomized pre-bracket path.
if (!bracket) {
const [firstBad] = unparseableSides;
if (firstBad) {
throw new Error(
`Participant ${firstBad.id} has invalid externalId "${firstBad.externalId}". ` +
`Expected: "US" or "Intl".`
);
}
const usTeams = teams.filter((t) => t.side === "US");
const intlTeams = teams.filter((t) => t.side === "Intl");
@ -291,8 +617,15 @@ export class LLWSSimulator implements Simulator {
if (intlTeams.length !== INTL_TEAM_COUNT) {
throw new Error(`Expected ${INTL_TEAM_COUNT} International teams, found ${intlTeams.length}.`);
}
}
// 5. Initialise placement count accumulators for all participants.
const usPool = bracket ? bracket.slots.US : teams.filter((t) => t.side === "US");
const intlPool = bracket ? bracket.slots.Intl : teams.filter((t) => t.side === "Intl");
const playUS = makePlayGame(SIDE_INDEX.US, bracket, parityFactor);
const playIntl = makePlayGame(SIDE_INDEX.Intl, bracket, parityFactor);
// 6. Initialise placement count accumulators for all participants.
const allIds = participants.map((p) => p.id);
const counts = new Map<string, PlacementCounts>(allIds.map((id) => [id, zeroCounts()]));
const bump = (id: string, key: keyof PlacementCounts) => {
@ -300,27 +633,33 @@ export class LLWSSimulator implements Simulator {
if (entry) entry[key]++;
};
// 6. Run Monte Carlo simulations.
// 7. Run Monte Carlo simulations.
for (let s = 0; s < numSimulations; s++) {
// The draw is modelled as random: shuffle each side into the 10 bracket slots
// (8 opening-round teams, then the 2 bye teams).
// With a real bracket the draw is fixed; without one it is modelled as random.
const usSlots = bracket ? usPool : shuffle([...usPool]);
const intlSlots = bracket ? intlPool : shuffle([...intlPool]);
const { sideChampion: usChamp, sideLoser: usLose } =
simulateSideBracket(shuffle([...usTeams]), bump);
simulateSideBracket(usSlots, bump, playUS);
const { sideChampion: intlChamp, sideLoser: intlLose } =
simulateSideBracket(shuffle([...intlTeams]), bump);
simulateSideBracket(intlSlots, bump, playIntl);
// Consolation game: 3rd / 4th place.
const consolation = simGame(usLose, intlLose);
const consolation = playCrossoverGame(
"Consolation Third Place", bracket, parityFactor, usLose, intlLose
);
bump(consolation.winner.participantId, "thirdPlace");
bump(consolation.loser.participantId, "fourthPlace");
// World Championship: 1st / 2nd place.
const ws = simGame(usChamp, intlChamp);
const ws = playCrossoverGame(
"World Championship", bracket, parityFactor, usChamp, intlChamp
);
bump(ws.winner.participantId, "champion");
bump(ws.loser.participantId, "finalist");
}
// 7. Convert counts to probability distributions.
// 8. Convert counts to probability distributions.
// Each of the two 58 tiers takes exactly 2 teams per sim (one per side), and
// the teams within a tier are tied, so the tier probability is split across
// its two positions.

View file

@ -183,10 +183,12 @@ const PROFILES: Record<SimulatorType, Omit<SimulatorManifestProfile, "simulatorT
setupSections: ["participants", "eloRatings", "futuresOdds", "bracket"],
},
llws_bracket: {
defaultConfig: { ...BASE_CONFIG, usTeamCount: 10, internationalTeamCount: 10 },
defaultConfig: { ...BASE_CONFIG, parityFactor: 550, usTeamCount: 10, internationalTeamCount: 10 },
requiredInputs: ["sourceOdds"],
optionalInputs: ["metadata"],
setupSections: ["participants", "futuresOdds"],
// The bracket is optional — without one the draw is randomized — but once it
// exists the simulator reads the real draw and honors completed results from it.
setupSections: ["participants", "futuresOdds", "bracket"],
},
college_hockey_bracket: {
// College hockey blends odds into Elo internally (and also uses NPI rank,

View file

@ -155,7 +155,7 @@ const REGISTRY: Record<SimulatorType, { info: SimulatorInfo; create: () => Simul
llws_bracket: {
info: {
name: "LLWS Bracket Monte Carlo",
description: "Simulates the 20-team Little League World Series: a 10-team double-elimination bracket per side (US & International), each producing a side champion, then the consolation game (3rd/4th) and the World Championship (1st/2nd). Uses championship futures odds for all win probabilities. Set externalId to 'US' or 'Intl'.",
description: "Simulates the 20-team Little League World Series: a 10-team double-elimination bracket per side (US & International), each producing a side champion, then the consolation game (3rd/4th) and the World Championship (1st/2nd). Championship futures odds are decompressed to single-game Elo. When an llws_20 bracket exists it simulates the real draw and honors completed results; otherwise the draw is randomized and externalId ('US' or 'Intl') sets the sides.",
},
create: () => new LLWSSimulator(),
},