Sam's Scores

Model performance

This project's whole discipline rests on keeping tuned-against data separate from a true holdout — so these are shown as separate reports, never blended into one number. 2026 Season is the live, ongoing season — real predictions graded in real time as games finish. Validation (2023-2024) is the data every modeling decision in this project was tuned against. Test (2025) is sealed: never touched by any tuning decision, looked at exactly once, after model development was finished.

Active model version
sams-scores-router-simulator v1
Trained on 2014–2026 · family: simulator

Graded against 386completed games. Confidence intervals here use a normal approximation (mean ± 1.96 × standard error), not the pipeline's percentile bootstrap — see the methodology page.

Overall accuracy
MAE margin: 12.12 [11.19, 13.04]
MAE total: 12.63 [11.72, 13.55]
Benchmark (opening line) MAE margin: 11.58 [10.66, 12.49]
Benchmark (opening line) MAE total: 12.28 [11.35, 13.22]
Straight-up win accuracy

How often the predicted winner was actually the winner — a different question from MAE, which only measures how close the score was.

Overall: 74.6% [70.3%, 79.0%]
Market favorite: 76.9% [72.7%, 81.2%]
Confidence levelAccuracy
Toss-up56.5% [47.4%, 65.6%]
Leansample too small
Likely83.2% [75.8%, 90.5%]
Confidentsample too small

A well-calibrated model should get more accurate as confidence increases — if "confident" calls aren't noticeably better than "toss-up" ones, the confidence label isn't telling you anything real.

Reliability

Predicted win probability vs. observed win rate, 5% bins. Dot size is game count.

MAE trend by week
Win accuracy trend by week

Each week is only ~40-70 games, so individual points are noisier than the segment tables below — this is a trend line, not a table of trustworthy standalone statistics.

Segmented accuracy

Below 100games in a segment, the guard shows "sample too small" instead of a number.

By conference tier

SegmentnMAE marginMAE totalBenchmark MAE margin
Power23912.78 [11.54, 14.02]12.23 [11.13, 13.33]11.99 [10.81, 13.17]
Group of 514111.01 [9.64, 12.38]13.37 [11.72, 15.02]10.93 [9.44, 12.42]
Independent6sample too smallsample too smallsample too small

By week bucket

SegmentnMAE marginMAE totalBenchmark MAE margin
weeks 1-421312.04 [10.82, 13.27]12.00 [10.79, 13.21]11.22 [10.03, 12.41]
weeks 5-917312.21 [10.80, 13.62]13.42 [12.02, 14.81]12.01 [10.59, 13.43]

By predicted margin magnitude

SegmentnMAE marginMAE totalBenchmark MAE margin
blowout20712.42 [11.23, 13.62]12.58 [11.32, 13.84]11.79 [10.63, 12.96]
close17911.76 [10.33, 13.20]12.70 [11.36, 14.03]11.32 [9.88, 12.76]
Model version history
NameVersionTrainedHoldout MAE marginHoldout MAE totalActive
sams-scores-router-simulatorv110/18/2026——active