Overview
⌄Acceptance bar
⌄The bar the proposal set for Stage A (absent or not), against the best cross-validated model and the promoted model on the holdout.
Leaderboard
⌄Pooled out-of-fold results on the purged rolling-origin folds. Click a column to sort. The holdout is untouched here.
Classification at the F1-optimal cut
Holdout
⌄Three-class check on the holdout (promoted pair)
Predictions against outcomes on the holdout, day by day
Holdout: daily expected absentees (promoted model) vs actual
Workers on one day
Invitation sizing (Layer 2)
⌄The concept
How it runs for Alakili
Replay against what the team did
Cost of unneeded shows against uncovered slots, by policy
Weekly saving against the team, by target (USD)
Checks
Holdout day by day
Shows minus Amazon's requirement per holdout day: what happened with the team's schedule, and what would have happened with the model's
Charts
⌄Reliability: predicted vs observed, ten bins
Brier per validation window
Daily expected count vs actual (promoted model, out-of-fold)
Feature importance (promoted model, top 15)
Ablation: Brier change when a feature family is removed (%). Green: the score got worse without it, so the family helps. Grey: no worse without it. The whole axis is under 1%, inside the ±3.5% noise on any single score, so grey means no measurable effect, not harm; weather and calendar stay in the served model regardless.
Tuning: best pooled Brier by trial
Models
⌄What each candidate is, how it was tuned and calibrated, and which version is on disk.
Data
⌄Rows and absence rate by month
Label sources by period
Folds
Decisions
⌄Log
Audit trail
⌄Every evaluation recorded in the ledger (data/results/ledger.jsonl): cross-validation and stack runs, holdout candidates, promotions and Layer 2 replays, newest first. The results folder keeps only the latest run of each model; the ledger keeps them all, so any past version can be traced and compared.
Compare two records
Metric by version
Cross-validation and stack runs as solid lines, holdout candidates dashed; one point per run day.
Stage A
Stage B
Promotions
Feature dictionary
⌄What each input to the models means. Every feature uses only information available before the scoring date. "Cold start" is the value a worker gets with no history.
Run a prediction
⌄Calls the prediction API (POST /v1/predictions). Works when this page is served by the API or the API address below is reachable.
Run a weekly schedule
⌄Calls POST /v1/schedule with every day of the week: Amazon's count and the walkers who declared availability that day. Returns how many more slots to open per day and the total, with each day's per-walker scores. Uses the API address and key entered in the prediction panel above.