Parameter Golf — Run Leaderboard

Question

Which Parameter Golf runs currently lead on quantized validation bits-per-byte, and what lineage moved the needle from the ROCm baseline?

Context

Live board for Parameter Golf. Quantized validation BPB is the number that survives deployment-style quantization — the competition-relevant meter. Float BPB sits beside it for diagnosis.

Research framing from the timing literature (prioritization research):

• The board is a lineage tracker, not a controlled A/B table — step counts, wallclock caps, and parent deltas differ by run.
• Early-signal promotions (partial RoPE, XSA, EMA, capacity) should appear as parent→child improvements; late-signal bets need longer runs before the board can judge them.
• Smoke runners without quantized BPB stay unranked so null metrics do not pollute the leader.

Findings

The sealed notebooks/public/results.json snapshot holds eight rows; six are ranked training runs and two are one-step smoke runners without quantized BPB.

• ROCm-patched baseline: 2.067 quantized val BPB after 251 steps under a 600-second wallclock cap on one MI300X (17.1M parameters).
• Current leader exp004_partial_rope_promo: 1.295 quantized val BPB over 2,463 steps at the same 17.1M budget.
• Between them: exp004 clean (1.403), exp001 with 11 layers + 3×MLP at 26.5M (1.431), exp003 EMA (1.491), exp002 XSA-4 (1.514).

Step counts and budgets differ. Each run’s change field records what moved relative to its parent.

Method

Scans synced result.json artifacts from RunPod / MI300X experiments (local checkout) or the sealed notebooks/public/results.json bundle (published WASM). Renders experiment lineage, parent delta, quantized and float validation bits-per-byte, step time, and parameter count. Null-BPB smoke runs are listed separately from the ranked board.

Limits

• Not a factorial experiment: different step budgets and parents mean deltas are historical, not causal estimates.
• Sealed public snapshot may lag the newest remote runs/**/result.json until the bundle is refreshed and rehashed.
• Leaderboard BPB numbers are not interchangeable with Technique Explorer timing anecdotes without matching train configs.

Artifact

Fully interactive: the published notebook runs Python in the browser via WebAssembly, rendering the sealed notebooks/public/results.json bundle pinned at the source commit. Local checkouts scan runs/**/result.json instead, so the same notebook serves both surfaces.