Measure Before You Predict

Question

Is a from-scratch, continually trained, collapse-proof byte-level text encoder a usable forecaster of its own input stream — or does the geometry that prevents collapse forbid forecasting by construction?

Context

Nostradamous asked whether JEPA-style representation learning plus forecasting could share one latent. LeJEPA SIGReg and related JEPA work push embeddings toward full-rank, isotropic geometry to fight collapse (JEPA and SIGReg for Memory Systems). Complex Phase Representation names the seam: coding-rate / volume terms spread states (good for retrieval); forecasting needs temporal neighbors close — adjacency vs separation fight over one real geometry.

Prior ideas under test:

• Collapse-proof metrics only: single-seed retrieval bumps do not count as progress.
• Minimal relational objective (InfoNCE + coding-rate log-det) may beat heavier SIGReg / MSE pull stacks for representation quality.
• If the transition head only ties persistence, the encoder succeeded at the wrong job for forecasting.

Findings

• Minimal relational objective (InfoNCE + coding-rate log-det at β = 1) yields an excellent representation: held-out retrieval 0.977 at effective rank 31/64, vs 0.66 at rank 9 for fresh init. β sweep peaks exactly at β = 1.
• It does not yield a forecaster. Under multi-seed margins (collapse-proof; single-seed bumps do not count), a trained transition head ties persistence — predict-the-present — on a verified rank-31 encoder.
• Predictability probe: recency-retrieval flat at ~1.5–2× chance across horizons, short fragments and long documents alike. DMD of the transition operator: ~59 modes at |λ| ≈ 1, identical across corpora.
• Structural diagnosis: the coding-rate term spreads states to fill space (good for retrieval); forecasting needs temporal neighbors close. One geometry cannot serve both by default.
• SIGReg, entropy floor, covariance, scale-cap, and all MSE pull-terms measured redundant with or harmful to the log-det volume term.

Method

Byte-level encoder trained from scratch on MLX under one rule: every claim reduces to a measurement; only collapse-proof metrics count. Multi-seed margins for claimed wins and DMD spectra for the transition operator reached the negative forecasting verdict in minutes of compute, not weeks of predictor engineering.

Limits

• Verdict is for this encoder class and objective stack — not a proof that no forecasting world model can work.
• Complex-phase / anti-Hermitian transport (Complex Phase Representation) is a proposed next geometry; this paper measures the real-valued collapse-proof stack that failed forecasting.
• Paper, theory notes, and run records stay in the private monorepo; the public surface is the hash-pinned dashboard snapshot.
• Live process / optimizer pulse views are secondary instruments, not the claim surface.

Artifact

Executed snapshot of the project’s collapse-proof evidence surface (live_dashboard.py) against local run history at the pinned source commit. Leads with a research brief, multi-seed forecast margins, and retrieval/rank instruments; optimizer pulse and live-process views are secondary. Notebook bytes are hash-pinned above.