Agora CompactCTM Shadow — Stage-1 Prediction Without Control

Question

Can a CompactCTM shadow predictor beat a simple empirical prior on held-out Agora transitions?

Context

Stage-1 maps CompactCTM ideas onto Agora Markov transitions with no control authority and no Sakana CTM paper parity claimed. In the Recursive Self Improvement Lab, this is a Club B measure/falsify result: a CTM-inspired proposer failed an external scoring gate, so it stays unpromoted. The held-out answer is no, which is why the report stays public as a negative result. Related Club B study surface: Continuous Thought Machines research atlas (disputed local custody, not paper parity).

Findings

• Claim class non-parity-agora-derivative; control_authority: false; promotion decision reject.
• Held-out CompactCTM mean Brier 0.6277 vs empirical prior 0.3578 (Δ −0.270); log loss also worse (1.041 vs 0.605). Not promoted.
• Circular rule-oracle Brier 0.2036 is recorded as a diagnostic ceiling and is promotion-ineligible.
• Lineage-atomic split: train 1203 / val 658 / test 733; held-out runs od-base-seed-101 and od-base-seed-303.
• Independent audit (agora-ctm-shadow-04): infrastructure reproducible; learned CompactCTM does not beat the empirical prior on held-out; do not promote.
• For RSI Lab readers: architectural resemblance to CTM plus passing unit tests is not held-out prediction value and is not inheritance.

Method

Shadow predictor trained and scored inside the private agora-agent-brain tree against opinion-diffusion transition samples. External Brier / log loss gates decide promotion; the model never steers the live simulation. This report publishes the sealed negative result, not a brain that improves Agora dynamics.

Let $B(\hat{p}, y)$ be mean Brier score on the held-out transition set. Promotion requires

under the frozen split. The observed scores violate that inequality.

Limits

• The model never steers live simulation.
• The circular rule-oracle is diagnostic only and promotion-ineligible.
• No CTM paper parity or learned control claim is made.
• Failure here does not refute Sakana’s CTM paper results; it falsifies this Agora derivative under its own gate.

Artifact

Sealed metrics and audit JSON live under private agora-agent-brain/artifacts/disposable-ctm-stage1/ (checkpoint compact-ctm-76cf99c7bee8d4af). Marimo surface: experiments/ctm_shadow.py. Repos stay private (source_public: false); no public notebook pin in this writeup.