Holdout SetEvery hypothesis registered before it was scored. Every verdict published.

Standing rules

Doctrine

These are not aspirations. Each was written down after a specific failure, and each is enforced somewhere in code, in a pre-registration, or in a build that refuses to run.

  1. Fix the bar before you see the score.

    A threshold chosen after the result is not a threshold, it is a description. Every hypothesis here is registered in a dated commit — the metric, the bar, the horizon, and the conditions that void it — before the scoring run exists.

    DR-26

  2. One run. The verdict is final.

    A hypothesis gets a single scoring run against its registered bar. Re-running with a nudged threshold is how a null result becomes a discovery. Where a re-registration is permitted at all, it is granted once, in advance, and spent deliberately.

    DR-27

  3. Metric first, engine second. Never the reverse.

    No strategy reaches implementation until the underlying signal has cleared the cost hurdle out of sample. Building the engine first means you have a system you are motivated to believe in before you have evidence it works.

    DR-24

  4. A safety mechanism coupled to the thing it protects is not a safety mechanism.

    A position was once stranded overnight because the process that should have closed it was the same process that had died. Protection now runs independently of the thing it protects, and does not depend on anyone remembering to look.

    DR-8, DR-11

  5. A verification that cannot fail loudly has not verified anything.

    A test once passed because its script was fed an empty input and exited zero. It proved nothing and looked identical to success. Every check must be observed failing before its passing is worth anything.

    DR-35

  6. Verify your own findings as harshly as someone else’s.

    The most flattering result this programme ever produced was a bug, and fixing it inverted the conclusion. An external review caught it by reading the code. Results that confirm what you hoped get more scrutiny, not less.

    DR-19

  7. History is append-only.

    Retractions are recorded, never rewritten. Two verdicts on this site were withdrawn after a data defect was fixed; both the original and the correction remain published. A record you can edit is not a record.

    DR-27

  8. Pooled effect is not harvestable return.

    A signal measured at several times its cost hurdle still lost to a costless buy-everything benchmark once constructed as an actual book. Overlapping windows, selection effects, and turnover eat the gap between a measurement and a strategy.

    DR-29

  9. The instrument gets tested against real data, including the data you trust.

    Acceptance tests written to catch bad corporate-action adjustments instead found an eight-month hole in the archive nobody had noticed. Tests written against real data find the failure you did not think to register.

    DR-25

  10. Allocate on evidence, never on prediction.

    Any future allocation across strategies weights what is measurably working, rather than forecasting which regime is coming. Regime classifiers are parameter-heavy, brittle, and fit beautifully to the past.

    DR-16