Survivorship & Look-Ahead Bias in Backtesting

Survivorship bias is testing a strategy only on the assets that survived to today - silently ignoring the ones that failed, delisted, or went to zero - which makes almost any strategy look better than it really was.

If you backtest a 'buy and hold the biggest coins' rule using today's list of big coins, you've baked in hindsight: you already know which ones made it. The coins that looked just as promising years ago and then collapsed never enter the test, so the strategy never 'buys' them and never takes those losses.

The same trap hits stock strategies tested only on companies still in an index today - the bankruptcies and delistings have been scrubbed out.

Look-ahead bias is when a strategy uses information it couldn't have known at the moment of the trade - quietly importing the future into a decision made in the past.

The classic version: a rule that 'buys at the day's low' or uses the closing price to make a decision earlier in that same day. In hindsight the low and the close are obvious; in real time you don't know them until the period is over.

Even subtle versions - using data that was revised after the fact, or an indicator that secretly peeks one bar ahead - can turn a losing strategy into a stellar backtest that's impossible to replicate live.

Guardrails that keep a backtest honest:

• Include the losers: test on data that contains assets which later failed or delisted, not just today's survivors.
• Step through time strictly: at each point, the rule may only use information available up to that moment - never the period's close or low while it's still forming.
• Use point-in-time data where possible, so you're not using figures that were revised after the fact.
• Be suspicious of any entry or exit that conveniently happens at perfect prices.

These biases are insidious precisely because the output looks clean. A strategy can show a flawless equity curve that was never achievable, and you'd never know from the numbers alone.

Good backtesting is, in large part, the discipline of refusing to use information you couldn't have had - and testing on the full universe, winners and losers alike.