Auditing Your Code for Lookahead Bias

Lookahead bias is not really a statistical idea — it’s a bug. It happens when a line of code hands your strategy a number it could not have known at the timestamp it claims to be making a decision. Because the leak lives in indexing, joins, and preprocessing rather than in the strategy logic you’re staring at, the only reliable way to find it is to audit the data path deliberately. This post is that audit.

Educational material, not trading advice. Algorithmic crypto trading is high-risk and most retail algo traders lose money.

Start from one question, asked of every value

The audit is a single question repeated: for each number the strategy reads at time t, would I have known that number, in exactly that form, at time t?

That phrasing does a lot of work. “Would I have known it” catches future bars. “In exactly that form” catches revised, adjusted, or later-corrected values. “At time t” catches delays — a number that existed but hadn’t reached you yet. If you cannot answer yes with confidence, treat it as a leak until proven otherwise.

The useful mental model is a replay: imagine your backtest as a tape being fed forward one row at a time, with everything after the current row physically unavailable. Any code that could not run under that constraint is suspect, even if it happens to produce the same answer today.

The off-by-one bar: decide on one, fill on the next

The most common leak is using a bar’s close both to generate a signal and to execute. You only know a candle’s close once the candle is finished, so the earliest realistic execution is the next bar. If your fill price is the same close that produced the signal, you have given the strategy a free look at the outcome.

In a vectorized backtest this is one shift:

signal = (close > close.rolling(20).mean()).astype(int)
position = signal.shift(1)            # act on the next bar
returns  = position * close.pct_change()

Two details are easy to get wrong. First, pct_change() on close-to-close pairs with a shifted position implicitly assumes you traded at the close of the signal bar’s successor — coherent, but you should know which convention you’ve adopted and write it down. Second, if you compute the signal from high or low, the leak is worse than one bar: an intrabar extreme is only known after the bar completes, so a rule like “buy if today’s low touched the band” is unimplementable as written.

A quick smoke test: shift the position by one more bar than you think is correct. If a large fraction of the result disappears, the edge is concentrated at the boundary between decision and execution, which is exactly where lookahead lives.

Preprocessing that leaks the whole history

Any transform fitted on the full dataset before splitting leaks the future into the past. Standardizing a feature by the mean and standard deviation of the entire series means every early row carries information about later ones. So does min-max scaling to the global range, and so does a PCA or a clustering fitted on everything.

The fix is to fit transforms only on data available at the time, which usually means expanding or rolling statistics:

mu    = feature.rolling(500).mean()
sigma = feature.rolling(500).std()
z     = (feature - mu) / sigma        # no global fit

Missing-value handling is the same trap wearing a different hat. fillna(method="bfill") copies a future observation backwards. Interpolation across a gap uses both endpoints, one of which is in the future. Dropping rows is often honest; filling forward is usually honest; filling backwards almost never is.

Resampling, joins, and timestamp semantics

Resampling silently chooses which end of an interval a label refers to. A bar labelled 12:00 may summarize 12:00–12:59 or 11:01–12:00 depending on the label and closed arguments. If you assume the label is the interval’s start but your library uses the end, every bar in your dataset is stamped an interval too early and the entire backtest reads the future. This is worth a dedicated look — see resampling OHLCV candles without corrupting them.

Joins across sources have the same failure mode with more moving parts. Merging a daily series onto an hourly one with a plain index join will, in most naive implementations, attach a day’s value to hours that preceded its availability. merge_asof with an explicit backwards direction and a tolerance is the pattern that respects arrival order; a raw join is not.

Timezones deserve one line of paranoia: mixing a UTC series with a local-time series shifts one relative to the other by whole hours, which is lookahead in one direction and lag in the other.

Data that didn’t exist yet

Some leaks aren’t in your code at all — they’re in the dataset. A price series that has been retroactively cleaned, deduplicated, or corrected contains information the live version didn’t have. Fundamental or on-chain metrics are often restated. A token’s list of trading pairs as it exists today is not the list that existed historically, which is the survivorship problem discussed in survivorship bias in crypto datasets.

Delay is the subtle sibling. Even a value that was never revised arrived at some point after the event it describes. If your backtest reads it at the event timestamp rather than the availability timestamp, it is trading on information that was still in flight. Where you don’t know the delay, model a pessimistic one and see whether the result survives.

Making the audit structural

A leak you can only find by reading code will come back. Build the defence into the harness instead:

Feed data forward. Where practical, run the strategy through an interface that physically cannot index past the current row. An event-driven loop gets this for free, which is one of its main advantages over a vectorized run — see event-driven vs. vectorized backtests.
Assert on timestamps. Add checks that every feature’s timestamp is less than or equal to the decision timestamp, and that no fill precedes its own signal.
Test with a deliberate leak. Insert a feature that is obviously the future — tomorrow’s return — and confirm the harness flags or refuses it. A harness that happily accepts a perfect oracle will happily accept a subtle one.
Re-run with extra lag. Keep a “paranoid mode” that adds one bar of delay everywhere. Compare it to the base run as a matter of routine, not as a debugging step.

The uncomfortable part of this audit is that it always makes results worse. That is the point: the naive version was never a measurement of your strategy, only of your indexing. Everything else you might do to validate an idea — holdouts, walk-forward, honest cost models — is wasted effort until the data path is clean, because a leak flatters every one of those tests too.