QuantCoin

Survivorship Bias in Crypto Datasets

Survivorship bias is what happens when your dataset only contains the assets that made it. Every backtest run on today’s list of tradeable coins is implicitly conditioned on those coins still existing, which is information the strategy would never have had. In crypto the effect is unusually severe, because the failure rate is high, delistings are frequent, and the datasets most people can get hold of are built by asking a venue what it trades right now.

Educational material, not trading advice. Algorithmic crypto trading is high-risk and most retail algo traders lose money.

Why the bias is worse here than in equities

Equity survivorship bias is a known, well-documented problem with commercial solutions; crypto’s version is the same problem with none of the infrastructure. Listed companies that die leave a paper trail — a delisting notice, a merger record, an index-membership history that vendors sell. A token that dies often just stops having volume. There is no corporate registry, no exchange-mandated announcement archive you can rely on, and no widely used point-in-time universe file.

Three properties compound it:

  • The universe churns fast. New pairs appear constantly and old ones are quietly retired. A list from a year ago and a list from today are meaningfully different populations, not the same one with a few edits.
  • Failure is total. An asset can lose essentially all of its value and all of its liquidity. In a survivor-only sample that outcome is simply absent, and it is exactly the outcome that dominates real-world risk.
  • Venue fragmentation hides deaths. A pair delisted from one venue may still trade elsewhere, so “did this asset die?” has a per-venue answer, not a global one. Your dataset’s boundaries determine what counts as a death.

What the bias does to a result

Survivorship bias inflates returns and deflates risk, and it does so most for exactly the strategy families people are most excited about. A cross-sectional momentum rule that ranks a basket of coins and buys the strongest is the clearest case: if the basket only contains assets that had a future, then “strong recently” is being measured on a population pre-filtered for not going to zero. The rule looks like it identifies winners when the sample has already done that job.

Mean-reversion suffers a mirror-image distortion. Buying weakness works beautifully when weakness is always temporary — which is what a survivor-only dataset guarantees, since permanent weakness removed the asset from the sample. The result is a strategy that appears to profit from dips because every dip in the data eventually recovered. For the underlying logic of both families, see momentum vs. mean-reversion.

Risk metrics are hit too. Maximum drawdown, volatility, and tail measures are all computed on a series that never contains a terminal loss. The number you report is not a conservative estimate of the real risk; it is a measurement of a different, friendlier world. That matters when the metric is feeding a sizing decision — see position sizing and risk management basics.

What a point-in-time universe actually is

A point-in-time universe is a function from a date to the set of assets that were tradeable on that date. Not “the assets that exist now and have data going back that far” — the assets that were actually available, including the ones that later disappeared. Conceptually:

def universe(as_of):
    return {
        sym for sym, (listed, delisted) in listings.items()
        if listed <= as_of and (delisted is None or as_of < delisted)
    }

The whole difficulty is populating listings honestly. Two dates per symbol — first available and last available — plus the price history up to that last date is enough to remove the bulk of the bias. You do not need a perfect corporate-action history; you need to stop pretending the dead never traded.

Building one from imperfect data

You can approximate a point-in-time universe from raw history, and an approximation beats ignoring the problem. The technique is to derive listing and delisting dates from the data’s own coverage and liquidity rather than from a vendor’s metadata:

  • Snapshot the universe on a schedule. The cheapest fix available to anyone starting today: once a week, record the full list of pairs a data source reports, with a timestamp, and never delete an old snapshot. In a year you own a point-in-time file nobody can sell you. Do this before you need it.
  • Derive listing dates from first observation. The first bar with real volume is a serviceable proxy for when a pair became tradeable. Be careful with sources that back-fill synthetic zero-volume bars.
  • Derive delisting from the end of coverage. If a symbol’s history stops and never resumes, treat the stop as a delisting rather than as a gap to be forward-filled. Forward-filling a dead asset’s last price is how a zero becomes a flat line.
  • Model the exit, don’t drop it. When an asset leaves the universe, the position in it has to go somewhere. Assume an exit at a punitive price, or assume total loss, and see whether the strategy survives either. Silently removing the row assumes a costless exit that nobody got.
  • Keep the zero-volume rows. A pair with no liquidity is not tradeable, and a backtest that fills orders in it is inventing a counterparty. Liquidity screens belong to the universe definition, not to a separate cleaning step.

Where survivorship hides in code

The bias usually enters through a convenience, not a decision. Watch for these patterns:

  • dropna() across a wide frame. Requiring every symbol to have data on every date reduces the panel to the intersection of long-lived assets — a survivorship filter applied by accident, in one method call.
  • “Top N by market cap” computed once. Ranking on today’s values and then applying that ranking to the whole history is lookahead and survivorship simultaneously. Rankings must be recomputed as of each date. The general form of that mistake is covered in auditing your code for lookahead bias.
  • A hand-curated symbol list. Any list a human typed reflects what the human has heard of, which is a survivorship filter with extra steps.
  • Minimum-history requirements. “Only include assets with at least two years of data” excludes everything that died young — which is most of what died.

The honest version is less exciting

Rebuild a study on a point-in-time universe with modelled exits and the results get worse, sometimes dramatically. That is not a failure of the rebuild. The survivor-only number measured a population selected using the future, and no downstream validation repairs a sample chosen that way — holdouts, walk-forward splits, and cost models all inherit the same flattering universe.

The practical takeaway is unglamorous and worth acting on immediately: start snapshotting your universe today, keep the dead assets, and treat every result computed on a survivor-only sample as an upper bound rather than an estimate.

market-databacktestingpython