Backtesting on a News Archive You Queried Today

Text feeds are the most-hyped alternative data a small research shop can actually get: headlines, publisher names, summaries, a date per story. The pitch writes itself — the market reacts to news, so a feed of news must contain something. What that pitch skips is that a news feed is a data-engineering problem long before it is an alpha source, and the engineering failures all point the same direction, which is toward a backtest that looks better than the world.

Educational material, not trading advice. Algorithmic crypto trading is high-risk and most retail algo traders lose money.

Three timestamps, and only one of them is yours

Every news item has at least three distinct times, and a backtest that confuses them is running on information nobody had. They are:

Publication time — what the publisher stamped on the article.
Index time — when an aggregator or search index first observed it and made it findable.
Observation time — when your own process, running its normal schedule, would have had the item in hand.

Only the third one is a legitimate feature timestamp. The first is a claim made by an interested party; the second belongs to an intermediary whose crawl cadence you do not control. The gap between publication and observation is not a rounding error either: an item published at 09:00 that a crawler surfaces at 09:11 and your poller collects at 09:15 is a fifteen-minute head start you will silently grant yourself if you align features on the publication stamp. On an intraday horizon that is the entire effect you were trying to measure.

The general form of this mistake — indexing a feature by when the information was created rather than when it was available — is the same one covered in auditing your code for lookahead bias. News just makes it easy to commit, because the archive hands you the wrong timestamp as its primary one.

Publication time is not an event time either

Even taken at face value, a publication stamp is a noisy, mutable, sometimes back-dated field. Articles get edited after the fact while keeping the original stamp. Others get republished with a fresh stamp and identical text. Some sources give you a date with no time at all, some give a relative string (“3 hours ago”) that has already been rounded before you parse it, and timezone handling varies by publisher. A liveblog updated forty times carries one timestamp for forty different pieces of information.

Practical consequences worth internalising: never treat the stamp as precise beyond the precision it was actually reported at, always store the raw string alongside your parsed value, and normalise everything to UTC at ingest rather than at analysis time. If your feature is bucketed to the hour and a meaningful share of your stamps are day-resolution only, those items are being placed at midnight — which for a 24/7 market is a real position at a real time.

The archive is a survivor sample

Querying a historical window today returns what survived to today, not what existed then. Stories get deleted, outlets go dark and take their archives with them, paywalls close over old content, headlines get rewritten, and near-duplicates get collapsed by the index long after publication. Ranking changes too: what an index returns for a query is a function of the index’s current state, so the “top results” for a date in the past are chosen partly by relevance signals that accumulated after that date.

This is exactly the structure described in survivorship bias in crypto datasets, transplanted to text. The direction of the bias is unfriendly: stories that mattered got linked, cited, and preserved, so a query-today archive is enriched in items that turned out to be important. A feature built on it is quietly conditioned on the future.

There is no clever correction for this. The only real fix is to stop backtesting on retrospective queries and start accumulating your own append-only capture — poll on a schedule, write every payload with your own receipt timestamp, and never overwrite. A year of honest capture is worth more than a decade of archive you pulled last week.

Forty outlets, one story

A wire story republished across forty sites is one event, and a naive count treats it as forty. This is the single most damaging modelling error in news features, because “article volume” is the first feature everyone builds and syndication is precisely what inflates it. What you end up measuring is how syndicatable a story was — a property of press-release distribution and outlet economics — rather than how significant it was.

Deduplication is the unglamorous core of the pipeline. A workable approach, in order of cost:

Exact-match on normalised title and body. Lowercase, strip punctuation and boilerplate, hash. Catches straight reprints, which are a large share of the problem.
Near-duplicate detection on shingles. Break the text into overlapping token n-grams, hash them, and compare sets with Jaccard similarity or MinHash. Catches reprints with a changed headline or a trimmed lede.
Cluster within an entity-and-time window. Group by shared entities inside a few-hour bucket, then dedupe within the group. Cheaper than comparing everything to everything, and it matches how the same event actually propagates.
Take the earliest observation in a cluster as the event. Then count clusters, not articles. If you also want a spread measure, count distinct publishers per cluster as its own feature rather than folding it into volume.

Note what the last two steps need: entities and publisher, per item, not just a blob of text. That shape is standard — the field list for Serply’s Google News endpoint covers publisher, date, summary, and entities per result — and it is why the metadata matters more than the prose for a first pass. Note also that one date field per result means the pipeline, not the provider, is responsible for recording when you saw it.

What to build before you build a signal

Before any feature reaches a strategy, the pipeline should be able to answer: when did we receive this, what did the raw payload look like, which cluster does it belong to, and which item in that cluster came first. Store all four. The same reasoning applies here as in a reproducible workflow for strategy research — if a feature cannot be regenerated bit-for-bit from stored inputs, its backtest is not a result.

And apply the same integrity checks you would to price data — gaps, duplicates, timezone drift, sudden changes in row counts that mean the source changed rather than the world did. The checklist in data quality checks for crypto price history transfers almost unchanged.

The takeaway is deflationary on purpose. Most of the work in an alternative-data feed is timestamp hygiene, deduplication, and honest capture, and none of it produces a signal. It only stops you from producing a fake one — which, given how flattering a retrospective news archive is, is the higher-value job.