The stocks that disappeared, and why they matter
Every figure in this note assumes no trading costs unless a paragraph says otherwise.
Last week I said my price data mostly drops companies that were delisted, and that this flatters every strategy. This week I want to explain why that matters so much, because it's the most common way a backtest lies without anyone meaning it to.
What goes missing
Stocks leave the market all the time. Some are bought out, some merge, and some go bust. When that happens, most price sources quietly stop listing them, and their history goes too.
So a backtest run today only sees the companies that made it to today. That's like judging a fund manager only on the funds they didn't close.
How big the gap is
I counted. Of the 24,015 US stocks and funds I can find, 9,338 have been delisted. That's 39%, nearly four in ten.
My usual price source has history for just 152 of those 9,338. So my tests so far have mostly been run on the survivors.
Why it flatters a strategy
The companies that vanish include many of the worst falls in the market. A pullback strategy buys stocks that have dropped. Some of those drops keep going to zero, and the stock is delisted. If those stocks aren't in the data, the strategy never buys them in the backtest, so it never takes the loss.
That isn't a small rounding error. It removes some of the worst trades a dip-buying rule would ever make.
What I found when I tested some of them
Last week I ran the pullback strategy on 192 delisted stocks, using the earlier cost assumptions. None of them passed the trader test. That isn't like for like with the survivors, so I'm not calling it proof of failure. But it's the direction you'd expect if the survivors were flattering the results.
What I'm doing about it
Prices for delisted stocks are in the pipeline. Once there's enough data, I'll re-run the main tests with the dead companies included, put the old and new numbers side by side and post an update.
If the results get worse, I'll say so here. If a strategy only works when the failures are hidden, it doesn't work.
The five headline stocks
The forward test on PRU.L, BANFP, BTO.TO, APA and ALLAN.PA is still running, and I'll report back on it in a few months' time.
A question to ask any backtest
When you see a backtest anywhere, mine included, ask one thing: did it include the stocks that no longer exist? If it doesn't say, assume it didn't, and treat the result as a best case.
You can read how I test on the methodology page, or see what the evidence says for a stock you know.
Not financial advice, and not a prediction. It's a record of what I measured and what it said.