What "work in progress" means on a research result

Some pages on StrategyLens are labelled work in progress, and each result carries a status: held up, did not hold up, or inconclusive. This note explains what those labels mean and why they are there.

The short version: research is never finished, and a result is only as strong as the tests it has passed. The labels say how far each one has got.

Why results are labelled at all

Backtest figures look precise. A percentage to one decimal place suggests certainty that the evidence often does not support. Labels give a reader the context a number alone cannot: whether the result has been checked on data it was not built on, and whether it survived.

Held up

A result that held up was checked on data that played no part in choosing the strategy's settings, such as later years held back from the start, and the result remained broadly the same. That is evidence, not proof. Markets change, and a result that held up once can still fail later.

Did not hold up

A result that did not hold up looked good on the data it was built on and weakened or reversed on the unseen data. This is the most common outcome in strategy research, and we publish it rather than hiding it. A result that did not hold up is useful: it tells you an idea that looked promising probably fitted the past rather than anything lasting.

Inconclusive

Inconclusive means a required test could not be run or did not give a clear answer, often because there was too little data, too few trades, or a missing input. It is not a negative result and not a positive one. It means the evidence does not support a claim either way.

Work in progress

A page marked work in progress is one whose content is still being built or checked, such as a new study page whose comparisons or explanations are still being added. The figures on it come from the published cards like every other page; the label says the page is not yet complete, not that its numbers are provisional guesses.

Why the labels change

Cards are regenerated as new data arrives and as tests are rerun. A result can move from inconclusive to held up or did not hold up as more evidence accumulates, and figures on every page change with each refresh. Each note on this blog dates its figures for that reason.

How to read a labelled result

Read the label before the number. A striking figure marked did not hold up is a cautionary tale, not an opportunity. A modest figure marked held up is more informative than a large one that is inconclusive. And none of the labels is a recommendation: they describe how a past test turned out, not what anyone should do.

What we have found so far

Most of what we test does not beat holding. Across the 38 published tickers, 2,565 of 21,014 backtests beat buying and holding the same ticker over the same dates (12.2%). We publish that alongside every result, because a research site that showed only its successes would look the same as one that had no failures to show.

Figures are from the 38 StrategyLens published ticker cards, updated 29 October 2026. They are hypothetical backtests with no trading costs, before tax, and they will change as the cards are refreshed.

Why we publish results that did not work

When many strategies are tested on many tickers, some will look good by chance. If only those are published, readers see a list of apparent successes and no way to judge how many failures sat behind them. Publishing the failures, and the share of tests that beat holding, lets a reader see the whole search and judge the successes in context.

It also keeps the research honest with itself. A result that did not hold up is recorded, so it cannot quietly be retried with slightly different settings until it passes.

How to use the labels

Treat held up as a reason to look more closely, not a reason to act. Treat did not hold up as a likely false lead. Treat inconclusive as a question still open. And treat work in progress as a page that may gain sections or explanations over time, with the figures on it dated so you can see how current they are.

If a label or figure on a page looks wrong, or the date beside it looks stale, that is worth telling us. A research site gets better by being checked, and a correction that is published openly is more useful to readers than a figure quietly changed. Every figure we print comes from a dated card for exactly that reason: so it can be checked.

This is research, not financial advice. Past results, real or simulated, do not tell you what will happen next.

What does held up mean on StrategyLens?

The result was checked on data not used to choose the strategy's settings and stayed broadly the same. It is evidence, not proof.

What does inconclusive mean?

A required test could not be run or gave no clear answer, often from too little data. It is neither a positive nor a negative result.

Are work in progress figures reliable?

The figures come from the published cards like every other page. The label means the page is still being completed, not that the numbers are guesses.