Strategy Validation: Backtesting and Forward Testing
This lesson sets out the requirements for testable rules, the procedure for backtesting, the problem of curve fitting and the role of forward testing on a demo account.
Risk note: educational content only. This article is not financial advice, investment advice, or a recommendation to buy or sell any instrument. Trading leveraged products can result in losses greater than expected, and each reader should understand the risks before trading.
A trading strategy is distinguished from an opinion about the market by evidence that its rules have produced a positive result across a sufficient number of trades. Testing is the process by which that evidence is obtained. It can also be performed incorrectly in ways that produce convincing but false results, so the methodological errors are as important as the method itself. This lesson describes the precision required of the rules, the conduct of a backtest over historical data, the manner in which excessive parameters fit a strategy to the past, and the forward test that verifies the strategy can be executed in real time.
The requirement for unambiguous rules
Before any test is conducted the strategy must be written as a set of rules that a third party could apply without consulting the author. The entry condition, the stop-loss placement, the exit method and the position size must each be defined precisely. A rule expressed in terms such as strong momentum or a clear trend is not testable, because two people applying it to the same chart would reach different decisions, and a strategy that cannot be tested cannot be validated.
The exercise of writing the rules to this standard frequently reveals that an idea which appeared coherent is in fact a collection of impressions. This is a useful result. It identifies at low cost an approach that would otherwise have been discovered to be unworkable only after capital had been committed.
Conduct of a backtest
A backtest applies the written rules to historical price data, trade by trade, and records the outcome of every trade that the rules would have generated. The record must include all trades, including those the trader would prefer to exclude with the benefit of hindsight; omitting trades on the grounds that they would obviously not have been taken invalidates the sample. A minimum of one hundred trades is required for the result to carry any statistical weight.
The test period should include trending, ranging and highly volatile conditions, since a strategy tested only during the conditions that favour it will overstate its performance. Spread, commission and, where positions are held overnight, financing must be deducted from every trade. A strategy with a small gross edge frequently has no edge once realistic costs are applied.
- A minimum of one hundred trades.
- Trending, ranging and volatile periods included.
- Spread, commission and financing deducted from every trade.
- No trades omitted on the basis of hindsight.
Curve fitting and parameter selection
Curve fitting is the process, usually unintentional, of adding conditions to a strategy until it performs well on the specific historical data used to test it. Given enough parameters, any set of past prices can be described with apparent precision. A rule that produces good results only with a 37-period moving average and only on a particular day of the week has been fitted to the history rather than derived from a persistent market behaviour, and it should not be expected to perform in the future.
Two safeguards limit this risk. The first is to keep the number of parameters small, since each additional parameter increases the number of ways the strategy can be adjusted to the past. The second is to require a rational explanation for each rule. A rule that describes a plausible market mechanism, such as the tendency of a trend to continue after a shallow retracement, is more likely to persist than a rule that merely coincides with a favourable period.
Forward testing on a demo account
A backtest establishes whether the rules would have been profitable. It does not establish whether the trader can execute them in real time, under the conditions of hesitation, distraction and temptation that accompany a live market. Forward testing addresses this by applying the strategy to the market as it unfolds, on a demo account, over a defined period.
The Onys Kapital demo account provides USD 10,000 in virtual funds on the live MetaTrader 5 platform, so that order entry, execution and platform behaviour are identical to those of a funded account. A forward test should run for at least one month. It is common for a strategy with a satisfactory backtest to prove impractical in forward testing, either because the rules are difficult to follow in real time or because the trader is unwilling to accept the sequence of losses the strategy generates.
Interpreting the results
The output of a test is a set of statistics: win rate, average gain, average loss, expectancy, maximum drawdown and the longest sequence of consecutive losses. These should be read together. A high win rate accompanied by a large average loss may be unprofitable, and a modest win rate accompanied by a favourable ratio of average gain to average loss may be robust. The maximum drawdown and losing sequence indicate the capital and the tolerance the strategy will require.
If the results are unsatisfactory, one element of the strategy should be altered and the test repeated. Altering several elements simultaneously makes it impossible to attribute any improvement to a particular change. If the results are satisfactory in both the backtest and the forward test, the strategy may progress to live trading at the minimum position size, as described in the final lesson of this course.