Evaluating a Trading System, 01/31/09
Evaluating a Trading System
What before how: The goal of evaluating a trading system is to forecast the future longevity of that system.
With that in mind, here are some simple criteria for a quick first pass evaluation of a trading system. A system which passes all of these criteria is more likely to be successful over the long term than one that does not:
- Review the backtest history and make sure it spans several market regimes and was consistently profitable across them all. With the amount of market data available today, there is no excuse for not having this information.
- Calculate the t score of the system to see how statistically significant the results are.
Generally, try to get 100 observations and a t score greater than 1.6. Higher is better.
Over 3 is very good. Be wary of results less than 2. Make sure a reasonable slippage and commission have
been removed, especially from high frequency systems.
t = sqrt(number of trades) * avg trade / std deviation of avg trade
- Calculate the optimal f$ value. Optimal f$ is important because it tells you the maximum leverage
(in contracts per $) the system can be traded w/o greatly increasing the odds of risk of ruin.
f = (((1 + win loss ratio) * probability winning) - 1) / win loss ratio
f$ = largest losing trade / f
[Note: you can substitute max drawdown for largest losing trade as every trader's ability to withstand drawdowns is suspect.]
A System Forecasting Tool that uses all these statistics to generate a 'bad - better - best' longevity forecast is available here.
Having passed these simple tests, control charts (see below) and monte carlo simulation can be used to further improve the forecast of a trading system's longevity.