How long should you backtest a system?
| I am frequently asked how long one should backtest a trading system. Though there's no easy answer, I will provide you with some guidelines. There are a few factors that you need to consider when determining the period for backtesting your trading system: |
- Trade frequency
How many trades per day does your trading system generate? It’s not important how long you backtest a trading system; it’s important that you receive enough trades to make statistically valid assumptions*: If your trading system generates three trades per day, i.e. 600 trades per year, then a year of testing gives you enough data to make reliable assumptions*. But if your trading system generates only three trades per month, i.e. 36 trades per year, then you should backtest a couple of years to receive reliable data.
- Underlying contract
You must consider the characteristics of the underlying contract. The chart below shows the average daily volume of the e-mini S&P:
It doesn’t make sense to backtest a trading system for the e-mini S&P before 1999, because the contract simply didn't exist! In my opnion it doesn't make sense to backtest an e-mini trading system before 2002 because at that time the market was completely different; less liquidity and different market participants. I believe that a reliable testing period for the e-mini S&P are the years 2002 – 2004.
* What is "statistically valid"?
| Recently I received an article from a Ph.D in statiscs. He explained the correlation between the sample size and the "margin of error" in the table below. The bigger the sample the smaller the margin of error, but usually a sample date of 200 trades should be sufficent. If your trading system generates enough trades, then you should use 500 - 600 trades. | |
