Over-optimization (overfitting) is the biggest threat to AOS strategies and also a topic of endless discussions that have no simple and general solution. In today’s article, I would like to share with you 5 tips, how I myself try to minimize the danger of overfitting.
1. Build your systems using 40-50% of all the historical data available to you. Leave the rest for robustness tests.
When I create my own strategies, I follow the rule to save as much data as possible. I try not to waste more data than is necessary to build a system, but I do leave some of it for robustness testing. For this reason, to build a system I use only a part of all the historical data that is available. I usually have 10-12 years of historical data and for the building I use only 3-4 years. This period is different every time I build a new strategy (sometimes I use 2-5 years, sometimes 7-10). The reason is simple: the less data you use to create the strategy itself, the lower the risk of overfitting. The rest of the data I leave for my robustness tests. If the strategy seems feasible in the small and limited sample of data, I start with the robustness tests using all the data. By using this method, I can be sure that the strategy is tested with the data that was not used for the construction of this strategy; this makes the test more objective and I reduce the risk of overfitting. Of course, this approach gives me a lot of “junk”, that is, a lot of potentially interesting strategies that fail robustness tests. But this is part of developing high-quality ATS: just by being patient and disciplined, you gain an advantage over other traders, because most of them do not have these qualities and tend to look for shortcuts that lead to finding a strategy. faster, however, the quality is questionable. Finding a solid, high-quality strategy takes time and patience. And the less data we use to build the strategy, the lower the risk of overfitting.
2. A good strategy should work in other markets in the same category
The ideal strategy is the one that also works in other different and inconsistent markets. Finding such a “gem” is really extremely demanding and I am not far from the truth when I say that even if you search very hard, it is a success to find 1 strategy per year that meets this criteria. Since we need to diversify our ATS portfolio, for practical reasons we cannot move at such a slow pace. And sometimes it is better to “lose the rules a bit” and accept strategies that pass all the tests of robustness in other markets of the same category. Finding these strategies is much more realistic, and the frequency of finding a strategy like this is at an acceptable rate. For example, I test a strategy built for the TF market so that it works also on ES, YM, EMD and preferably also on NQ, ie it works roughly on all markets in an index category. Like a strategy built for the TY market, I want it to work well in the US and FV markets, meaning it works well in all markets in the bond category. Before moving on to the next point, let me make an important note: sometimes it happens that during the verification of the strategy in other markets, you will find a situation where the strategy works even better in another market than the one for which it was created. In that case, you will be tempted to trade the strategy in this market. don’t do it. It’s just another form of optimization and these little optimizations add up and lead to a big overfitting. Simply trade the strategy in the market it was created for, or trade all markets in the category. Learn to minimize attempts to choose the best of the range of acceptable results, as this already leads to overfitting.
3. Never change a strategy after you have finished the robustness tests and chosen the parameters for live trading
When the strategy passes all the robustness tests and you choose the correct parameters for live trading, do not modify the strategy anymore! parameters This is a high degree of overfitting after which the strategy will almost certainly fail. Yes, you will have good backtesting equity, but the live trading results will be a huge disappointment. You should approach the entire optimization process sensitively and at all times make as few changes as possible. Changing parameters repeatedly to find the best ones leads to significant overfitting. Accept the fact that once you choose the parameters for your live trading, they are final and should not be changed. If you have more than one set of good performance parameters, you should prefer the average to the best. The best results tend to be overfitting. I would start trading live with all available parameter sets or I would choose the average.
4. Don’t skip the paper trading period
One of the simplest and most effective ways to eliminate a potentially over-optimized strategy is the paper trading period. When you complete the strategy, run it for 3 months on the simulated account and see if the strategy follows your expectations (some of the traders I know even go for 6 months). You won’t gain any advantage by skipping this period or trying to find shortcuts. The only truly invisible data (the actual out-of-sample data) is the one that does not yet exist. And that is exactly the reason why no final verification test can compete with the trading period on paper when we test the strategy on live data. Being impatient gets you nowhere. The best way to keep going is patience and discipline.
5. Do not use time frames that are too low (to reduce “distortion”)
Based on my experience, I can say that the smaller the time frame you use to create a strategy, the more distortions there will be on the charts. How is it related to overfitting? Very simple. Good equity on a short time frame is more of an overfit than an advantage. Higher time frames contain less distortion, therefore there is a better assumption that there is real upside behind the equity we see. Personally, I use various time frames, but never less than 10 minutes. For day trading strategies I usually use 15, 30, 45, but even 60 minute time frames. For my swing strategies, I go even higher: to 80, 120, 140, 180, 200, or even up to 240 minutes. I have multiple strategies over multiple time periods, which I consider to be another form of diversification. One final note on time frames: check your final strategy across multiple time frames. You might find a time frame that gives you better results than the original, don’t trade it live, it’s another form of over-optimization. Use the time frame for which you have created the strategy and, in the worst case, choose the one with average results. Everything I’ve outlined above seems like small things, but when you put them together, it’s a big step toward reducing overfitting.
