Disclaimer: The opinions expressed here are for general informational purposes only and are not intended to provide specific advice or recommendations for any specific security or investment product. You should never invest money that you cannot afford to lose. Before trading using complex financial products, please ensure to understand the risks involved. Past performance is no guarantee of future results.
Table of Contents:
For the investors reading this article, you can skip ahead to “What is Overfitting?”
By this point you may have run a few backtests, on a simple script or using one of our educational scripts. You may have also achieved a very high Profit/Loss % and are wanting to run the strategy with real assets. Before this happens, I want to caution you and help set expectations.
What is overfitting?
If an analysis corresponds too closely with a limited set of data we call this overfitting. Markets inherently have noise, that is to say small and frequent idiosyncrasies in the price data. When modelling a strategy, we want to avoid optimizing for a specific period because there is a chance this won’t predict the future.
It’d be like tuning a car specifically for one racetrack, while expecting it to perform well anywhere.
To illustrate:
In the example above, 100 versions of the default script were backtested for 5 months. One of which yielded over 93% profit. If an imaginary user went live for the next 5 months using these ‘best’ parameters, they might be disappointed. Nobody ever complained about more money, but 12.7% is notably less when compared to the results of the previous 5 months. The point is, the past doesn’t necessarily predict the future.
How do I avoid overfitting?
1. Test for random periods
Running a backtest during bull and bear markets is a good first step. However, the reality is you are probably not starting a strategy at the beginning of a new market cycle, but in the middle of one. Sideways markets can be especially frustrating as a strategy tries to decide bear and bull trends. How would you react if your strategy lost money for the first 3 months?
2. Be granular in your testing
Due to market volatility, it’s possible that one good month (or one good trade) is enough to offset a large loss. Profitable trades and max drawdown MDD offer a window to overall performance, but does not tell a detailed story. To help, we’ve made our own Power BI Template that organizes trade data for deeper analysis.
3. Run a Paper Trade (forward test)
This one’s more of a reminder.
I want to end by saying that these methods are not comprehensive. Each strategy is different, making it hard to offer blanket advice. The point of this article was to raise awareness.
Despite automated trading, markets are still largely driven by human participation. Those who pay close attention to the markets will see opportunities for profit using …
Welcome back, data science fans!
Continuing from the second article, where I discussed the clustering of buy signals, we’re now ready to get into the meat …
And here we have it: the last installment of the data science article series! In our previous article, we created some correlated clusters and came …
Our previous blog post, The Rise of the Independent Trader, talked about a plan to level the playing field between Wall Street and Main Street. …
