Backtesting in not a research tool, ok fine! we have understood it! But, if we can’t optimize our parameters using backtesting, how would we do it? Using a walk forward optimization. The goal will be to combine several backtests in the right way to be able to optimize the parameters without incorporating any bias. In this article, we will explain the walk-forward optimization from scratch, and we will explain the advantages and the limitations of this approach.
1. WHY USING THE WALK FORWARD OPTIMIZATION
In the previous articles “What is backtesting in trading” and “The 6 biggest mistakes for portfolio backtesting”, we have talked about the backtesting process and briefly about the robustness testing. They are excellent to test the solidity of your strategy but not to optimize the parameters of it. To optimize the parameters, one of the most used technic is the walk forward optimization.
Indeed, the major advantage of the walk-forward optimization is that we consider the time series characteristic which avoids incorporating any look-ahead bias like with the robustness testing (if we keep the parameters for live trading)
Join Our Newsletter
Be the first to receive our latest quant trading content (personal notes, discount, new articles).
2. IN SAMPLE VS OUT SAMPLE
When you are doing a backtest, it is frequent to hear about in-sample and out-sample and you need to master these terms. The in-sample will be the sample used to create the strategy and find the best “theorical” parameters. Then, we backtest on the out-sample to backtest on unknown data.
When we do a simple backtest, we take between 70% and 90% for the in sample (also called train set) and between 10% and 30% for the out sample (the test set). However, when we optimize the parameters using a walk-forward optimization, we will combine several in samples and out samples which will lead to many advantages.
Figure: In sample vs out sample into a simple backtest
3. WHAT IS A WALK FORWARD OPTIMIZATION IN TRADING
Fortunately, this time, no fancy name for this method! The definition is in the name… We will walk forward in the asset price to optimize the parameters of the trading strategy. That’s it! (detailed process in the next section)
In other terms, we will use different couples of in sample and out sample to optimize the parameters of the strategy dynamically. The goal is to have the best parameters for the current market condition, at any time.
When you are doing a walk-forward optimization, you have two possibilities: the anchored or the unanchored optimization: one will anchor the in sample set at the beginning of the dataset and the other will make it go forward at the same time as the out sample.
Figure: Anchored vs unanchored walk forward optimization
4. THE STEP BY STEP WALK FORWARD OPTIMIZATION PROCESS
If you are into the Alpha Quant Program, you have already access to a Python code template of the walk-forward optimization implementation, but for the others, I will give you all the steps you need to follow to implement this method.
STEP 1: DEFINE THE OPTIMIZATION PARAMETERS
Even before doing anything, you need to think about the parameters you will use. The first one that we talked about in the previous section is the anchored vs unanchored method: use the one you prefer.
Moreover, we know that we use several in samples and out samples, but we need now to define the length of each of them. Consider that you need to keep a balance between having the more samples that you can but also enough trade in each sample to obtain a reliable optimization.
And last but not least, we need to define an objective function (also called criterion). Indeed, we will perform an optimization: find the best parameters. But the best parameters according to which criterion? It depends on your choice, but I advise you to take into account always the return AND the risk, not only the return. The best known criterion is the Sharpe ratio but I have a preference for the Calmar ratio (annual return divided by the max drawdown).
STEP 2: OPTIMIZE THE PARAMETERS ON THE FIRST IN SAMPLE
This part is relatively easy, you take your first in sample and you test all the possible combinations between your trading strategy parameters, and you keep the best one.
Let’s imagine you are using a SMA + RSI strategy (note that this approach can be used for more complex methods like machine learning without any problem). The possible values for the SMA period are [10, 20, 30, 50, 60] and the period for the potential values for the RSI period are [10,11,12,13,14,15,16]. You need to test all the possibilities (SMA=10 – RSI=10, SMA=11 – RSI=10,…, SMA=60-RSI=16).
Figure: Output testing all the combination on one in-sample (best parameters in green)
SMA period | RSI period | Criterion |
10 | 10 | 0.57 |
10 | 11 | 0.38 |
10 | 12 | 0.61 |
10 | 13 | 0.31 |
... | ... | ... |
60 | 13 | 1.21 |
60 | 14 | 0.97 |
60 | 15 | 0.81 |
60 | 16 | 0.92 |
Each combination will be associated to a criterion value, then depending on the criterion you take the minimum or the maximum value (if you only want to minimize the risk, you will minimize the volatility or the drawdown but if you want to maximize the Sharpe ratio you take the set of parameters associated to the max Sharpe ratio)
STEP 3: OUT SAMPLE BACKTEST WITH THE BEST PARAMETERS
Thanks to the step number 2, we have the best parameters on the in sample. Now, we need to test this set of parameters on the out sample (test set). Indeed, even if it should be the best parameters, nothing say that it will be good enough to put money on it.
When I use a walk forward optimization, most of the time, I do not use only the best parameters of the current in sample. I use an exponential moving average on the previous parameters. The advantage from it is that if you have too much extreme condition in the current in sample, the average will reduce this impact. But, on the other hand, if the trend continues, the best new parameters will be quickly adapted to this new market condition because we use an exponential moving average.
THEN, YOU REPEAT STEP 1, 2 & 3 UNTIL THE END OF YOUR DATASET
5. THE BENEFITS OF THE WALK FORWARD OPTIMIZATION
The walk-forward optimization is better in all the points than a simple backtest but keep in mind that it is still not enough to validate a trading strategy. However, let’s be a bit optimistic and explain what we earn from this method:
- A longer out-sample period: as we split our dataset in several in samples and out samples, we optimize our parameters by pieces. So, the out sample can represent nearly 70% using the walk-forward instead of max 30% with a simple backtest
- Optimization method backtested: of course, we have optimal parameters for the live trading now, but we also have a backtested method to optimize them. Indeed, if you optimized the parameters each 6-months in the backtest, you would do the same for live trading instead of with a simple backtest where the parameters are fixed until the end.
- More variability points: in the article “The 6 biggest mistakes for portfolio backtesting“, we talked about the variability problem. The advantage of the walk forward optimization is that we will reduce this problem as we use several couples of in and out samples. But we do not remove it.
6. THE WALK FORWARD OPTIMIZATION LIMITATIONS
We would not use several methods to validate a trading strategy if the walk-forward does not have any pain points. So, be aware of them is the best way to combine this method with many others to reduce the limitations of the walk forward optimization. Let me detail them:
- Historical path backtest: we backtest several paths especially with the anchored method, but they still come from the historical data ordered as a time series. So, even if it is a clear improvement from a simple backtest, we need to upgrade it using a robustness testing or using generated data.
- Number of trades required: we didn’t abord this problem before but, it is important to highlight it. This method demands a lot of trades to be really helpful. Indeed, if you have a strategy with 50 trades over 10 years, it is clearly not enough. 50 trades per year become a great threshold. But, on the other hand, no need to focus on adding more trades if your strategy does not provide them, just reduce the number of in sample and out sample couples.
It was the first real backtesting method we highlight since the beginning of this series. If you have any question, feel free to ask your question on my public discord forum or directly in private messages on Linkedin.
👇🏼 Join the newsletter to be informed when the next article of the series will be issued
Join Our Newsletter
Be the first to receive our latest quant trading content (personal notes, discount, new articles).
Lucas Inglese
Lucas is a self-taught Quantitative Analyst, holding degrees in Mathematics and Economics from the University of Strasbourg. Embarking on an independent learning journey, he delved deeply into data science and quantitative finance, eventually mastering the disciplines. Lucas has developed numerous bots, sharing his insights and expertise on LinkedIn, where he regularly posts content. His understanding and empathy for beginners in this complex field led him to author several books and create the comprehensive “Alpha Quant Program.” This program offers e-learning videos, monthly projects, and continuous 7-day-a-week support. Through his online courses and publications, Lucas has successfully guided over 67,000 individuals in their pursuit of knowledge in quantitative finance.
Related Posts
Trading Backtest Explained – 3 real life exemples
Understand how to implement the different methods to do a backtest is good. But knowing how to use them to
Why using Monte-Carlo Simulations in Trading?
If you are interested by quant finance, you should have heard about Monte Carlo simulations in trading. However, the concept
Robustness testing – Find reliable trading strategies !
Doing a backtest is good. Be able to trust it is better! That’s where the robustness testing takes place. Indeed,
The 6 Biggest Portfolio Backtesting Mistakes
If you are backtesting a single trading strategy or you are doing a portfolio backtesting, you need to avoid the
The Scientific Backtesting Guide
If you are here, you know that backtesting on the historical data is clearly not enough to obtain the reliability
What is a Backtesting in Trading?
We all have already seen amazing backtesting in trading: 99% are false! This is the first article of a series