WalkForward: decreasing results. Why?

[banteka]

Hello:

I have some questions regarding WalkForward Optimization.
I have attached an image with the results of my WFO analysis in order to illustrate what I want to explain.




Ok. These are the results of my WFO. As you see, the results are ordered by date. The analysis goes from June, 01 2013 to August, 01 2014.
The Settings are as follow:




Well. I have a question regarding the Total Net Profit column. Why are the results decreasing as the WalkForward test advances in time?
It is supposed that the OUT OF SAMPLE period is always the same: 30 days.

NOTE: The Total Of trades column also decreases...

This behaviour always happens, with any instrument, any period and any settings.

Thank you very much.

Banteka

[NinjaTrader_Bertrand]

Banteka, thanks for the post - there would be nothing per se in NT that I would expect to dictate that. Would you see this as well if for example comparing to a test with our SampleMACrossOver script?

Just thinking about the general setup here - you're using a fixed 12 range chart, which can produce a varying amount of bars to work on depending on the given range of your market. That is combined with 'static' stops / targets based on ticks - perhaps the opportunities for your approach on the range chart are decreasing?

[banteka]

Thank you Bertrand for your quick answer.

I am changing the IN SAMPLE and OUT OF SAMPLE periods in the WalkForward settings and I have noticed one interesting thing:
When the OUT OF SAMPLE period is less than 28 days, the decreasing values do not appear. For example, I have tried with an IN SAMPLE of 30 days and an OUT OF SAMPLE value of 10 days and these are the results:




However, the number of trades is excesive for ONLY 10 DAYS!!!. I do not understand these results.

Could someone test his own strategy with RANGE BARS and checking the results?

[NinjaTrader_Bertrand]

banteka, thanks for the update - would this be your own script or an addon done by a third party? If you check your Control Center log do you see any errors when you run with an increased test period?

If you for example let the SampleMA script, that is preinstalled with NT, run on the same dataset and optimization / test ranges, would outcome would be seen?

[banteka]

banteka, thanks for the update - would this be your own script or an addon done by a third party? If you check your Control Center log do you see any errors when you run with an increased test period?

If you for example let the SampleMA script, that is preinstalled with NT, run on the same dataset and optimization / test ranges, would outcome would be seen?

It's my own script. There are no errors in the Log pane.
I have tested the SampleMACrossOverscript with the same databank and, although the decreasing behavior does not appear, the profits are excesive too...
I'm afraid my databank is corrupted. I am going to delete all my databank and download it again. I'll keep you informed. Perhaps there are duplicated data. Could this happens (duplicated data in the databank) and NinjaTrader does not complains?.

Regards.

[NinjaTrader_Bertrand]

Ok thanks for letting me know - we could also give it a test run here for comparison if you like. Duplicated data should not be an issue, what you can relatively easy do is running from a fresh set of cache for your data. Exit NT completely and go to Documents\NinjaTrader 7\db\cache and delete all subfolders seen here. Then restart and redo your test in disconnected state > NT will rebuild the cache and only use your local stored data.

[banteka]

Hello again Bertrand I have followed your instructions. I have deleted all data from my NT7 environment as you suggested,
Then, I have downloaded again data from my DEMO CONTINUUM ACCOUNT using the Historical Data Manager.
But when I have tested again my script, the same results have appeared: very large profits but decreasing in time.

Could I send you my script in order for you to test it with your own data?
If so, please tell me how.

NOTE: I cannot run a WalkForward, Backtest or Opimization if my NT7 is not connected to the MARKET REPLAY CONNECTION. If I connect to CONTINUUM, only void results appear in the pane (only zero values). The same occurs if no connection to any account. But connected to MARKET REPLAY it works properly. Is this normal?

See this picture:



Regards

[NinjaTrader_Brandon]

Hello banteka,

Thanks for your reply.

Please send your script to platformsupport[at]ninjatrader[dot]com
Include a link to this forum topic in the message.

[banteka]

Hello banteka,

Thanks for your reply.

Please send your script to platformsupport[at]ninjatrader[dot]com
Include a link to this forum topic in the message.

Thank you Brandon. I've just sent it.

Regards

[banteka]

Ok. Maybe I have found the solution. I have added MULTITIMEFRAME to my strategy. The main TimeFrame is RANGE 12 and I have added a second TF by adding this line to the Initialize() function as ChelseaB suggested here:

Add(Instrument.FullName,PeriodType.Tick, 1, MarketDataType.Last);

And in OnBarUpdate() I split by checking BarsInProgress.

This way, when BarsInProgress==0 (Bar by Bar) I do the main timeframe stuff (p.e. checking FirstTickOfBar())

And when BarsInProgress==1 (Tick by Tick) I do other stuff (p.e. reading indicators, etc).

Now, the WalkForward process is longer but the results are more suitable.

Nt Team: Am I doing right?

[NinjaTrader_Jesse]

Hello banteka,

I see nothing wrong with the way that you have done this as you have correctly added the tick series.

I believe that the conclusion you have come to seems plausible, the walk forward would take additional time as there is now additional steps and if you are getting more correct results I would say you are on the right track. You are also correctly "splitting" the data series with your BarsInProgress.

If after making the change the results are still not expected please let us know.

I look forward to being of further assistance.

[banteka]

Ok. Problem was solved at all. Implementing MultiTimeFrame was the solution.
Now, results are more reliable...Check the following picture:



And this is the WALKFORWARD result:



Anyway, thank you very much to everybody. This is really a very good forum, a very good product and, above all, a very good TEAM.

Now, I'll test my strategy in a DEMO ACCOUNT.

Banteka

[NinjaTrader_Jesse]

Hello banteka,

Thank you for updating this thread with your outcome we do appreciate having these type of answers on the forum. I am glad this has resolved the results you were getting.

Please let me know if I may be of additional assistance.