Walk Forward - same parameters, different results?

[puravida]

Hi community,

i am new to WF and don't get it why the results vary everytime i execute a WF with same parameters... in my (obviously wrong) thinking, the results should always be the same, since the optimization is always picking the best parameters (does it?) which should be the same for the optimization-period if nothing was changed. How in this case is it possible to get reliable results to use for the next period? If i click 10 times on WF, i will get 10 different values which is not much of worth...

Can someone please help me solving that knot? Many thanks in advance

[koganam]

Hi community,

i am new to WF and don't get it why the results vary everytime i execute a WF with same parameters... in my (obviously wrong) thinking, the results should always be the same, since the optimization is always picking the best parameters (does it?) which should be the same for the optimization-period if nothing was changed. How in this case is it possible to get reliable results to use for the next period? If i click 10 times on WF, i will get 10 different values which is not much of worth...

Can someone please help me solving that knot? Many thanks in advance

Which optimizer are you using: Default or Genetic?

[NinjaTrader_PatrickH]

Hello puravida,

Thank you for your post.

The Genetic option for 'Optimizer' as suggested by Koganam could be the cause here. If you are using Genetic try using Default instead.

For information on Genetic optimizations please visit the following link: http://www.ninjatrader.com/support/h..._algorithm.htm

Please let me know if you have any questions.

[puravida]

Hello and thanks for your help so far.

I tested it now using default instead of genetic and the parameters that are displayed in the instrument tab (which are the ones to use for next iteration, right?) seem to be the same now.

Still i see the profit factor vary between 1.5 and 1.4 and sometimes it even spikes to 2.0 everytime i do a new WF... why is that? Should the default optimizer not always take *the* best value which would be the same everytime? Why is the profit factor changing then?

[NinjaTrader_PatrickH]

Hello puravida,

Thank you for your response.

The change here could be due to being connected to your data feed provider, as data will be re-downloaded each time you run the test when connected.

[puravida]

Thanks for your help.

That not seems to be the reason, i disconnected from Kinetick and hitting the walk forward several times with the sample ma crossover strategy and the resulting profit factor still varies by several points everytime. I did not changed any parameters.

[NinjaTrader_PatrickH]

Hello puravida,

Thank you for your response.

I am not seeing the same behavior on my end. Can you attach a screenshot of the parameters used in your WFO for the SampleMaCrossover?

[cachevery]

I got the same problem with the Replay when i`m testing a strategy.With the same settings and time period i always get different outcome.Why is that?

[puravida]

Here are two samples of a wf with same settings and different results. I tested this with other strategies and instruments and it occurs also there. As you can see, the profit factor is different and therefore both walks where not 100% the same... the question is why?

[cachevery]

Here are two samples of a wf with same settings and different results. I tested this with other strategies and instruments and it occurs also there. As you can see, the profit factor is different and therefore both walks where not 100% the same... the question is why?

Why doesn`t support give the 'why'?????????????????????????????????????

[NinjaTrader_Bertrand]

Why doesn`t support give the 'why'?????????????????????????????????????

cachevery, we're looking into this report and will update the thread shortly.

I also deleted your last post in this thread on purpose as the language contained was unacceptable for our forums, please consider this when reposting in the future. Thanks.

[cachevery]

cachevery, we're looking into this report and will update the thread shortly.

I also deleted your last post in this thread on purpose as the language contained was unacceptable for our forums, please consider this when reposting in the future. Thanks.

look, you are looking at the 2 to 5 years reports,where the problems hasn`t been solved as of yet.Can you tell me why i`ve just bought the Replay data worth of $900 from your site backtestdata.com which turnes out to be worthless?Moreso,i didn`t even recieved it yet,even though i bought it 3 days ago.WHERE IS THE DATA I BOUGHT?

My intention was to open an account with you,now why should i?I could put up to $1M,for e.g.,and you tell me that the platform isn`t workable.What kind your playing here,i don`t get it.

[cachevery]

i`d probably ask for refund for that 230 GB of garbage!

Guys,don`t deal with the backtestdata.com.no replies ,no support,no delivery,nothing, NADA!!!Whart`s wrong with the industry now???Where is the integrity????

[NinjaTrader_Bertrand]

cachevery, though of course I can understand the frustrating aspect of this business, I ask you again here please to watch your language and manners used on this public forum, we want to keep this a highly productive environment for all our clients, present and future.

We're not directly affiliated with the company you purchased the data with, so please discuss your purchase with them.

To be clear, this appears to be Market Replay data > which is not used for your Strategy Analyzer backtesting or Walk Forward testing. That is strictly done from historical data usually downloaded from a datafeed connection like Kinetick or brokerage server like Continuum / Rithmic.

As for the items raised here - I would suggest to ensure testing in a disconnected state and see if you can still reproduce the differences, as if you would be connected > the current day would be reloaded every time potentially introducing a variable here in the tests.

Thanks,

[cachevery]

cachevery, though of course I can understand the frustrating aspect of this business, I ask you again here please to watch your language and manners used on this public forum, we want to keep this a highly productive environment for all our clients, present and future.

We're not directly affiliated with the company you purchased the data with, so please discuss your purchase with them.

To be clear, this appears to be Market Replay data > which is not used for your Strategy Analyzer backtesting or Walk Forward testing. That is strictly done from historical data usually downloaded from a datafeed connection like Kinetick or brokerage server like Continuum / Rithmic.

As for the items raised here - I would suggest to ensure testing in a disconnected state and see if you can still reproduce the differences, as if you would be connected > the current day would be reloaded every time potentially introducing a variable here in the tests.

Thanks,

Sure i`m frustrated,having spent years and 10th thousand of $ on a supposedly ''world leading''platform just to find out it doesn`t work.

How am i supposed to do the test on the Market Replay in a disconnected state.Do you realise what your saying?It`s getting ridiculuos here...