Strategy Analyzer - Freezing during Optimization

[NinjaTrader_Jim]

Hello Karsten,

Quality Assurance has reviewed and have not been able to reproduce. I went back and tested on clean installations of the current build and our internal builds, and I was not able to reproduce longer delays from disconnections.

Some thing we could try in an effort to narrow down where longer backtesting times are coming with the Strategy Analyzer using SampleMACrossover would be to set up a clean environment so you can also perform the same "clean" test as I have. I expect that you would not see delays when connected or disconnected here as I am not seeing them in the clean environment.

If you flip back to the existing environment, and can then reproduce delays when disconnected, This would point to something on your platform that is contributing that we could narrow down. If that is the case, please reach out to me over email at scriptingsupport [at] ninjatrader [dot] com with the text "Attn Jim 2162858" in the subject line. I will then want to ask for files or get connected to see if I can get my platform to reproduce the same. This can hopefully eliminate some of the randomness and get us something that is testable that we can look into.

Clean Environment Test:

Creating a clean environment can be done by following the steps below:

1. Close NinjaTrader 8, and rename the "NinjaTrader 8" folder in My Documents to something like: "NinjaTrader 8 OLD" Do not delete this folder.
2. Uninstall NinjaTrader from the Windows Control Panel
3. Delete the C:\Program Files (x86)\NinjaTrader 8 folder
4. Reinstall using the installer from http://ninjatrader.com/PlatformDirect
5. Backtest SampleMACrossover from the first of the year with tick data, and then test again while disconnected and again after reconnecting

If you ever need to switch back to your original platform, you may do so by closing NinjaTrader and swapping the platform folder names.

For example, Close NinjaTrader and rename the new "NinjaTrader 8" folder to "NinjaTrader 8 NEW" and the "NinjaTrader 8 OLD" folder to "NinjaTrader 8." Then restart the platform. Simply put: NinjaTrader 8 will always load the "NinjaTrader 8" folder in My Documents.

We look forward to assisting.

[KarstenKafl]

Hi Jim

Thanks for your reply.
Unfortunatly now im confused about what you write.

You wrote to me:
"I can say that my tests when connected are generally the same with your strategy or the SampleMACrossover strategy, about 30 seconds testing from the first of the year.

However, I can also say that when I am disconnected that times are much faster. I'm inquiring with our Quality Assurance team since I would expect similar speeds connected or disconnected when the data is already downloaded."

Then you wrote:
"Quality Assurance has reviewed and have not been able to reproduce. I went back and tested on clean installations of the current build and our internal builds, and I was not able to reproduce longer delays from disconnections."

That last sentence i believe is the complete opposite of what you experienced - which you descibed above?

So first of all - you wrote you had much faster times when disconnected.

Does this mean that you have only fast time all the time now in new version of Ninja trader? - so 1 sec or so in same setup running 2020 year?

Or this means that now - you have "Only" slow times = 30 sec. with new ninja trader clean builds?

If the last is the case - then im not sure if i want to update/clean my ninja trader - sinse in the current version at lease i had "no load time" occuring some times.


Best regards
Karsten

[NinjaTrader_Jim]

Hello Karsten,

I was able to see longer loading times in post 14, but we could not reproduce the symptoms in a consistent way that can be analyzed. When I tested clean installations of the current version and our internal build, I am getting backtest times that are as fast when I am disconnected as I am when connected.

If you set up a clean environment of the latest version, I am confident you will see the same fast backtest times I reported in post #16. This will also help to narrow down other factors that may be adding randomness. If you can reproduce symptoms in the clean environment of the current build, there would be less factors involved and we would be better able to reproduce the symptoms you are reporting.

After testing the clean environment, and you do not see symptoms, but you do see the symptoms when switching back to the existing environment, this would mean that there is something else involved with the existing environment that is contributing. We can take further steps to narrow down from there by adding components to the clean environment and testing to see if the long backtest symptoms come back. If they do, we should then work from there as other factors and randomness would be eliminated.

I would then want to work with you more closely over email so we can request files from this clean environment that was used for testing, and so we can take the same steps to hit the same symptoms on our end, without the randomness.

Pleas let me know if you have any questions.

[KarstenKafl]

Hi Jim

Im not completely sure if you are maybe not reading my posts correcty? and i feel like i need to repeat myself a lot.

Please correct me if im wrong - but you wrote "However, I can also say that when I am disconnected that times are much faster. " THIS could be what i experience.
I have almost random times - but sometimes (and even sometimes when disconnedted) i had SHORTER load times.
But you wrote to me that you have experienced shorter load times when disconenected.

So my question is - now - suddenly - do you get loading times much faster than the 30 sec. you wrote before?

So is that the case now all the time - when you load the same periode etc? - or are we now "forced" to use the "slow" version all the time??? which is 30 sec. load time on your computer - no matter if it is loaded already or not - or whatever the reason is that it load in 1 sec. instead of 30- 75 sec. ?

If it is the case that now - all the time - you load in 1 sec. - or so before starting the actual calcualtion - then im happy to instlal the latest version. '
If the case is "the opposite" that now you need to allways load the 30 - 75 sec. - when doing the script that i supplied - then i have no interest in upgradgin ?

Thanks
Karsten

[NinjaTrader_Jim]

Hello Karsten,

I was able to get slow backtests in post 14. This was not consistently reproduceable, and could not be analyzed. I gave it another try with a clean installation of the current build and our internal build and could not create the slow backtesting times.

I am no longer able to reproduce slow backtest times and the backtest times are fast.

Please upgrade and test backtesting. If symptoms persist, please set up the clean environment and test again there.

[KarstenKafl]

Hi Jim

Now i run into similar experiences.

It was never fixed the last time - but i moved away from testing with ticks -and instead now i use just minutes instead and so far i have been able to keep time down for loading the strategies.

But now again i get some weird things going on.

I load strategy into strategy analyzer - it run on simple 15 minutes bars on YM 12 22 - from 01-01.2006 to 30.09.2022.

When i run this as backtest in the Strategy analyzer - the "simple" strategy takes 10-15 minutes to load.
When i use the exact same settings and just run strategy in a 15 minute chart (not using strategy analyzer) - but with same data loaded - from 01.01.2006 - then the strategy is done after 20 seconds.

I have 4 multiple instruments loading into the script - loading day vaues only, i use several indicators that are "home made" -
I cleaned up the "environment" / reinstalled all - as you explaine in post News and Announcements

If i run the Sample MA crossover on same instrument YM - in strategy analyzer - on same time frames etc - it finish after 1 sec or so - even though it runs thorugh 17374 trades.
The strategy im testing is only 1500 trades or so.

I imagine you will comment on my "home made" indicators - but my concern is of cause why/how can the exact same tests - take 20 seconds in chart - but take 15 minuts when running in strategy analyzer? It ends up with same result - sinse it is the same test it is running.

[NinjaTrader_ChrisL]

Hi Karsten, thanks for posting. Your description makes me assume there are heavy performance hits coming from within the custom strategy, the indicators being used in the strategy, and/or the strategy analyzer setup. If you can run a simple regular backtest in the strategy analyzer on the SampleMaCrossover quickly, your strategy must be scrutinized further. If you would like to post an example of your strategy that I can run on my machine (no external dependencies, no custom indicators) I will be happy to have a quick look at it.

Kind regards,
-ChrisL