8.0.0.4 not caching if not Break at EOD?

[dalebru]

Here is a link to a 5 minute Jing screencast: http://screencast.com/t/padbGAjTN3s

Summary: I really like the new Break at EOD feature, where I can have large Volume or Tick or Renko bars that do NOT break at the EOD. But they are tick-based and are necessarily slow the first time they are created if "Days to load" is a year or more, even when all of the ticks have already been downloaded to my disk. (My problem seems to be the same whether or not I am Connected, provided the ticks for the time period have been downloaded.)

The second time I reference that same bar definition, NT8 loads them very quickly, just as NT7 did, and I thought that was because of caching bars into ..Documents\NinjaTrader 8\db\cache\CME US Index Futures ETH.US Eastern Standard Time\TICK\ES 12-15. I could be wrong about that.

But IFF I uncheck Break at EOD, NT8 doesn't seem to be any faster on subsequent loads than on the first load, as if caching is not being done. So when I load a workspace with a few of such charts, I must wait many minutes before all of those charts in the workspace are loaded.

[NinjaTrader_Matthew]

That's a great observation and question. I'll need to get with development to analyze and fully understand what you're reporting.

I can confirm that there is caching occurring in the both BreakEOD and non-BreakEOD charts.

I also know that the non-BreakEOD is saved in a single cache file, rather than broken into multiple sessions like you see with the BreakEOD files (e.g., for a years of data, using rough numbers, we're reading from one 20MB~ file rather than several 50KB~ files)

As far as why it takes so much longer to read from one cache file than the other is what I need to understand better and will get back to you if this is a confirmed bug or expected results.

[NinjaTrader_Matthew]

I was able to get confirmation from development on what is happening.

For tick intervals using Non-BreakEOD, these bars series need to be build from 1 tick series, because they depend on when series started. There is little performance value in caching the Non-BreakEOD bars since it's essentially the same as reading from the 1-tick file. As a result, we removed the caching since this was not improving anything in our testing. Only BreakEOD bars will be cached.

[dalebru]

I was able to get confirmation from development on what is happening.

For tick intervals using Non-BreakEOD, these bars series need to be build from 1 tick series, because they depend on when series started. There is little performance value in caching the Non-BreakEOD bars since it's essentially the same as reading from the 1-tick file. As a result, we removed the caching since this was not improving anything in our testing. Only BreakEOD bars will be cached.

I think your decision is dead wrong, and certainly I will never use Break at EOD again. For me, Break at EOD is only useful on very large bars, and without caching NT 8 goes from slow to unbearable.
I would think it would be easy enough to "kill" the cache when an earlier-than-cached Bars Request comes in.
But it's your product and of course you have a right to make as many mistakes as you like.

[NinjaTrader_Matthew]

But the cache is still a 1-tick series, it would not do anything for performance. For this type of series, we cannot break the cache up into sessions like we see with regular bars, which is why you have a performance improvement on those bar series.

We had implemented caching and as you noted in your observation, it was not any faster as the first load. As a result of your post, we reviewed these scenarios and concluded that there is no difference between caching these bar types and reading from the tick file. The cache will be removed in B6 and there will not be any difference in performance as a result of us removing this.

[dalebru]

Yes, no caching is better than the 1-tick cache.