Multi Series Indicators not working in Tick Replay

[Ricam]

Applies to indicators where a secondary data series has been created by the AddDataSeries method and the secondary series is for an instrument other than than the instrument on the chart. Applies to charts with tick based time frame, have not tested on other time frames yet.

This applies ONLY when the tick replay box is checked. When tick replay is not checked, performance is exactly as in previous versions of NT8 beta.

On all the beta versions of NT8, the bar data of both series would be in correct chronological order relative to each other.

Now, all of the historical ticks for the added series (bip=1) load while the primary series is still on CurrentBars[0] = -1.

[NinjaTrader_PatrickH]

Hello Ricam,

Thank you for your report.

I will bring this to development's attention.

[Ricam]

Were you able to duplicate the problem I mentioned? Thanks.

[Ricam]

Script to demonstrate multi series sequencing error in Tick Replay

When this script runs, the sequence of bar numbers triggering Update events, and the bar times, print to the Output Window.

If the tick replay box is checked on the Data Series page, the Primary Series remains stuck at Bars[0] = -2 as the Secondary Series loads. If the box is unchecked, the sequencing seems to be normal.

Doesn't seem to matter whether the secondary instrument is different than the primary instrument.

Thanks again.

[NinjaTrader_PatrickH]

Thank you for your patience, Ricam.

I checked with development and the behavior we are seeing with your script is accurate. Each bars series will update up to the current bar, and then the next bar series will do so when using Tick Replay or not.

With Tick Replay the secondary bar series appears to update first and then the primary bar series. You will need to ensure you are developing the systems specifically for Tick Replay or not.
Please refer to our help guide on developing for Tick Replay: http://ninjatrader.com/support/helpG...ick_replay.htm

[Ricam]

New tick replay implementation degrades Ninjatrader functionality

Patrick H said:

I checked with development and the behavior we are seeing with your script is accurate. Each bars series will update up to the current bar, and then the next bar series will do so when using Tick Replay or not.

The underlined portion is not correct. For non-tick replay, the primary series and all of the added series are synchronized. All of the ticks for all of the series play back in the correct chronological order. NT7 always worked that way. NT8 still works that way, except in Tick Replay.

Handling of non primary series in Tick Replay just changed from the way it worked in all the public betas up to RC2. The new changes seem to make any non primary series unusable in Tick Replay, greatly reducing Ninjatrader's functionality with no apparent benefit.

With Tick Replay the secondary bar series appears to update first and then the primary bar series.

"Appears to" update first?? It DOES update first.

You will need to ensure you are developing the systems specifically for Tick Replay or not.
Please refer to our help guide on developing for Tick Replay: http://ninjatrader.com/support/helpG...ick_replay.htm

The Tick Replay page in the Help Guide does not mention anything about non primary dataseries that have been created with the Add method.

The way Tick Replay works now, I cannot see how to use historical data from a secondary series with a Tick Replay series. If this is possible, perhaps you could show how to do that with an example.

The previous Tick Replay behavior was that all of the series were synchronized so that the historical ticks including those of added data series would trigger bar update events in the correct chronological order, just like they do in real time. With this behavior, multi series indicators on tick replay charts worked exactly the same, with identical code, for historical and for real time. Now, the synchronization has been lost and there is no practical way to use the historical data from the added series. When CurrentBars[0] is stuck at -2, all of the data for the non primary series loads and flushes down the drain; there is no practical way to use it;

Custom indicators that worked great up to the last beta no longer plot historical data. For example, an indicator that showed the buy sell volume of the ES on a chart of the YM still works in real time, but no longer backfills.

Also, the documentation states that the latest best Bid and Ask are stored in the historical tick files with each trade price. Therefore if you provided a way to access this information for an added one tick secondary series, there would no longer be a need to use the Tick Replay mode.

If it is true that the latest Bid and Ask at the time of each trade are stored with the Last value in the historical tick file, which implies that the Bid and Ask are in the same record in the file as the Last, I find it difficult to understand why GetCurrentBid() and GetCurrentAsk() can't be changed to return historical data for a one tick data series.

[Ricam]

Another bug in Tick Replay

Once again this is something that never happened in any of the public betas up to RC2.

1. In NON Tick Replay a secondary series provides data for the instrument that is named in the first argument of the AddDataSeries method, as expected
   
2. After I check the Tick Replay box, that same secondary series switches over to giving data for the primary instrument, ignoring the first argument of the AddDataSeries method.
   
3. Then after I uncheck the box, the secondary series continues to deliver data for the primary instrument.

Items 2 and 3 are not in any way what would be expected by any reasonable person..

[radekj]

Ricam,
strategies does not work too if you use Tick-Replay in 8.0.1.0 !

[aslane]

This behavior makes the platform completely unusable in simple scenarios.

If you have a simple time based chart, and a simple calc on close indicator that adds a Daily data series, everything works correctly when tick replay is NOT enabled, in that the historical bars play out with the daily bars properly interspersed with the time based bars. When you then turn on tick replay, all of the daily bars are played out first followed by the time based bars, which potentially completely changes the behavior of an indicator.

Even if I assume tick replay will always be on, the use model is horrible at best.

This has to be fixed.

[Ricam]

Ticks out of order

In Post #3 of this thread,

http://ninjatrader.com/support/forum/showthread.php?t=89841

NT Matthew, discussing multi series indicators, said:

If you require the bars to process in the same sequence between real-time and historical, enabling TickReplay should resolve this scenario for you. However, do keep us posted if you see anything you would find unexpected with that setup.

Consistent sequencing in all states was discarded in the transition form RC2 to 8.0.1.
There don't seem to be any beneficial results from this change.

TickReplay seems to be more trouble than it's worth. There must be a simpler way to access the best bid and ask for historical data of a one tick data series.

If this isn't fixed, possible solution would be an updated NT8 Gomrecorder.

[NinjaTrader_Brett]

Ricam,

I started up research to find out what went wrong, I know we worked on a case of bar sequencing for non-tick replay bars that needed to be corrected. It appears something got changed for Tick Replay unexpectedly. Will check into and report back.

-Brett

[Ricam]

hi Brett, thank you for getting this moving.

"If it is true that the latest Bid and Ask at the time of each trade are stored with the Last value in the historical tick file, which implies that the Bid and Ask are in the same record in the file as the Last, I find it difficult to understand why GetCurrentBid() and GetCurrentAsk() can't be changed to return historical data for a one tick data series."

Please keep my previous comments in mind since performance is much better if you don't have to use Tick Replay at all. Tick Replay takes more time to load, and can cause unexpected behavior with indicators that are not coded with it in mind.

[NinjaTrader_Brett]

Ricam,

We applied changes to resolve TIck Replay behavior when you are using multiple series in NinjaScript. This was a regression that was the result of other fixes related.

For those of you working with Tick Replay and multiple series, can you please contact me at platformsupport at ninjatrader dot com ATTN: Brett and reference this post if you would be will to test an advanced build holding these changes. Since I'd like to try to make sure we get this fixed before release 2 which will be coming up soon.

-Brett

[ProfitPilgrim]

User dataseries sequencing control?

I have a sophisticated custom indicator that uses multiple series and timeframes. The developer, who has given me the source code to see if I can come up with a fix, explained to me that without control of the sequence in which various symbols in various timeframes are processed its accuracy declines significantly. So, while this issue is in development I'm wondering if attention might be given to how much user control can be given to the sequencing of added deta series. Is this perhaps possible now?

Thanks for so consistently living up to the now famous responsive support of Ninjatrader, as seen in this thread. I've today requested from Brett testing privileges for the corrected code.

[NinjaTrader_Brett]

If your utilizing tick replay there was a regression in 8.0.1.0 which resulted in what you are seeing. With 8.0.2.0 we have corrected that, I have sent a few builds out in preview and got good feedback. I received your note for an advance build and will send to you as well. Thank You.

Expect 8.0.2.0 coming beginning of next week barring the unforeseen.