Indicator values Discrepancy Historical/Realtime

[YakusaTrader]

Hello,

I am backtesting a strategy with a secondary Time-frame, and indicator values in historical data are shifted 1 minute compared to RealTime data or historical data with TickReplay ON.
I am using 1 range (primary timeframe) to backtest CL 05-19.

My secondary timeframe is:
AddKagi("CL 05-19", BarsPeriodType.Minute, 1, 3, ReversalType.Tick, MarketDataType.Last);

1. Is there a reason why printed values of any indicator are shifted 1 minute from historical to realtime? (This only happened to me with Kagi so far, Minutes timeframe are OK)
2. I think there is a mistake in the following syntax: AddKagi(stringinstrumentName,Data.BarsPeriodTypebaseBarsPeriodType,intbaseBarsPeriodTypeValue,intreversal,Data.ReversalTypereversalType,Data.MarketDataTypemarketDataType)


should be:

AddKagi(stringinstrumentName,Data.BarsPeriodTypebaseBarsPeriodType,int reversal, int baseBarsPeriodTypeValue,Data.ReversalTypereversalType,Data.MarketDataTypemarketDataType)

Can someone please help?

[NinjaTrader_ChelseaB]

Hello YakusaTrader,

In historical OnBarUpdate() updates when the bar closes. 0 bars ago refers to the most recently updated bar which will be the most recently closed bar (and not the currently processing bar).
When in real-time if Calculate is OnPriceChange or OnEachTick, the most recently updated bar is the currently building bar. 0 bars ago will refer to the currently building bar.

Below is a public link to the help guide on how bar data is referenced.
https://ninjatrader.com/support/help...taIsReferenced


The AddKagi() definition in the help guide has the spaces you are suggesting.
https://ninjatrader.com/support/help...us/addkagi.htm

AddKagi(string instrumentName, Data.BarsPeriodType baseBarsPeriodType, int baseBarsPeriodTypeValue, int reversal, Data.ReversalType reversalType, Data.MarketDataType marketDataType)

For example:

AddKagi("AAPL", BarsPeriodType.Minute, 1, 2, ReversalType.Tick, MarketDataType.Last);

Are you looking at something other than the help guide?

[YakusaTrader]

I am using Calculate.OnBarClose, so Historical or Realtime should return the same value.
To make things easier, I attached a script, so you can reproduce the same behavior.

I am printing values of SMA indicator on secondary Data (Kagi). My primary data is 15 seconds"NQ 06-19"

Print(Time[0] + " Close[0]: " + Close[0] + " SMA Value: " + smaFast[0] + " State: " + State);

After attaching the strategy to a 15 seconds NQ 06-19 chart, I get:

4/3/2019 6:25:45 PM Close[0]: 7582.25 SMA Value: 7574.5 State: Realtime
4/3/2019 6:26:00 PM Close[0]: 7582 SMA Value: 7574.5 State: Realtime
4/3/2019 6:26:15 PM Close[0]: 7582 SMA Value: 7574.5 State: Realtime
4/3/2019 6:26:30 PM Close[0]: 7580.5 SMA Value: 7574.5 State: Realtime
4/3/2019 6:26:45 PM Close[0]: 7580.25 SMA Value: 7574.5 State: Realtime
4/3/2019 6:27:00 PM Close[0]: 7580.25 SMA Value: 7574.5 State: Realtime
4/3/2019 6:27:15 PM Close[0]: 7580.25 SMA Value: 7574.5 State: Realtime
4/3/2019 6:27:30 PM Close[0]: 7580.75 SMA Value: 7574.5 State: Realtime
4/3/2019 6:28:00 PM Close[0]: 7580.5 SMA Value: 7574.5 State: Realtime
4/3/2019 6:28:45 PM Close[0]: 7580.5 SMA Value: 7574.5 State: Realtime
4/3/2019 6:29:00 PM Close[0]: 7580.75 SMA Value: 7574.5 State: Realtime
4/3/2019 6:29:30 PM Close[0]: 7580.5 SMA Value: 7574.5 State: Realtime
4/3/2019 6:29:45 PM Close[0]: 7580 SMA Value: 7574.5 State: Realtime
4/3/2019 6:30:00 PM Close[0]: 7579.25 SMA Value: 7581.75 State: Realtime
4/3/2019 6:30:15 PM Close[0]: 7578.75 SMA Value: 7581.75 State: Realtime
4/3/2019 6:30:30 PM Close[0]: 7579 SMA Value: 7581.75 State: Realtime
4/3/2019 6:30:45 PM Close[0]: 7579.75 SMA Value: 7581.75 State: Realtime
4/3/2019 6:31:00 PM Close[0]: 7579.75 SMA Value: 7581.75 State: Realtime
4/3/2019 6:31:15 PM Close[0]: 7579 SMA Value: 7581.75 State: Realtime
4/3/2019 6:31:30 PM Close[0]: 7578 SMA Value: 7581.75 State: Realtime
4/3/2019 6:31:45 PM Close[0]: 7579.75 SMA Value: 7581.75 State: Realtime

Then I reloaded the script (F5):

4/3/2019 6:20:30 PM Close[0]: 7581.75 SMA Value: 7574.5 State: Historical
4/3/2019 6:20:45 PM Close[0]: 7581.25 SMA Value: 7574.5 State: Historical
4/3/2019 6:21:00 PM Close[0]: 7581 SMA Value: 7574.5 State: Historical
4/3/2019 6:21:15 PM Close[0]: 7580.5 SMA Value: 7574.5 State: Historical
4/3/2019 6:21:30 PM Close[0]: 7581.25 SMA Value: 7574.5 State: Historical
4/3/2019 6:21:45 PM Close[0]: 7581.25 SMA Value: 7574.5 State: Historical
4/3/2019 6:22:00 PM Close[0]: 7581.25 SMA Value: 7574.5 State: Historical
4/3/2019 6:22:15 PM Close[0]: 7581.5 SMA Value: 7574.5 State: Historical
4/3/2019 6:22:30 PM Close[0]: 7581.25 SMA Value: 7574.5 State: Historical
4/3/2019 6:22:45 PM Close[0]: 7581.75 SMA Value: 7574.5 State: Historical
4/3/2019 6:23:00 PM Close[0]: 7581.75 SMA Value: 7574.5 State: Historical
4/3/2019 6:23:15 PM Close[0]: 7581.75 SMA Value: 7574.5 State: Historical
4/3/2019 6:23:30 PM Close[0]: 7582 SMA Value: 7574.5 State: Historical
4/3/2019 6:23:45 PM Close[0]: 7582.25 SMA Value: 7574.5 State: Historical
4/3/2019 6:24:00 PM Close[0]: 7581.75 SMA Value: 7581.75 State: Historical
4/3/2019 6:24:15 PM Close[0]: 7582 SMA Value: 7581.75 State: Historical
4/3/2019 6:24:30 PM Close[0]: 7582.5 SMA Value: 7581.75 State: Historical
4/3/2019 6:24:45 PM Close[0]: 7582.5 SMA Value: 7581.75 State: Historical
4/3/2019 6:25:00 PM Close[0]: 7582.5 SMA Value: 7581.75 State: Historical
4/3/2019 6:25:15 PM Close[0]: 7582.5 SMA Value: 7581.75 State: Historical
4/3/2019 6:25:30 PM Close[0]: 7582.25 SMA Value: 7581.75 State: Historical
4/3/2019 6:25:45 PM Close[0]: 7582.25 SMA Value: 7581.75 State: Historical
4/3/2019 6:26:00 PM Close[0]: 7582 SMA Value: 7581.75 State: Historical
4/3/2019 6:26:15 PM Close[0]: 7582 SMA Value: 7581.75 State: Historical
4/3/2019 6:26:30 PM Close[0]: 7580.5 SMA Value: 7581.75 State: Historical
4/3/2019 6:26:45 PM Close[0]: 7580.25 SMA Value: 7581.75 State: Historical
4/3/2019 6:27:00 PM Close[0]: 7580.25 SMA Value: 7581.75 State: Historical
4/3/2019 6:27:15 PM Close[0]: 7580.25 SMA Value: 7581.75 State: Historical
4/3/2019 6:27:30 PM Close[0]: 7580.75 SMA Value: 7581.75 State: Historical
4/3/2019 6:28:00 PM Close[0]: 7580.5 SMA Value: 7581.75 State: Historical
4/3/2019 6:28:45 PM Close[0]: 7580.5 SMA Value: 7581.75 State: Historical
4/3/2019 6:29:00 PM Close[0]: 7580.75 SMA Value: 7581.75 State: Historical
4/3/2019 6:29:30 PM Close[0]: 7580.5 SMA Value: 7581.75 State: Historical
4/3/2019 6:29:45 PM Close[0]: 7580 SMA Value: 7581.75 State: Historical
4/3/2019 6:30:00 PM Close[0]: 7579.25 SMA Value: 7581.75 State: Historical
4/3/2019 6:30:15 PM Close[0]: 7578.75 SMA Value: 7581.75 State: Historical
4/3/2019 6:30:30 PM Close[0]: 7579 SMA Value: 7581.75 State: Historical
4/3/2019 6:30:45 PM Close[0]: 7579.75 SMA Value: 7581.75 State: Historical
4/3/2019 6:31:00 PM Close[0]: 7579.75 SMA Value: 7581.75 State: Historical
4/3/2019 6:31:15 PM Close[0]: 7579 SMA Value: 7581.75 State: Historical
4/3/2019 6:31:30 PM Close[0]: 7578 SMA Value: 7581.75 State: Historical
4/3/2019 6:31:45 PM Close[0]: 7579.75 SMA Value: 7581.75 State: Historical
4/3/2019 6:32:00 PM Close[0]: 7580 SMA Value: 7581.75 State: Historical
4/3/2019 6:32:15 PM Close[0]: 7578.25 SMA Value: 7581.75 State: Historical
Enabling NinjaScript strategy 'TestSMA/xxxxxx'
4/3/2019 6:32:30 PM Close[0]: 7578.25 SMA Value: 7581.75 State: Realtime
4/3/2019 6:32:45 PM Close[0]: 7578 SMA Value: 7581.75 State: Realtime
4/3/2019 6:33:00 PM Close[0]: 7577.75 SMA Value: 7581.75 State: Realtime
4/3/2019 6:33:15 PM Close[0]: 7578 SMA Value: 7581.75 State: Realtime
4/3/2019 6:33:30 PM Close[0]: 7578 SMA Value: 7581.75 State: Realtime
4/3/2019 6:34:00 PM Close[0]: 7578.25 SMA Value: 7581.75 State: Realtime

You can see that SMA indicator values are different from historical to Realtime.
Shouldn't be the same?

[NinjaTrader_ChelseaB]

Hello YakusaTrader,

The Close prices all appear to be the same.

There seems to be less data in real-time. A different number of bars on the chart could be changing the calculation.

Is the number of bars changing when you reload the historical data?

Is the chart blank when you are enabling the script?

[YakusaTrader]

Close price is the same but not indicator's values.
The script was loaded from the same chart, same bars (I used start date, custom range, 03/31/2019 - just to have consistent starting bars) .
The chart was not blank when I enabled the script.

I was able to reproduce this on 2 computers. I think this has to do with Kagi chart.
Indicator values are the same when I replace secondary Kagi chart with regular time based chart, e.g. 6-minute chart instead of 6-minute Kagi chart.

Again, all I did is enabling the script from the chart after loading it on a 15 seconds with starting date of 03/31/2019, then collecting realtime prints of indicator value, them refreshing the script (F5), then comparing the realtime indicators values to their historical values.

[NinjaTrader_ChelseaB]

Hello YakusaTrader,

Can I test the script?
(Or preferably a reduced version of the script that has all non-related code removed)

To export a NinjaTrader 8 NinjaScript so this can be shared and imported by the recipient do the following:

1. Click Tools -> Export -> NinjaScript...
2. Click the 'add' link -> check the box(es) for the script(s) and reference(s) you want to include
3. Click the 'Export' button
4. Enter a unique name for the file in the value for 'File name:'
5. Choose a save location -> click Save
6. Click OK to clear the export location message

By default your exported file will be in the following location:

• (My) Documents/NinjaTrader 8/bin/Custom/ExportNinjaScript/<export_file_name.zip>

Below is a link to the help guide on Exporting NinjaScripts.

http://ninjatrader.com/support/helpGuides/nt8/en-us/export.htm

[YakusaTrader]

Absolutely, it was attached to my second reply.

[NinjaTrader_ChelseaB]

Hello YakusaTrader,

I have tested the script.

I am finding the value for the SMA is the same in real-time as it is after reloading in historical.

The first thing I am noticing is that in the output I have generated, the first bars start at the same time. This is letting me know the data is the same and the number of bars is the same.

Attached is the output for you to analyze.

I'm seeing the SMA Value: 7600 on the first real-time bar at 11:23 AM on 4/8 matches the same historical bar after the script is reloaded.
The last real-time bar at 12:16 PM bar on 4/8 at 7611.5 also matches the historical bar at the same time after being reloaded.

[YakusaTrader]

Are you sure?

Please double check the data you just sent me.

8/2019 11:31:00 AM     Close[0]: 2892.5       SMA Value: 7600        State: Historical
4/8/2019 11:32:00 AM     Close[0]: 2892.5       SMA Value: 7600        State: Historical
4/8/2019 11:33:00 AM     Close[0]: 2893.25       SMA Value: 7600        State: Historical
4/8/2019 11:34:00 AM     Close[0]: 2893       SMA Value: 7600        State: Historical
4/8/2019 11:35:00 AM     Close[0]: 2893.25       SMA Value: 7600        State: Historical
4/8/2019 11:36:00 AM     Close[0]: 2893.5       SMA Value: 7611        State: Historical
4/8/2019 11:37:00 AM     Close[0]: 2893.25       SMA Value: 7611        State: Historical
4/8/2019 11:38:00 AM     Close[0]: 2893.25       SMA Value: 7611        State: Historical
4/8/2019 11:39:00 AM     Close[0]: 2893.5       SMA Value: 7611        State: Historical
4/8/2019 11:40:00 AM     Close[0]: 2893.25       SMA Value: 7611        State: Historical
4/8/2019 11:41:00 AM     Close[0]: 2893.25       SMA Value: 7611        State: Historical
4/8/2019 11:42:00 AM     Close[0]: 2892.75       SMA Value: 7611        State: Historical
4/8/2019 11:43:00 AM     Close[0]: 2893       SMA Value: 7611        State: Historical

4/8/2019 11:32:00 AM Close[0]: 2892.5 SMA Value: 7600 State: Realtime
4/8/2019 11:33:00 AM Close[0]: 2893.25 SMA Value: 7600 State: Realtime
4/8/2019 11:34:00 AM Close[0]: 2893 SMA Value: 7600 State: Realtime
4/8/2019 11:35:00 AM Close[0]: 2893.25 SMA Value: 7600 State: Realtime
4/8/2019 11:36:00 AM Close[0]: 2893.5 SMA Value: 7600 State: Realtime
4/8/2019 11:37:00 AM Close[0]: 2893.25 SMA Value: 7600 State: Realtime
4/8/2019 11:38:00 AM Close[0]: 2893.25 SMA Value: 7600 State: Realtime
4/8/2019 11:39:00 AM Close[0]: 2893.5 SMA Value: 7600 State: Realtime
4/8/2019 11:40:00 AM Close[0]: 2893.25 SMA Value: 7600 State: Realtime
4/8/2019 11:41:00 AM Close[0]: 2893.25 SMA Value: 7600 State: Realtime
4/8/2019 11:42:00 AM Close[0]: 2892.75 SMA Value: 7600 State: Realtime
4/8/2019 11:43:00 AM Close[0]: 2893 SMA Value: 7611 State: Realtime

[YakusaTrader]

You know what, don't worry about it.
I was able to fix it by creating a new bar style.

[NinjaTrader_ChelseaB]

Hello YakusaTrader,

I had a deeper look into this and you are correct. Its coming down to when the secondary bar closes.

I believe this has to do with how tick data is processed historically, but I'm currently inquiring further.

[NinjaTrader_ChelseaB]

Hello YakusaTrader,

Thanks for your patience on this.
I've gotten some new perspective on Kagi bars that would explain the bar close times.

Three important points from our dev team:

Kagi bars close after the reversal is seen, but the the close time in historical data is marked with the last tick received before the reversal. The first tick that triggers the reversal is part of the new bar.

In real-time the close time of the bar is the current time when the reversal is triggered.

Since the close of the bar is marked with the timestamp of the last update to the bar, historical data gets an earlier timestamp than when the reversal triggered the formation of a new bar.

Therefore, the Kagi bar closes with an earlier timestamp historically after the primary series closes, and it comes out in historical with a different order.

This means that now the Kagi bar has closed before all of those primary minute bars and the SMA value is shifted back to that time.