Strategy result changing when reloading

[PJtrades]

Hello,

Not sure if this is covered anywhere and I have done numerous searches, so apologies if this is a redundant post.

I am a novice with any programming and even with the use of the tools such as Strategy Builder and so I may be missing something obvious but here goes.

So, I have created a strategy through the NT8 Strategy Builder that uses Order Flow Cumulative Delta, Order Flow VWAP, a couple of moving averages and BetterRenko and puts up and down arrows on the chart. Like I said, I used the builder and I've not unlocked the code or anything to change the programming at all. I am testing by running from live data and it seems to be working except in some instances when it is giving me false signals. When I then take the strategy off the chart and replace it and reload historical data, the false signals are gone. I do have this set to Calculate on Bar Close, so it's not an instance of the criteria changing at the close of the bar. I am at a loss.

Thanks in advance for any help in this matter,

PJ

[NinjaTrader_Kate]

Hello PJtrades,

Thank you for your post.

This behavior is likely due to your use of Renko bars. Renko bars, when in real-time, re-draw the open based on the direction of the bar. This behavior can't be simulated with historical data. Each bar of historical data has a High, Low, Open and Close which are static and do not change as you might see in real time. This is a technical limitation of this bar type and is not limited to the NinjaTrader platform.

• Do you see similar results when running the same test on the SampleMaCrossOver strategy in NinjaTrader with the same settings as your strategy?
• Who are you connected to? This is displayed in green on lower left corner of the Control Center window.
• Are you connected to your data feed provider when running this test?
• What instrument(s) (and expiry if applicable) have you selected?
• Is your strategy a multi instrument or multi time frame strategy?
• Do you receive an error on screen? Are there errors on the Log tab of the Control Center? If so, what do these errors report?

Thanks in advance; I look forward to further investigating this item.

[PJtrades]

Hi Kate,

Thanks for your quick reply. It will take some time for me to get through your bullet points. I will get back with you soon.

PJ

[PJtrades]

I am familiar with the behavior of Renko bars, however BetterRenko bars do not repaint. I am attaching a screenshot of a current bar comparing regular candles to BetterRenko. The current price (Close in the data window) is the same on both candles(regular candles on the left in the screenshot). As for your other questions, see below:

• Do you see similar results when running the same test on the SampleMaCrossOver strategy in NinjaTrader with the same settings as your strategy?

Not exactly sure what I am looking for here. I loaded the strategy on a chart and it doesn't draw anything. It did take a trade while I was just now typing.

• Who are you connected to? This is displayed in green on lower left corner of the Control Center window.

Rithmic

• Are you connected to your data feed provider when running this test?

Yes, as I said, I am running things as if I was trading live but just taking the trades in Sim.

• What instrument(s) (and expiry if applicable) have you selected?

This is NQ 03-19

• Is your strategy a multi instrument or multi time frame strategy?

No

• Do you receive an error on screen? Are there errors on the Log tab of the Control Center? If so, what do these errors report?

No, the only entries in the log related to the strategy are from enabling and disabling the strategy.

Thanks,

PJ

[NinjaTrader_Kate]

Hello PJtrades,

Thank you for your reply.

We did not create the 'BetterRenko' custom bar type, so we would have no direct information on why it works the way it does. Please reach out to the original developer for more information specific to this bar type.

That being said, I did download the Better Renko bar type and do some testing against regular Renko bars - I noted that the open was different between the two charts, and sometimes the two indicated different directions which could account for the false signals you're seeing.

Please also review the help guide document on the differences on real-time vs backtest (historical).

https://ninjatrader.com/support/helpGuides/nt8/en-us/discrepancies_real-time_vs_bac.htm

When running a strategy on historical data, only the Open, High, Low, and Close will be available and there will be no intra-bar data. This means actions cannot happen intra-bar, fills cannot happen intra-bar. All prices and actions come from and occur when the bar closes as this is all the information that is known.

Because of this, OnBarUpdate will only update 'On bar close' as it does not have the intra-bar information necessary for 'On price change' or 'On each tick'.

This could also certainly account for the signals that you're seeing that disappear when historical data is reloaded - when it was running in real time, it could use that intra-bar data, but when you reload the historical data the strategy is recalculated using the historical data instead and the "false signals" disappear.

If you are comfortable with providing your strategy so I may test on my end, could you please export it, then either post it here, or you can email it to platformsupport[at]ninjatrader[dot]com, referencing "ATTN Kate W" and the link to this thread in the body of the email.

Instructions on exporting below:

https://ninjatrader.com/support/helpGuides/nt8/en-us/?export.htm#ExportingninjascriptAsSourceFiles

Thanks in advance; I look forward to resolving this for you.

[PJtrades]

Hi Kate,

Sorry I didn't reply sooner. I am trying to sort through some things here on my own. I did find somewhere else in the forums where there was some kind of issue with time stamps or something along those lines with non time-based bars. Not sure if this is where my problem is but it would make sense, as my issue is always a signal that is late by one bar. Anyway, I will follow up with you soon by exporting the strategy and sending it.

Thanks!

[NinjaTrader_Kate]

Hello PJtrades,

Thank you for your reply.

I'll be keeping an eye out for your email with your strategy.

Thanks in advance; I look forward to further assisting you.

[NinjaTrader_Kate]

Hello PJTrades,

Thank you for sending your strategy in.

If using tick based charts, such as your BetterRenko bars, all it takes is a single tick difference between real-time and historical data to generate completely different looking charts. This in turn would impact the calculations of your strategy should the data sets be different.

This forum post goes into great detail on how to use prints to help figure out where the differences might lie - it might have to do with errant ticks in the real time data that are filtered out in the historical data, but this should get you going in the correct direction. You can even add these using the Strategy Builder.

https://ninjatrader.com/support/forum/forum/ninjatrader-7/strategy-development-aa/99209-date-conditions-not-working

Please add prints to your script that print the open, high, low and close of the bar. Also, print out all values used in the condition in question that are evaluating differently. With the printout information we can assess what is different between the two. Please attach this information to your next reply and I will help you analyze the output.

Please let us know if we may be of further assistance to you.

[PJtrades]

Hi Kate,

I will be attaching the text docs for the output. Is that correct?

Before I do that, I wanted to point out what I found. I am attaching a screen shot and a text file that just shows the printed output from the area in question in my screenshot:

It seems that the Cumulative Delta values are not right. In the screenshot you will notice the 11:58:43 bar is the bar I have the data box open for. It shows the Cumulative Delta Close value as 1084. The Output Print shows the CD Close as 1108.I have not reloaded the NinjaScript since just prior to these last two signals in the screenshot. If I reload it, I'm sure it will move those two arrows over one bar to the right on both.

Again, I will follow with the output files. I just want to be sure it is indeed text files that you need.

Thanks again.

[PJtrades]

And here is the print from just the three candles around the signal. I wasn't sure how to attach both the screenshot and this to the same post.

[PJtrades]

Couldn't get the files attached because the size.

Just had another idea. Trying again as zip of both files

[NinjaTrader_Kate]

Hello PJtrades,

Thank you for your replies.

First, I'd just like to compliment you on how nicely formatted your prints are - they are some of the best we've seen. Great job on those!

It looks like from your screenshot that you're using Global Crosshairs, is that correct? Is your cursor over a different chart? The time the cursor is on doesn't match the time on the Data Box in the screenshot. Also, the bar you've pointed out in the screenshot is only included in one of the files you provided - none of the others get to the 11:58:43.322 bar.

Am I understanding correctly that the file with the 3 prints was produced when running on real time data?

Let's focus on the 11:58:43.322 bar - have you reloaded the strategy yet? What results do you get for this bar then? Do the results when you reload the strategy for this bar match what you're expecting to see in the data box as far as the Cumulative Delta?

I'm curious to see what results you get - if you could provide the print for the 11:58:43.322 bar after the reload that would be great.

[PJtrades]

Wow, thanks for the compliment. I'd have to say that I owe the formatting to Chelsea, the gentleman who did the video explaining how to use Strategy Builder to create them. He did a great job of walking through the process.

As for the screenshot, I wasn't using global crosshairs. I'm not sure why line was there. In fact I noticed that myself and had to check twice to see if the data was still for the correct bar. It's just that the screen capturing software I use has a launch button that lives on the top of my screen and I carefully moved the mouse straight up to the button while keeping the vertical on the desired bar.

The files that I sent were just the prints I had done prior to that point. I will follow up with the others.

Yes, the file with the three prints was from running real time data.

Focusing on the 11:58:43:322 bar, here is the print from after reloading the script:

11:58:43.3220000 || CumDelta[0]: 1085 > CumDelta[1]: 1109 || EMA (6)of Delta[0]: 1110.93861574298 > EMA (34) of Delta[0]: 1196.40418360959 || CumDelta[1]: 1109 < CumDelta[2]: 1105 || CumDelta[0]: 1085 > EMA (6)[0]: 1110.93861574298 || Close[0]: 7091.5 > Low[0]: 7091.5 || Close[1]: 7092.5 < High[1]: 7092.5 --OR-- Close[2]: 7092 < High[2]: 7092 || Close[0]: 7091.5 > VWAP: 7091.46190798715 || Close[0]: 7091.5 < VWAP Std Dev 2: 7103.2130199714 || Low[0]: 7091.5 <= VWAP +1 tick: 7091.71190798715 || Open: 7092.5 || High: 7092.5 || Low: 7091.5 || Close: 7091.5

The prints were done using the criteria for long signals, not short signals so the logic here wouldn't end up proving this true but looking at the values, you can see in fact there is a difference from the real time data. In fact, the signal on this particular bar disappeared entirely because the bar before which is the bar that should have signaled didn't close on the right side of the VWAP standard deviation, which is one of the criteria You can see in the new screenshot that it is gone entirely. I marked the signals with vertical lines before reloading.

I will attach the output file to the next post.

Thanks!

[PJtrades]

Here is the file with the prints from the day after the reload.

[NinjaTrader_Kate]

Hello PJtrades,

Thank you for your reply.

So, just to recap - here's the two prints, the first one was your one that used real-time data, the second is after reloading:

11:58:43.3220000 || CumDelta[0]: 1108 > CumDelta[1]: 1103 || EMA (6)of Delta[0]: 1118.05580817656 > EMA (34) of Delta[0]: 1200.48605732038 || CumDelta[1]: 1103 < CumDelta[2]: 1083 || CumDelta[0]: 1108 > EMA (6)[0]: 1118.05580817656 || Close[0]: 7091.5 > Low[0]: 7091.5 || Close[1]: 7092.5 < High[1]: 7092.5 --OR-- Close[2]: 7092 < High[2]: 7092 || Close[0]: 7091.5 > VWAP: 7091.46190798715 || Close[0]: 7091.5 < VWAP Std Dev 2: 7103.2130199714 || Low[0]: 7091.5 <= VWAP +1 tick: 7091.71190798715 || Open: 7092.5 || High: 7092.5 || Low: 7091.5 || Close: 7091.5

11:58:43.3220000 || CumDelta[0]: 1085 > CumDelta[1]: 1109 || EMA (6)of Delta[0]: 1110.93861574298 > EMA (34) of Delta[0]: 1196.40418360959 || CumDelta[1]: 1109 < CumDelta[2]: 1105 || CumDelta[0]: 1085 > EMA (6)[0]: 1110.93861574298 || Close[0]: 7091.5 > Low[0]: 7091.5 || Close[1]: 7092.5 < High[1]: 7092.5 --OR-- Close[2]: 7092 < High[2]: 7092 || Close[0]: 7091.5 > VWAP: 7091.46190798715 || Close[0]: 7091.5 < VWAP Std Dev 2: 7103.2130199714 || Low[0]: 7091.5 <= VWAP +1 tick: 7091.71190798715 || Open: 7092.5 || High: 7092.5 || Low: 7091.5 || Close: 7091.5

We can definitely see a difference between the two - they're giving close, but not the same data. The bar data appears to be the same, however, the indicators values are what's different. I suggest turning on Tick Replay for this strategy and repeating these tests - Cumulative Delta needs tick data coming in, and Tick Replay may be able to make these two match.

Here's a link to our help guide that explains how to turn on Tick Replay:

https://ninjatrader.com/support/helpGuides/nt8/en-us/?tick_replay.htm

Please let me know the results of this test.