NT8061 - Playback-MR takes wrong trades

[sledge]

More of something informational - maybe someone else will notice this in the future.

I was comparing NT8 Playback-MR with same strategy/data on 2 different system - and was getting different results from 2016/2017. Substantial enough such that I found a weeks worth of data missing and some other data files missing. So I sync'd them up.

I ran a full replay from 1/20/2014 to 4/21/2017 on both setups. There was a difference of $4000 in my results.

I compared both results - Laptop #2 - took 2 "bad trades" on 3/7/2014 and 3/27/2014. They were bad because they didn't follow the entry condition at all. Both values were not > the requirement. The days before replayed normal and no data was missing.

This strategy resets everyday around 4pm and 9:30 am, just in case there is missing data. I didn't save off the bad results - I figured this would be easy to reproduce - I'd be able to put in some debug statements and figure out what was going wrong.

on laptop #2:
I replayed the 2 days in question. It didn't take the trade! argh!
I replayed the whole week. It didn't take the trade! argh!
I restarted the machine and followed same steps I always do and I replayed from the start (1/20/2014), and it didn't take the trade! So much for being able to reproduce the bad trades.

I'm glad it didn't take the trade because it isn't supposed to. But HOW it took the trade the other night is a mystery. I was running no other processes such as TV that night on that laptop.

I'm currently re-running the entire replay and will compare again to the other laptop and see if the results match or differ. Maybe some other days will be "bad".

There might be some timing issue in replay that causes things to get out of sync. (I noticed the values on the day before 3/6 finished with a condition that would cause a good condition if it were used the next day, but the chart with plots loaded in fine)...

This maybe next to impossible to reproduce, especially if I switch to some test code setup.

So: trust but verify on 2 systems.

[NinjaTrader_Jesse]

Hello Sledge,

That does sound perplexing but it also sounds like it could have been a one-time occurrence perhaps related to the data change/sync? The only thing I could think of would be if you see this again, save your custom folder and market data before syncing the computers up. We could try to recreate it by switching to the old copy and then sync the data again to see if its related to that.

I would be interested if you do find a reproducible situation where it is not obeying the logic or doing otherwise.

If you have further details I will be observing the post to follow if I can help.

I look forward to being of further assistance.

[sledge]

I've been experiencing more of this with running 3 strategies with slightly different parameters.

I could tell from the start (3 months) that trades look pretty crazy.

I added to a previous print condition inside each strategy where the trade entry is made to printout the conditions - and this replay is 5 months in, and looks perfect so far. All 3 strategies have the same results, instead of completely different results.

There might be some sort of compilation issue when save as/changing conditions and adding to charts. Something isn't right somewhere...

I'm going to let this replay finish, and replay some more with these same strategies tomorrow night and see if these strategies go bonkers again.... and I'll at least have a printout of the conditions, and might be able to recreate something with some sample code.

*With regards to this post - I am using 8.0.7.0*

[sledge]

to the old copy and then sync the data again to see if its related to that.

I would be interested if you do find a reproducible situation where it is not obeying the logic or doing otherwise.

If you have further details I will be observing the post to follow if I can help.
.

I've recreated the condition in a separate clean strategy/indicator in NT8 8070. I will post them after a little more testing.

Basically - I create an indicator with BarsArray[0] (which would happen to be ES).

Now inside the indicator - I'm finding dates/times that are out of order. For example, the replay time is 1/31/2014 10:00am, but at 1/31/2014 @ 9:30:15 am - I have a Print (ToDay(Time[0])) of 1/29/2014 with print (ToTime(Time[0])) = 9:37:30

Note: Inside the strategy I do have if (BarsInProgress!=0) return. I do have a secondary series which I will test without.


BINGO - I re-ran replay - and the out of order date was not present.

I'll test some more current dates also in 8.0.7.0 and maybe try some previous NT 8 versions to try to pinpoint when this started. I can't say I noticed until 8.0.6.1 when I started to lock down the strategy and run same tests on different machines to sync up all data.

This is on 2 different machines, Win7 64 bit, in both NT8 32/64 bit.

And another print on replay #2 from 4 days earlier inside the indicator.

[sledge]

Time[0] is not always correct with multi-dataseries/strategy in Indicator.

I would be interested if you do find a reproducible situation where it is not obeying the logic or doing otherwise.

If you have further details I will be observing the post to follow if I can help.

I look forward to being of further assistance.

Here it is. I have tested it as far back at 8.0.3.0.

Time[0] is not always correct with multi-dataseries/strategy in Indicator.

When a secondary series is used in the main strategy - Time[0] in the indicator gets out of whack and is in ERROR. The strategy doesn't have the Time[0] error.

And to top it off - it is random. You won't get it to repeat exactly the same way the next replay run.


I have tested with converted NT7 and downloaded data from the replay servers, and different machines. It should be reproducible on your end.

Steps to recreate
----------
1) Install indicator and strategy.
2) Download 2-4 weeks or more of data for 2 instruments if you do not have it.
3) Modify strategy to include 2nd instrument. EXA: ES main, 6E secondary.
(using the same instrument and 2nd instrument doesn't produce the problem: EXA ES/ES.) I am using ES/^VIX in my real setup.
4) Setup workspace, Add ES 06-17 - 15 second chart. Open output window. Save workspace.
5) Connect to replay. Set up dates.
6) Add strategy to chart.
7) Run replay. I have produced the issue all the way down to 50x from MAX.
8) Watch output window. It will only print when Time[0] is out of whack.

I'm not too concerned about being off 15 seconds, but when it is off by days or hours - something more is wrong.

For example:

Failed Replay Time=20170227 120445 $2348.75 LBT=3/21/2017 2:35:15 PM

BarsArray[0].LastBarTime is correct, the market replay is at 3/21/2017 at 2:35:15pm, but Time[0] is saying it is 2/27/2017 at 12:04:45 PM. ES was at 2348.75.

Rerunning replay (in 8.0.7.0, you must disconnect replay and then reconnect) - will causes other time differences as mentioned.

If you remove the AddDataSeries, no problems. Everything is in sync.

Also of note: sometimes there might be no problems or 1 or 2, and then sometimes there are a ton of problems.

I'm not sure if more cores/threads is also an issue... I've got 1 hit on an Atom Z3735F in 8.0.3.0, but 8.0.7.0 on my 5820k is a total mess.

[NinjaTrader_Jesse]

Hello,

Thank you for posting the details.

I noted that you are using the LastBarTime which is not currently documented, I will likely need to question development on using property in a strategy rather than a BarsType which seems to be the only place I can see this is used.

I will review what you have provided and reply back once I have.

I look forward to being of further assistance.

[sledge]

Hello,

Thank you for posting the details.

I noted that you are using the LastBarTime which is not currently documented, I will likely need to question development on using property in a strategy rather than a BarsType which seems to be the only place I can see this is used.

I will review what you have provided and reply back once I have.

I look forward to being of further assistance.

Ok thanks for looking into this.

Just to be clear, I've been using Time [0] in the indicator from the strategy which randomly fails.

LastBarTime was only something I could find to compare to that proved Time [0] was failing in the indicator which is called from the strategy.

The strategy's use of Time [0] is not causing a problem.

[NinjaTrader_Jesse]

Hello,

I just wanted to post back on this item.

I checked with development on the use of LastBarTime and in this case, the condition you are creating would not be expected to always equal the Time[0] object so I don't believe this would be a valid way for you to check this item.

LastBarTime is outside of the OnBarUpdate context so it is not guaranteed to be in sync like standard Strategy properties that are inherited.

Perhaps you could utilize Drawing Objects and store the time as text/tag, or potentially write to file the Time[0]/BarsAgo to match the Times to the chart? I am not certain of a way you could check that the time is valid by using a different time object. I believe storing the time as Text or as a Tag may be easier, you could double click on an object you think is in error and view the stored time/text to see if it matches the bar potentially.

I look forward to being of further assistance.

[sledge]

Hello,

I just wanted to post back on this item.

I checked with development on the use of LastBarTime and in this case, the condition you are creating would not be expected to always equal the Time[0] object so I don't believe this would be a valid way for you to check this item.

LastBarTime is outside of the OnBarUpdate context so it is not guaranteed to be in sync like standard Strategy properties that are inherited.

In my tests LastBarTime matches replay time.

Time [0] is it out sync by DAYS sometimes.

I'll write it to a file and show that way then.

[sledge]

Perhaps you could utilize Drawing Objects and store the time as text/tag, or potentially write to file the Time[0]/BarsAgo to match the Times to the chart? I am not certain of a way you could check that the time is valid by using a different time object. I believe storing the time as Text or as a Tag may be easier, you could double click on an object you think is in error and view the stored time/text to see if it matches the bar potentially.

I look forward to being of further assistance.

Create a folder called "Test" in your My Documents, or change the directory location in the code of the Indicator. I open and create files daily and write the information to them.

For example, My Documents was moved to my S:\ drive with 2 1.2TB PCIe NVME 750 in Raid 0.

Opened New File:S:\mydocs2016\Test\2017-03-20-time[0]-test.txt
Closed File:S:\mydocs2016\Test\2017-03-20-time[0]-test.txt

You can manually look through the files, but you will find the ***^^^^^^^^^^^^^^^^^^^^^^^^^^^*** line check with LastBarPrice will point it out very quick.

Repeating the runs will cause randomness. Never the same place.

3/20 first run had 1 spot. (Not attached).
3/20 second had was clean. (Attached).
3/21 has 2 spots.

3/20/2017 3:00:15pm does not belong on 3/21 replay!!!!

Hopefully development can fix this bug without slowing down replay. (It happens at any speed in replay).

[SIZE="1"][FONT="Courier New"]ReplayTime=20170321  94045 Close[0]=2376.75 High[0]=2376.75 Open[0]=2376.75 Low[0]=2376.75   Times[0][0]=20170321  94045
[B][COLOR="Red"]ReplayTime=20170320  150015 Close[0]=2367.5 High[0]=2367.75 Open[0]=2367.5 Low[0]=2367.25   Times[0][0]=20170320  150015[/COLOR][/B]
***^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^****
ReplayTime=20170321  94045 Close[0]=2376.75 High[0]=2377 Open[0]=2376.75 Low[0]=2376.75   Times[0][0]=20170321  94045
ReplayTime=20170321  94100 Close[0]=2376.75 High[0]=2376.75 Open[0]=2376.75 Low[0]=2376.75   Times[0][0]=20170321  94100[/FONT][/SIZE]

11:18:30 am is used again at 11:31:15am??

[SIZE="1"]
ReplayTime=20170321  113115 Close[0]=2350 High[0]=2350 Open[0]=2350 Low[0]=2350   Times[0][0]=20170321  113115
[B][COLOR="Red"]ReplayTime=20170321  111830 Close[0]=2351.5 High[0]=2351.75 Open[0]=2351.75 Low[0]=2351   Times[0][0]=20170321  111830[/COLOR][/B]
***^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^****
ReplayTime=20170321  113115 Close[0]=2349.75 High[0]=2350 Open[0]=2350 Low[0]=2349.75   Times[0][0]=20170321  113115[/SIZE]

11:34am is during 11:36:15am??

[SIZE="1"]ReplayTime=20170321  113615 Close[0]=2346.5 High[0]=2346.5 Open[0]=2346.5 Low[0]=2346.5   Times[0][0]=20170321  113615
[B][COLOR="Red"]ReplayTime=20170321  113400 Close[0]=2347.75 High[0]=2348.5 Open[0]=2348.25 Low[0]=2347.5   Times[0][0]=20170321  113400
[/COLOR][/B]***^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^****
ReplayTime=20170321  113615 Close[0]=2346.5 High[0]=2346.75 Open[0]=2346.5 Low[0]=2346.25   Times[0][0]=20170321  113615[/SIZE]

[NinjaTrader_Jesse]

Hello,

Thank you for the reply.

I reviewed the revised samples but these are still using the syntax we confirmed in the last post that should not be used for accuracy reasons:

BarsArray[0].LastBarTime

This property is not documented for NinjaScript use and is only used in BarTypes specifically that ship with the platform. After questioning development regarding the condition you created, it was determined that would not be a valid check as it is not an inherited NinjaScript property

LastBarTime is outside of the OnBarUpdate context so it is not guaranteed to be in sync like standard Strategy properties that are inherited.

I am not certain of a way for you to validate the Time[0] object against another property because I am unaware of any other synced/inherited Time properties. Perhaps you could test Time[0] against Times[0][0] or use Drawing Objects and write the Time to the object as noted previously. The attached test specifically would not be a good way to test this concept as the properties are not guaranteed to be synced so they could very well have different values as you are showing.

Please let me know if I may be of further assistance.

[sledge]

Times [0][0] is messed up too.


Get me someone else to look at this because you're honestly not getting it. I need this raised and an getting tired of wasting my time with this bug that I've proven.

LastBarTime had nothing to do with my original code and the fact that in comparing it to pride Time [0] is wrong is secondary,I'm not printing out LastBarTime.

[NinjaTrader_PatrickH]

Hello sledge,

Thank you for your response.

Jesse and I are looking at line 106 of the indicator where you are still calling BarsArray[0].LastBarTime. Without this check do you still see the same behavior?

I look forward to your response.

[sledge]

I can't see the source right now, but in the last code I posted with the daily file written out, the LastBarTime only is used to print out ^^^^^^^^^^^^^ so you have a chance of knowing where in the log file the Time[0] has failed. Else you have to skim at everyline yourself for the out of sequence Time [0].

Further testing I did this past weekend shows Times [0][0] is also wrong. And the whole dataseries going back in time is wrong with regards to replay time, but looks correct for the incorrect series, wherever that is derived from.

[NinjaTrader_PatrickH]

Hello sledge,

Thank you for your response.

I think focus has been on your code but I understand what you are detailing. I will be testing on my end and I will follow up here when I have further information.