Datafeed transition problem

[JohnQ]

I don't really understand the problem, only the symptoms if that makes sense. I am pretty sure someone will understand what I am dealing with.

Since it's likely to be asked in this context, assume COBC == true;
The order triggering is done something like this:
// where hOset is the bar offset for historical chart
bar1 = BOP(sellBOPPeriod, sellBOPSmooth).TheBOP[1 + hOset];
bar2 = BOP(sellBOPPeriod, sellBOPSmooth).TheBOP[2 + hOset];
if(Close[1 + hOset] > High[2 + hOset] && bar1 > bar2)
EnterShort();

The symptoms:
If I load the strategy to a running replay or live connection (inside the session time frame, IE bars are plotting). None of the historical order timing is correct, none of the plotting that tracks those orders is correct because the orders are not correct. The timing of the orders is wrong and in some cases the tracking of if orders are open or closed seems to break down. I know this because I basically plot(and output window) everything related to execution.

Under ALL other circumstances everything is working correctly.

So, I have these questions:
1. Help me understand what is going on here. Why is this happening?
2. What approaches are practical to getting a strategy to load onto a running chart and not have the sequencing or whatever issue?
3. Is this issue a limitation of NT? Meaning it's just not possible to load a strategy historically onto a "live" chart and have everything happen and plot correctly both historically and from then on?

My thinking is that in the above condition (loading to a plotting chart) I need to "hold" the datafeed back until the first tick of the NEXT bar. That is the only thing I see different here (data comes in mid bar vs. on first tick as with session startup). My attempts at forcing the sync of the datafeed to the first tick of the next bar in live chart have proved futile....

MANY MANY thanks in advance!

JohnQ

[NinjaTrader_ChelseaB]

Hello JohnQ,

When you mention incorrect, what exactly is incorrect about them? Are you seeing orders that appear on the next bar from the trigger bar with Calculate on bar close set to True?

If so, that is expected. The order is placed after the trigger bar closes. Thus the order is included with the next bar because it was placed just after the next bar opened.

If the strategy were running with COBC as false, or if the script has intra-bar granularity and submitted orders on a smaller timeframe, this would cause the timestamp of the order to be within the bar it is triggered on as the order may be submitted intra-bar. This order would show on the same bar as the bar that triggered the order submission.

This is outlined in the help guide Discrepancies: Real-Time vs Backtest.

http://www.ninjatrader.com/support/helpGuides/nt7/discrepancies_real_time_vs_bac.htm

Or is the issue something else?

[JohnQ]

ChelseaB,
Thanks a ton for the quick reply.
This is something else.

COBC is set to false. However, the orders are restricted to BarsInProgress == 0 && FirstTickOfBar == true
Yes multi-timeframe script. I bring in a 3 second chart during historical backtest to improve the accuracy.
I allow only the trade exit logic to process outside of BarsInProgress == 0
NONE of the order submission code resides outside of BarsInProg == 0

This issue occurs with both replay and live conditions.
If I am plotting chart (and say 1:30 into a 5min bar), hit F5 (or equivalent) historical trades shown will be messed up (don't happen at the right time, MarketPostion tracking is off). If I do the same thing, during a time outside the session window, everything works correctly.

The issue ONLY happens in conditions where the strategy is being started (replay or live) within the trade-able session window (chart plot is moving). If I startup outside the session window. Everything historical (and forward) will be correct.

I wish it were possible to explain better "exactly" what happens. That is part of my difficulty, if it just looked like the chart was off by a bar, that would be one thing. In this case, everything (that works otherwise) falls apart...

I plot a mark every time there is an attempt to trade. The mark is dot if MarketPostion == Flat the mark is a diamond when MarketPosition != flat.

When this issue occurs I am getting diamond marks outside of where the system is really in a trade. The trades are not happening as expected (timing appears off by 1 bar but it's difficult to say because all the tracking gets messed up), but it also appears the tracking of MarketPosition is breaking down... If I had to guess, it looks like the trade has fired 2 bars late in the problem condition. MarketPostion issue not withstanding. I don't know what to think, much of the trading shown when this occurs shouldn't even have happened. I have logic to prevent trades from being taken when I detect MarketPosition != flat, with this condition I have diamonds all over the chart... Most of these should be dots.

Thank you for trying to help me figure this out.

JohnQ

[JohnQ]

OK,
I fixed it.
The fix leaves me with a new question if anyone is still following.
So prior to submitting the order I use:
Instrument.MasterInstrument.Round2TickSize(myEntry Price);

That fixed everything related to the original problem.

The issue now being, I still see NT strategy results that look as if the fill price wasn't tick corrected. I call the above code JUST prior to submitting the order. I don't see how I get fills outside the menu. (IE. 4299.525)

This isn't a really big deal. I use my own DB to do evaluation.
The strategy follows the real entry price.
It bothers me because NT says it filled X in the results, but it really didn't fill that price on the chart. Whenever NT doesn't agree with itself, it gives me pause.

Thanks
JohnQ

[NinjaTrader_ChelseaB]

Hello JohnQ,

Just to clarify, the issue you are experiencing is differences between real-time orders and historical orders, is this correct?

Or are you doing a backtest and the trades are changing the next time you do a backtest?

If the differences are between real-time and historical, then you need a 1 tick secondary series and not a 3 second series as 3 seconds can make a world of difference. (Thats 3 seconds of tick data missing for accurate fill prices. Even with 3 seconds I would expect there to be differences between real-time and historical. With 1 tick granularity I would expect much closer results.)

I suggest you add prints to your code. Print the values being used in the conditions and see if any of these are changing.

Then print the price that the order is placed at (or preferable turn TraceOrders = true; on).

Are there differences with the values used in the conditions?
Is the submission price of the order the same?
Is the submission time of the order the same?

[JohnQ]

ChelseaB,
Did you read my previous post?
Don't take this the wrong way but I am getting the vibe you didn't. I was left with the same "why do I bother" feeling when I had a go-round with support via email, regarding "Count.Bars vs. Count" some time back.

If you don't have the time to read what someone posts/emails, how could the response you give be other than confusing to that person?

Been here for years, written 1000's of lines of code. I have like 10 posts.
Not more than 1/2 dozen support email tick #'s in my email history.
I won't be a pest, you are clearly busy....

It's OK, got it working, nothing to fix here, others need responses too....
Thanks for the responses to my question/s...

Best regards,
Sean

[NinjaTrader_ChelseaB]

Hello JohnQ,

I am responding to the following statement:
"If I am plotting chart (and say 1:30 into a 5min bar), hit F5 (or equivalent) historical trades shown will be messed up (don't happen at the right time, MarketPostion tracking is off). If I do the same thing, during a time outside the session window, everything works correctly."

This I am understanding as orders were placed, the script is reloaded, the orders are different.

Is this not the case? When you mention this, are you trying to say that the orders are the same after reloading?

I can see that you have rounded to tick size, but this shouldn't be necessary if the data is the same. Prints in your code will tell you if the data is exactly the same and if it is being placed at the same time and price.

I'd like to at least exactly clarify the exact behavior you are seeing. For example. My strategy placed an order at 3:00 PM. After reloading the historical trade is placed at 3:01 PM. This is an exact issue that I can look into.

However, I am having some difficulty understanding exactly what the issue is from your description.

You mention "I wish it were possible to explain better "exactly" what happens." I am also trying to understand exactly what is happening so that I may assist.

What is the exact issue that you are experiencing?

Do you have prints that I can use to see what is different before and after? Actual output numbers we can compare?