Strategy works different on real data and historic

[mcosta72]

Hello My Friends,

I am working in a kind of more CPU consuming strategy with CalculateOnBarClose = false; and I am getting very weird results when running in real time data (Simulated Account).
Code sends commands to cancels all open or pending orders when (position.MarketPosition == MarketPosition.Flat) but in the real time data the position is growing like 5 and 10 and 30... until it crashes for limit or order management reasons.
Running in market replay data works with no bugs.

Please give me some light!

Thanks

[NinjaTrader_Jim]

Hello mcosta72,

Thanks for opening the thread.

When you have TraceOrders enabled, are you seeing additional submissions in the Output window when you see your position increase?

When you place prints outside the conditions that control your order methods to print the values of these conditions, are they becoming true and allowing more submissions?

Using prints and commenting sections of your code, can you narrow down the code to part that creates the issue?

Here are some links you may reference on debugging:

Debugging - http://ninjatrader.com/support/forum...ead.php?t=3418

TraceOrders - http://ninjatrader.com/support/forum...ead.php?t=3627

I look forward to being of further help.

[mcosta72]

Hi Jim,
It is always great when you guys reply quickly.
I used the TraceOrders to identify where the orders come from and the results are really strange because it shows lines that would be impossible to reach if the conditions are satisfied.
I go with Unmanaged = true; and this part to cancel Stop or Target when the other one is filled:
if (position.MarketPosition == MarketPosition.Flat)
{
if(Ordem != null && MyStop != null)
{
if(Ordem.OrderState.ToString() == "Filled" && MyStop.OrderState.ToString() == "Filled")
{
if(MyTarget != null)
{
if(MyTarget.OrderState.ToString() == "Accepted" || MyTarget.OrderState.ToString() == "Working")
{
CancelOrder(MyTarget);
MyTarget = null;
}
}
Ordem = null;
MyStop = null;
}
}
// TARGET
if (MyTarget != null && MyTarget.OrderState.ToString() == "Filled")
{
if(MyStop.OrderState.ToString() == "Accepted" || MyStop.OrderState.ToString() == "Working")
{
CancelOrder(MyStop);
MyStop = null;
MyTarget = null;
Ordem = null;
}
}
}

So, if the Position is flat, Order = null, no reason to fall into a condition where:
if (Ordem != null && Ordem.OrderState.ToString() == "Filled" && Ordem.OrderAction.ToString() == "Buy")

I am using Ordem as my Order. I thought it could be if it has to finish all instances and commands inside OnBarUpdate() in order to check if the Position is flat. Any chance?
Thank you!

[NinjaTrader_Jim]

Hello mcosta72,

Conditions that do not seem possible should not always assumed to be impossible. I would still advise to add prints to verify the logic that is allowing these actions to take place. This will give a complete picture of the running logic and will make it easier to devise a solution.

IOrder objects won't get their assignment within the same iteration of OnBarUpdate(). We advise to use OnOrderUpdate() for IOrder object assignment and handling. I would also advise to assign and nullify orders in OnOrderUpdate() instead of cancelling them and then setting to null in OnBarUpdate().

I have also attached a copy of an Unmanaged strategy that can be used as a reference for order placement with the Unmanaged Approach.

OnOrderUpdate() - https://ninjatrader.com/support/help...rderupdate.htm

Please let me know if I may be of further assistance.

[mcosta72]

Your document was very useful to explore OCO with OnExecution() functions.
Great help, thanks

[pstrusi]

IOrder objects won't get their assignment within the same iteration of OnBarUpdate(). We advise to use OnOrderUpdate() for IOrder object assignment and handling. I would also advise to assign and nullify orders in OnOrderUpdate() instead of cancelling them and then setting to null in OnBarUpdate().

Let's say, that unexpectedly an Order's State by unknown reason, "hanged up " in one of the supposedly transitional states, such as: Initialized, Submitted, TriggerPending...etc; since that order doesn't change at all any longer because the issue, I think OnOrderUpdate won't be called and it left us in stranded situation. So I have some doubts:

1. In this weird situation, is it possible to check the Order's State within OnBarUpdate() ?

2. If we confirmed that the Order is stranded, fix, how to reset it ?

Thanks

[NinjaTrader_Jim]

Hello pstrusi,

OnOrderUpdate() will tell you the last state of the order as it updates. You will have the last order state when it was received and you can see if certain amount of time passes after you receive that update to take an action you desire.

As the order resides at the exchange and its state is simply reported back to NinjaTrader, I would not expect an order to get stuck in a transitional state. You should call the Orders Desk of your broker if you find your self in such a live scenario that you are uncertain of.

Please let me know if you have any additional questions.

[pstrusi]

OnOrderUpdate() will tell you the last state of the order as it updates. You will have the last order state when it was received and you can see if certain amount of time passes after you receive that update to take an action you desire.

Let's say that you begin to measure time from the very last State, let's say "Initialized", but it reaches the limit and still the Order's State is stranded as "Initialized", what kind of actions could you do Out of the Method OnOrderUpdate() ?, because it won't be called any longer ? or is there any action that might take in OnBarUpdate() on this Order ?

As the order resides at the exchange and its state is simply reported back to NinjaTrader, I would not expect an order to get stuck in a transitional state. You should call the Orders Desk of your broker if you find your self in such a live scenario that you are uncertain of.

As you've previously cited an intelligent quote: "Conditions that do not seem possible should not always assumed to be impossible", so, let's say that the Order's State hanged as "initialized", it means that it didn't reach the exchange, just got stuck in NT, and I've been able to confirm this, would it be a possible solution to reset this Order to Null or there's nothing to but to disable and enable the Strategy ?

Thanks

[NinjaTrader_Jim]

Hello pstrusi,

If an order reaches Initialized but never reaches Submitted, please reach out to us as this would mean NinjaTrader did not submit your order to the broker.

You can call CancelOrder() and then set the Order/IOrder object to null when the canceled order receives a state of OrderState.Cancelled in OnOrderUpdate().

If this order never reached the exchange, I would not expect the exchange to return a valid cancellation as it never received the order to cancel.

If you ever come across this in a live scenario, please let us know as this would imply something is not happening on the platform end which should be expected.

[pstrusi]

....
If you ever come across this in a live scenario, please let us know as this would imply something is not happening on the platform end which should be expected.

Well, I'm suspicious that I might have been dealing with an improbable scenario like this, dealing with "Rejected orders in NT8". To make a long history short, it was the simply scenario than after a rejection, you should simply detect that state and reset the order to Null. Despite this logic is well configured in the Strategy, it fails that "Reset" because despite of it, it returns again to non-null for unknown reason, and since I've set previously to "RealtimeErrorHandling.IgnoreAllErrors" , the Strategy continues to be alive but stranded with an opened position. I couldn't understand why if the Order State is reset to Null, and the Strategy continues to work triggering buy and sell signals normally ( confirmed seeing the Output window ) new orders didn't get submitted any more. I imagined that something happens to the order's state that prevented new submits. So, reviewing the log for errors last round, I've found this strict order of events:

1. The initial rejected order error

Order='8d18bb30675b4c60914d574db2c97265/Sim101' Name='LONG' New state='Rejected' Instrument='EURUSD' Action='Buy' Limit price=1.17366 Stop price=0 Quantity=200,000 Type='Limit' Time in force=GTC Oco='' Filled=0 Fill price=0 Error='Order rejected' Native error='Limit price can't be greater than current ask.'
Here the Strategy set that order.state from rejected to null

2. Then, the error message from system :

Sim101, Limit price can't be greater than current ask. affected Order: Buy 200000 Limit @ 1.17366

3. And finally, this message that it seems out of place because it had to be the first one.

Order='8d18bb30675b4c60914d574db2c97265/Sim101' Name='LONG' New state='Submitted' Instrument='EURUSD' Action='Buy' Limit price=1.17366 Stop price=0 Quantity=200,000 Type='Limit' Time in force=GTC Oco='' Filled=0 Fill price=0 Error='No error' Native error=''

This perhaps might explain, why despite the logic set the order.state to null, some unknown reason makes the order state to be "submitted" indefinitely.

I tried this in a long post but so far it's been an exhausting procedure with no solution.

https://ninjatrader.com/support/forum/showthread.php?t=104538

So, I really hope that this situation may be solved. I wish it was a custom error in the logic but I'm pretty sure it's not the case.

Thanks in advance

[NinjaTrader_Jim]

Hello pstrusi,

It seems that my colleagues Alan and Jesse are working with you on these issues at this time.

It sounds like two separate issues for why the rejection is allowed in Sim, and why you are seeing the order submission after the fact.

As far as I see, Alan has been looking into why the rejection is occurs in Sim and it is not yet conclusive if paper trading accounts allow this when actual live accounts do not. Once that is determined, a decision can be made to modify the Simulator. As you have brought this up with Alan, he will be your point of contact on that matter.

Jesse is still working with you on the order submission that occurs after the fact. I believe he was looking for a reduced sample and steps where he could reproduce the behavior you are seeing on his end to provide further input.

In the future, please keep these inquiries with the technician you are working with and associated thread. We ask this because that technician will have a better understanding of the issue at hand and it assures we can efficiently assist our other clients.

[pstrusi]

Thanks Jim, I sincerely appreciate the help.

Regards