Strategy position not updating on live account

[agrock]

I'm facing a weird behavior with a managed strategy. Sometimes the position doesn't get updated as required. Look at the example below:

* The exit condition was fullfilled

if (ToTime(this.Time[0]) >= this.ExitTime)
{
    if (this.Position.MarketPosition == MarketPosition.Long)
        this.ExitLong(this.Contracts);
    else if (this.Position.MarketPosition == MarketPosition.Short)
        this.ExitShort(this.Contracts);
}

https://www.dropbox.com/s/itwz0ecipicw34p/1.png?dl=0

* Empty positions tab, as expected

https://www.dropbox.com/s/el9kxt4eyv4h1o4/2.png?dl=0

* But my strategy is still shown as holding 1L

https://www.dropbox.com/s/dlwybdpcw1plb7y/3.png?dl=0

* Orders tab for reference

https://www.dropbox.com/s/968mbwcx24c52a7/4.png?dl=0

Once it happens, my strategy will not enter again, since I have a MarketPosition == Flat check before entering a new trade.

Considerations:
* I'm using NT 8.0.14.2 (64 bits).
* I'm just using managed order methods.
* It just happens with my live connection (Rithmic). I don't have this problem with backtesting or market replay.
* It doesn't happen all time, and (to date) just after a Exit method (market order) or a stop loss fill (stop order).

Attached below all the strategy code that I can share with you:

namespace NinjaTrader.NinjaScript.Strategies
{
    public class MyStrategy : Strategy
    {
        #region Fields

        private Order entryOrderLong;
        private Order entryOrderShort;

        #endregion

        #region Properties

        [Range(1, 3), NinjaScriptProperty]
        public int Contracts { get; set; }

        [Range(110000, 160000), NinjaScriptProperty]
        public double ExitTime { get; set; }

        [Range(1, int.MaxValue), NinjaScriptProperty]
        public int StopTicks { get; set; }

        [Range(1, int.MaxValue), NinjaScriptProperty]
        public int TargetTicks { get; set; }

        #endregion

        protected override void OnBarUpdate ()
        {
            int volMultiplier = <some multiplier>;

            if (<some entry long condition> && this.Position.MarketPosition == MarketPosition.Flat)
            {
                this.SetProfitTarget(CalculationMode.Ticks, this.TargetTicks * volMultiplier);
                this.SetStopLoss(CalculationMode.Ticks, this.StopTicks * volMultiplier);
				
                this.entryOrderLong = this.EnterLongLimit(this.Contracts, <some limit price>);
            }

            if (<some entry short condition> && this.Position.MarketPosition == MarketPosition.Flat)
            {
                this.SetProfitTarget(CalculationMode.Ticks, this.TargetTicks * volMultiplier);
                this.SetStopLoss(CalculationMode.Ticks, this.StopTicks * volMultiplier);
				
                this.entryOrderShort = this.EnterShortLimit(this.Contracts, <some limit price>);
            }

            if (ToTime(this.Time[0]) >= this.ExitTime)
            {
                if (this.Position.MarketPosition == MarketPosition.Long)
                    this.ExitLong(this.Contracts);
                else if (this.Position.MarketPosition == MarketPosition.Short)
                    this.ExitShort(this.Contracts);
            }
        }
		
		protected override void OnOrderUpdate (Order order, double limitPrice, double stopPrice, int quantity, int filled, double averageFillPrice, OrderState orderState, DateTime time, ErrorCode error, string comment)
        {
            if (order.OrderState == OrderState.Filled)
            {
                if (order == this.entryOrderLong)
                {
                    if (this.entryOrderShort != null) this.CancelOrder(this.entryOrderShort);
                    return;
                }

                if (order == this.entryOrderShort)
                {
                    if (this.entryOrderLong != null) this.CancelOrder(this.entryOrderLong);
                    return;
                }
            }
        }

        protected override void OnStateChange ()
        {
            if (this.State == State.SetDefaults)
            {
                this.Calculate = Calculate.OnBarClose;
		<initialization of other variables>
            }
        }
    }
}

[NinjaTrader_Jim]

Welcome to the forums agrock!

Market Replay data will mimic realtime data and fill orders using NinjaTrader's built in simulation engine. If you can encounter this issue using a live connection and a NinjaTrader Sim account, you should be able to reproduce the occurrence with Market Replay data.

Another item that should be noted is that the strategy position is not the same as the account position. The strategy may come up with a different strategy position based on the historical data it processes.

I would suggest to test with the SampleOnOrderUpdate strategy which also takes an approach to using Order objects to manage the strategies position. If you do not see an issue with order/position handling with this strategy, then there would appear to be an issue with your strategies implementation. If SampleOnOrderUpdate gives ill behavior then there would be a platform issue we could troubleshoot.

I would also suggest to add prints throughout your code to observe:

• Exactly when your order submission methods are reached
• What the strategy position is when encountering your Position checks so you can observe the strategies position at that time

Trace Orders can also be used to monitor the order feedback from your strategy.

Debugging tips and using Trace Orders are linked below

Debugging Tips - https://ninjatrader.com/support/help...script_cod.htm

TraceOrders - https://ninjatrader.com/support/help...aceorders2.htm

SampleOnOrderUpdate - https://ninjatrader.com/support/help...and_onexec.htm

I've also included some reading for Strategy Position vs. Account Position - https://ninjatrader.com/support/help..._account_p.htm

If this information does not help resolve your inquiry, could you share the results of the debugging steps you have taken as well as reducing the strategy to work on 1 side of the market?

I look forward to being of further assistance.

[bltdavid]

* I'm using NT 8.0.14.2 (64 bits).

Have you tried with newer release 8.0.15.1?

[agrock]

Strategy position not updating on live account

Update on this issue:

I did the following:

* Installed latest NT 8 release (8.0.15.1) from zero.
* Added trace orders.
* Added some custom trace messages.
* Replaced OnOrderUpdate for OnExecutionUpdate.
* Now I manually set MarketPosition after a target/stop fill.

Look below the updated code:

protected override void OnBarUpdate ()
{
	[B]this.Print(string.Format("OnBarUpdate ({0})-> Position: '{1}'.", DateTime.Now, this.Position.MarketPosition));[/B]
	
	int volMultiplier = <some multiplier>;

	if (<some entry long condition> && this.Position.MarketPosition == MarketPosition.Flat)
	{
		this.SetProfitTarget(CalculationMode.Ticks, this.TargetTicks * volMultiplier);
		this.SetStopLoss(CalculationMode.Ticks, this.StopTicks * volMultiplier);
		
		this.entryOrderLong = this.EnterLongLimit(this.Contracts, <some limit price>);
	}

	if (<some entry short condition> && this.Position.MarketPosition == MarketPosition.Flat)
	{
		this.SetProfitTarget(CalculationMode.Ticks, this.TargetTicks * volMultiplier);
		this.SetStopLoss(CalculationMode.Ticks, this.StopTicks * volMultiplier);
		
		this.entryOrderShort = this.EnterShortLimit(this.Contracts, <some limit price>);
	}

	if (ToTime(this.Time[0]) >= this.ExitTime)
	{
		if (this.Position.MarketPosition == MarketPosition.Long)
			this.ExitLong(this.Contracts);
		else if (this.Position.MarketPosition == MarketPosition.Short)
			this.ExitShort(this.Contracts);
	}
}

protected override void OnExecutionUpdate (Execution execution, string executionId, double price, int quantity, MarketPosition marketPosition, string orderId, DateTime time)
{
	Order order = execution.Order;

	if (order.OrderState == OrderState.Filled)
	{
		[B]this.Print(string.Format("OnExecutionUpdate ({0})-> Order '{1}' filled.", DateTime.Now, order.Name));[/B]

		if (order == this.entryOrderLong)
		{
			[B]this.Position.MarketPosition = MarketPosition.Long;[/B]
			if (this.entryOrderShort != null)
			{
				this.CancelOrder(this.entryOrderShort);
			}
			return;
		}

		if (order == this.entryOrderShort)
		{
			[B]this.Position.MarketPosition = MarketPosition.Short;[/B]
			if (this.entryOrderLong != null)
			{
				this.CancelOrder(this.entryOrderLong);
			}
			return;
		}

		if (order.Name == "Profit target" || order.Name == "Stop loss")
		{
			[B]this.Position.MarketPosition = MarketPosition.Flat;[/B]
		}
	}
}

protected override void OnPositionUpdate (Position position, double averagePrice, int quantity, MarketPosition marketPosition)
{
	this.Print(string.Format("OnPositionUpdate ({0})-> Position: '{1}'.", DateTime.Now, this.Position.MarketPosition));
}

Just a minutes ago, running the strategy concurrently on a live connection (Rithmic) and a sim connection (Continuum) gives different output. It seems to be that OnPositionUpdate it's not getting called on the live account after a target/stop fill, look:

Live

https://www.dropbox.com/s/qzjfn6cxdck1pzu/6.png?dl=0

Sim

https://www.dropbox.com/s/srzlzxkdesyc2kt/7.png?dl=0

Manually setting the position seems to work. As you can see on the live image, although OnPositionUpdate is not called, the strategy status is now Flat as expected. But I think that's a very weird behavior that should be checked on the platform side.

This behavior happened also with another strategy, a much simpler one trading just one side of the market. I will give you an example as soon as it happens again.

[bltdavid]

It seems to be that OnPositionUpdate it's not getting called on the live account

I've noticed this in NT7 as well.

Unfortunately, I found that OnPositionUpdate is not guaranteed to run in every situation, which is probably a bug.

I found I could not trust OnPositionUpdate for certain key things, such as updating a database.

I eventually stopped using it and found other ways to watch for changes in the position.

It appears this bug still exists in NT8.

[NinjaTrader_Jim]

Hello agrock,

Thanks for testing and providing that the detail. I was able to see similar issues when testing on a Rithmic paper trading account on my end and I have submitted a report for the behavior for further review.

@bltdavid, I have just seen your post. I will test NT7 as well and pass the information onward.

I'll update this post/thread as more information becomes available.

[bltdavid]

I was able to see similar issues when testing on a Rithmic paper trading account on my end and I have submitted a report for the behavior for further review.

Hi Jim,

By any chance, are you able to test for this behavior on NT7 as well?

[NinjaTrader_Jim]

Hello bltdavid,

I tested with NinjaTrader 7 this morning and I did not see issue on a paper trading account. I've included my test so can see what I have done on my end. I took a similar approach with NinjaTrader 8 to observe the OnPositionUpdate and Position object updating behavior.

Demo - https://drive.google.com/file/d/1nSA...w?usp=drivesdk

If you are encountering some of the behavior agrock reports in this thread with NinjaTrader 7, could you provide an example similar to the one I posted that would clearly demonstrate the issue?

I look forward to being of further assistance with this matter.

[bltdavid]

paper trading account

Define 'paper trading account'.

Are you saying you can reproduce this issue on NT8 in a paper trading account?

But when trying to reproduce in NT7, using that same paper trading account, you are not able to reproduce?

On NT8, can your testcase reproduce the issue 'at will' in a reliable and consistent way? A consistent reproducible testcase is paramount to giving your engineers a golden roadmap to fix it ...

It's been awhile since this happened to me. I will dig up some notes and respond again later in more detail .. in the meantime, from memory ...

The only times I saw this behavior were on live trading accounts involving real money.

I was never able to completely define the reasons for the issue. The only times the problem would manifest would be in a fast moving market with multiple contracts and multiple targets, but never in a reliable predictable way.

Frankly, I never knew how to reproduce it. I just remember discovering that OnPositionUpdate was not 100% guaranteed to be sent to my strategy. If I recall, it was only the 'Flat' position that was periodically missing (but then again, that was the only position change I was really paying attention to).

I know I saw it on my RCG account w/Continuum and I'm pretty sure I saw it on my NT Brokerage account w/Rithmic. Live strategy, live account, real money. Fortunately, I have my own debug logging with varying levels of verbosity and I was eventually able to discover the problem and write a remedy that eliminated OnPositionUpdate as a central player.

Otherwise, I never encountered the issue. Never in strategy analyzer, never using Sim101, and (so far, I think) never in my TopstepTrading combines.

I'll try to put together a small NT7 test strategy that 'catches' when OnPositionUpdate is not called -- similar to what I'm doing now.

[NinjaTrader_Jim]

Hello bltdavid,

By "paper trading account," I mean a demo account with that brokerage technology. This would result in the order, execution and position updates coming from the associated adapter that NinjaTrader uses to connect to that provider. When using a Simulation account on NinjaTrader (Sim101,) order feedback, executions and position updates are provided through NinjaTrader's Simulation engine.

Based off of agrock's information, I was able to create a test case similar to the NT7 one I shared that monitors OnPositionUpdate and the Position object. NinjaTrader 8 was not behaving as I would expect it and I reported the case further on. Our role is very largely to take issue reports and work with you to make a clear case that defines the problem so Development can look into fixing. We have a case for NinjaTrader 8 and that ticket is currently on the QA desk awaiting further review.

I was not able to find a case for NinjaTrader 7 (on the same paper trading account) which is why I shared my test. I did not see issue with OnPositionUpdate or with the Position object with that test. If you encounter something taking a different approach, we'd like to understand it and report it.

I'll continue to provide updates to the NinjaTrader 8 issue as information develops.

[agrock]

Strategy position not updating on live account

An update on this issue. I ported the strategy to NT7 and tested it on the same live account. The OnPositionUpdate method is called as expected, look:

https://www.dropbox.com/s/jonah97x16ttmn1/8.png?dl=0

It's a bug introduced in NT8?

[NinjaTrader_Jim]

Hello everyone,

Thanks for your patience.

The Rithmic API can submit position events ahead of related order and execution update events. As a result, this is a limitation with NinjaTrader when using Rithmic.

At this time, I would suggest using OnExecutionUpdate() to manage the strategy's position with Rithmic Connections.

We will update the OnPositionUpdate() Documentation to mention this limitation for the Rithmic connection.

@gplatis, this issue has to do with our Rithmic Connection reporting position updates that are out-of-order or inconsistent with the actual position. Using the Market Replay or Playback Connections would not have this same issue. If you come across a similar scenario for Market Replay/Playback, could you write in with an example strategy and some steps we could take on our end to platformsupport [at] ninjatrader [dot] com with the text "Attn Jim" so we can look into this further?

I look forward to being of any further assistance.