Problem with MarketPosition.Flat in Strategy Analyser

[nemnem]

Hello,

I have a Problem I can't overcome, but first let me make clear that everything is working perfectly fine as long "Exit on Session Close" is activated. Its also how my Strategy was built as I didn't have intention to keep the contracts overnight/weekend. However, I'm looking to include a Performance Metric, and so I needed to keep the trades open, and I deactivated the "Exit on Session close".

Now to the Problem, if I let the Analyzer run it will not use the whole time range (1 April to today) but stop mostly in the first 2-3 Months. First it looked like there would be a problem with the rollover, but I can move the Date forward to any Date, and it will have the same behavior to stop at a Random Spot. As you can guess I thought first I have a problem in my Script and turned on my debugging output and I was very surprised to see that, if I click in the analyzer one series after another, in the Output there is everything as it should be! It shows that there is being traded after the last Trade the Analyzer shows/calculates.

In my Strategy I work a lot with "Position.MarketPosition == MarketPosition.Flat", also in my Buy/Sell Logic function to prevent to be recalled if the Trade is already open.
I started to deactivate everything, in my entry logic, so that in the end there was just something like "if MA up" go long and "if MA down" go short and if "Position.MarketPosition == MarketPosition.Flat", even I thought I'm crazy I commented out this last line and there you go, it calculates all trades in the Analyzer as with "Exit on Session close".

I tried to make my own "intrade" variable with OnExecutionUpdate, as I was in hope it maybe uses a other logic but of course this is not any help. I also can't imagine what could prevent the
Analyzer to do the Job as it should, then with "Exit on Session close".it works fine with Trades going up to 2000-3000 Trades a Series. And from other side i can't work without "Position.MarketPosition == MarketPosition.Flat"as it messes up my logic in other functions and also from a performance point of view. Or is there something else to use?

Any Ideas?

Nemnem

[NinjaTrader_Kate]

Hello Nemnem,

Thank you for your post.

I've moved this post to the proper forum for discussion of a strategy you've programmed yourself.

Generally if the Strategy Analyzer ceases functioning part way through a run, we'd see some kind of error reported. Do you see any error in the log tab of the Control Center in regards to the strategy?

Do you have the latest version of NinjaTrader installed? The version number can be found under Help > About. The current version is 8.0.23.2. If you are not on the current version, please update and run your test again.

To update NinjaTrader, please follow the steps below:

• First, copy your license key from NinjaTrader under Help> License Key then exit NinjaTrader
• Click on the link: https://ninjatrader.com/PlatformDirect
• Enter your license key and press Submit
• Select 'Download'
• Critical: Before running the installer, ensure NinjaTrader is closed.

If you simplify the script so it only enters in one direction and any user inputs are hardcoded, do you see the same?

Thanks in advance; I look forward to assisting you further.

[nemnem]

Hello Kate,

I've moved this post to the proper forum for discussion of a strategy you've programmed yourself.

Sorry, yes that was my fault

Generally if the Strategy Analyzer ceases functioning part way through a run, we'd see some kind of error reported. Do you see any error in the log tab of the Control Center in regards to the strategy?

no I do not have any Error in the Logs and as I said I can actually clearly see if I activate debug that all trades are calculated. So for example if there is a Solution with "Exit on Session End" activated that produces 70 Trades then, without "Exit on Session End", in Output window it shows me 70 Trades but the same line in Analyzer Stops to "add on" after 7.

Do you have the latest version of NinjaTrader installed? The version number can be found under Help > About. The current version is 8.0.23.2. If you are not on the current version, please update and run your test again.

I actually run 8.0.21.1, so I'll make a update tomorrow to see if this will fix it.

If you simplify the script so it only enters in one direction and any user inputs are hardcoded, do you see the same?

I see your point,my script is very large indeed, but I created it that everything is in his space and all strings tie together at one point. So I'm able to reduce the entry and exit logic as I wish from
"Rocket" to "Stoneage" and the only line where I get different result is if"Position.MarketPosition == MarketPosition.Flat" is removed. Also to finish your question I trade always just one way per Stratgy/solution but it does the same if its just going Long or Short.

But I can look into to create a simple script to recreate the error on such a simple script that I can post it for download.

Greetings

Adrian

[NinjaTrader_Kate]

Hello Nemnem,

Thank you for your reply.

Great, I would definitely do the update as there's been some updates to the Strategy Analyzer since the version you're running.

Are you only seeing this occur in the Strategy Analyzer or do you also see it occur when added to a chart running on the Sim101 account?

Thanks in advance; I look forward to assisting you further.

[nemnem]

Hey Kate,

Great, I would definitely do the update as there's been some updates to the Strategy Analyzer since the version you're running.

Unfortunatley this wasn't any help, the problem was still there after the update

Are you only seeing this occur in the Strategy Analyzer or do you also see it occur when added to a chart running on the Sim101 account?

Not so far, but I'm still on this problem, I just wanted to drop a update that I still work on this and also made some progress but I'll update you as soon I can understand
better whats going on.

Greetings

[nemnem]

Hey Kate, I actually made some progress on this problem and I want to share the results with you guys as I'm a bit in a death corner.

I made a copy of the Original Script and started to throw a print() in every corner to see what's going on, I also started to remove the functionality of some modules until I get the wrongdoer.

First a Picture from my Analyzer Result to get started....

https://ibb.co/tK4LXCw

As you can see on the top there is going everything smooth and the only difference in this two run is the checked or unchecked "Close on Session Exit"

Now a bit of history how I got there, I started to remove the Exit Modules and had already a first Result as I deleted my "Trailstop" Exit that you can see on the next picture on
the left side...

https://ibb.co/tYyFPds

of course normally the "ExitLong" / "ExitShort" is not commented out but if its in place it will already stop in this case in the first Trade but in other series there were stops on 2 or 3 but also after 17 or 70... anyway, if I comment it out it will actually stop at another Exit, in this case it was my "Panik" Exit, that doesn't happens so often, so I decided to keep it this way.

On the left side you can see what "Panik" is doing, it is actually a prevention, if a certain Maxloss a day is reached the "stilltrade" variable is not set anymore to 1, and he will not continue trading.
Of course this is just happening if the Script is working as it should, what it's not right now, but I'll get to that in a minute.

But if I would comment out this ExitLong/ExitShort in the Panik Module the Analyzer is running with or without "Exit on Session Close" to the end so as it should!

Next Step...

I actually did what you suggested and hardcoded the entry to the most simple thing I have in my script, its a indicator loop that works very simple if "min1Dloop" is 1 its looping up, if -1 its looping down, if 0 its not doing either of both. Now here is a picture of the hardcoded Entry and also where the Bug happens...

https://ibb.co/qrscmRV

The Green rectangle represents this "Panik" Exit Trade, To Guide you through this, the Buy/Sell Decision Module is normally very complex but it's striped down to the core, as you can see there is also the "Stilltrade" variable from the Panik Module but its commented and has no effect on the entry in this case. We also can see from Minute 22:00 that the loop is -1 and it makes what is should and activates "golongshort to -1"

In the Execution Module is also everything going well, I entered a print for the 3 entry conditions that you can see in the next line with the "0 0 -1". now EnterShort is executed and we can see this
with the OnExecutionUpdate that is always Printing this "Change in OnExecutionUpdate" once for the going Short and the Second for Exiting the trade. On the bottom you can see my debug that is showing me that we're in the module and the entry and stop price.

Normally of course it wouldn't allow the script to trade like this there would be preventions for going right in the trade again after we just exited it, but this is all deactivated.

In the Red Box we see that he wants to trade again and does exactly the same again the only difference we can see is that OnExecutionUpdate is not triggered and so also no trade is happening. As MarketPosition is Still Flat he is now of course always looping in this Execution function and now he tries to enter every minute of the trade.

Here I have now to say I was actually wrong with what I assumed is happening, then you maybe remember I told you I could see the debug lines entering every trade, but I couldn't see the
trades being also executed.

But this behavior is now happening with or also without "Exit on Session Close" I also don't know why he is not entering the trade again. Is there a other debug level I can activate that tells me why he is not entering the trade?

But let's continue where the difference is, in the next picture... it's the last one...

https://ibb.co/cLQjh1h

As you can see this behavior is happening on both sides, but after the day is over and some days later the loop is going to be again -1 on the right side with "exit on Session End" is behaving normally and goes on with the trades. However on the left side he is also doing everything right as he wants to trade but stuck in this loop, without actually to trade.

Now my wild guess is that in this "Exit on Session End" has to be something that is actually resting something so that the Script is then going to continue with a, lets call it "fresh" day.
And without that he stays stuck in this odd routine to not Triggering the EntryShort? So now the 100 Million question is why is he not trading, and how can I find out?

Thanks for the patience

[NinjaTrader_Kate]

Hello nemnem,

Thank you for your reply.

In your Output I'm seeing your prints but I'm not seeing you have the TraceOrders function turned on. The first thing I would recommend would be to turn on the Order Trace function to see if orders are being ignored after a certain point:

Strategy Builder > Default Properties > More Properties > Trace Orders, or:

if (State == State.SetDefaults)
{
TraceOrders = true;
}

Once you then recompile the strategy, you can open a new NinjaScript Output window under New > NinjaScript Output. This will print a log of any orders submitted by the strategy during while it's running, along with any ignored orders. You can then look through and see what may be occurring.

Here is a link to our help guide that goes into more detail on tracing orders:

https://ninjatrader.com/support/help...aceorders2.htm

If you turn this on and repeat your test, do you see any notations about orders being ignored?

Thanks in advance; I look forward to assisting you further.

[nemnem]

ok found the wrongdoer

30.07.2020 22:31:00 Strategy 'testing2/-1': Entered internal SubmitOrderManaged() method at 30.07.2020 22:31:00: BarsInProgress=1 Action=BuyToCover OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Panik_123' FromEntrySignal=''
30.07.2020 22:31:00 Strategy 'testing2/-1': Entered internal SubmitOrderManaged() method at 30.07.2020 22:31:00: BarsInProgress=1 Action=BuyToCover OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='InitStop_' FromEntrySignal=''
30.07.2020 22:31:00 Strategy 'testing2/-1: Cancelled pending exit order, since associated position is closed, orderId='NT-00236-41741' account='Backtest' name='InitStop_' orderState=Working instrument='NQ 03-21' orderAction=BuyToCover orderType='Market' limitPrice=0 stopPrice=0 quantity=1 tif=Gtc oco='' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2020-07-30 22:31:00' gtd='2099-12-01' statementDate='2021-02-11'

So if I see this right its getting jammed as it gets two Exit Order in the Same BarsInProgress function.

[NinjaTrader_Kate]

Hello nemnem,

Thank you for your reply.

This is pretty much what I was expecting. A historical overfill whereby 3 orders (for example, entry/target/stop) filled on the same bar causes the Strategy Analyzer to basically freeze up. The root of this problem is with how Calculate.OnBarClose handles backtest/historical fills since it would only know the finished bar (not how it was built). Basically your logic is such that two exit orders are sent for the same bar, it then tries to cancel one and can't and just grinds to a halt. I'd suggest trying a few things. First, you could use a bool that's set if one has been submitted and check that before submitting the other exit, and reset the bool on the next bar. You could also try running it using Order Fill Resolution: High, as that should allow the strategy better granularity and hopefully avoid the overfill scenario.

Please let us know if we may be of further assistance to you.

[nemnem]

As every exit has his if (Position.MarketPosition == MarketPosition.Long ) or short I never thought this could be a problem but looks like I need to make a bypass variable if once Exit was triggered to preven another Exit to happen.in the same BarsInProgress.

[nemnem]

Thanks a lot Kate, you didn't know how long I was on this and I normally don't make a fuss and just run in the forum, I tried hard to get it solved but was so stuck for days.

Thanks for the Patience

Adrian