multi-instrument strategy failed to execute

[zstheorist]

I have just purchased a subscription in order to forward test my strategy with my IB account. I had a strange bug yesterday in one of the strategies I was testing.
I have a strategy that executes a buy based on a different instrument. Eg.
In Initialize() I have
Add("MSFT",PeriodType.Minute,1);
In OnBarUpdate() I have
if (CrossAbove(SMA(Closes[1],10), SMA(Closes[1],20), 1))
{
EnterLong(DefaultQuantity, "");
}
[There are similar statements for shorts, etc. I've confirmed details do not matter.]
In weeks of backtesting and live on Monday and Wensday this executed fine applied to a chart of YM.
Yesterday (Tuesday) the strategy never placed an order. I confirmed via the NT 6.5.1000.4 logs and IB logs that the strategy started, but then never did anything the entire day.
What was different yesterday? The only difference was that I did not have a separate chart of MSFT open in the background to collect data.
This looks like there is a bug in that data is not reaching my live strategy unless there is a different window open. The YM chart did claim to load MSFT when I started the strategy, so it knew it needed to, but it clearly did not receive the MSFT after that. The plotted away normally.
I am a bit concerned about applying this to real money until the origin of this anomaly is tracked down. A temporary fix is to just open another chart for the second instrument, but I want to be certain this is nothing deeper.

[NinjaTrader_JoshP]

To confirm please try to print data values from both bar series. Add this to your code:

if (BarsInProgress == 0)
     Print(Time[0] + " Primary Bar OHLC: " + Open[0] + " " + High[0] + " " + Low[0] + " " + Close[0]);
if (BarsInProgress == 1)
     Print(Time[0] + " MSFT Bar OHLC: " + Open[0] + " " + High[0] + " " + Low[0] + " " + Close[0]);

If your strategy is not receiving MSFT you will never print the MSFT lines.

[zstheorist]

Thanks. I'll try that.
I do not think I mentioned it before, but a backtest after the fact triggered just as I expected. I'll see what I can track down when the markets open Monday on a non-essential trade.

[NinjaTrader_JoshP]

Hi,

I am not sure what you mean by "backtest after the fact triggered". Could you please explain? Thanks.

[zstheorist]

update

Hi,

I am not sure what you mean by "backtest after the fact triggered". Could you please explain? Thanks.

Sorry, I was in the hospital earlier this week.
1. I tried printing prices (as suggested above) and they come through to the output window, so I really cannot figure out what happened.

2. What I meant was about backtesting was: after the day was over, I opened charts for all instruments I track to make certain the data was there. After disconnecting from IB I ran my strategy through a backtest including the day tht just occurred and the backtest had a simulated trade opening right when I expected and closing when I expected (for a loss it happens).

So running real-time that day, something got confused and no orders were generated for this strategy only, but it worked the day before and teh day after when I had the "MSFT" open. Since I was sick this week nothing ran real time. For now I will just assume this was a strange fluke - maybe a data glitch or something - and try running for real again next week.

Thank you for the help.

[NinjaTrader_JoshP]

Differences in fills are common for a number of reasons. Please see this article about the discrepancies between real-time and backtesting. You cannot expect the two to be the same. Especially if your strategy utilizes CalculateOnBarClose set to false.

http://www.ninjatrader-support.com/HelpGuideV6/DiscrepanciesReal-TimeVsBacktest.html

[zstheorist]

Differences in fills are common for a number of reasons. Please see this article about the discrepancies between real-time and backtesting. You cannot expect the two to be the same. Especially if your strategy utilizes CalculateOnBarClose set to false.

http://www.ninjatrader-support.com/H...sBacktest.html

OK, that is completely unhelpful and irrelevant. I am having a clear software failure now. The ONLY legitimate reason that NO ORDER would have been placed was that my data feed disappeared briefly. Since I did not have the other chart open, that was a possibility. No longer.

Today I lost money because 2 trades never executed. No order was ever placed. When I started this in the morning the strategy was running on a chart of ES with a executions based on crossovers of SPY.
The charts, the logs, and the trace file have absolutely no record of anything from my strategy beyond it starting up and shutting down at the end of the RTH day.
I am 100% certain that the crossovers occurred. And, since all my code says is if a crossover occurs, then enter a market order; there simply is nothing here to not execute.
I am down to 3 possible causes I can imagine:
1. My account at NT has been disabled for some reason.
2. NinjaTrader itself has a serious bug.
3. CrossAbove and CrossBelow have bugs.
Let's examine 3. On a minute chart of SPY, there were a series of prices returned by the the SMA's I was examining today. Over the relevant 3-minute period: SMA1 = 123.61 123.60 123.59
SMA2 = 123.60 123.60 123.60
According to direct tests (including mocking up an indicator to color the background when CrossBelow fires), this should have triggered when SMA1 returned 123.59.

What am I supposed to do from here? The strategy ALWAYS works in backtests, and WAS working sometimes (2/4 days) when connected to IB.

[NinjaTrader_Ray]

1. This is not a technical possibility
2. This is always possible but we would need a reproducible scenario plus elimination of any user logic flaw
3.What you do from here is debug. Add Print() statements to locations where order methods are being called. Enabled TraceOrders property to true and cross reference when a print statement confirmed that an order entry method was called. You should see a print statemen along with a trace that an order method was called. If it was called but ignored, a message describing why it was ignored would be logged.

[zstheorist]

OK, this is long, but I've tracked down 2 issues - that may be related:
First, I have always operated with CalculateOnBarClose=true (this may be significant in the failures). Second, the ONLY code in OnBarUpdate is the modified section of code (I increased lookback to 3 bars):
if (BarsInProgress == 0) {
if (CrossAbove(SMA(Closes[1],Period1), SMA(Closes[1],Period2), 3))
{
Print("EnterLong:"+ToTime(Time[0])+";"+Close[0]+";"+SMA(Closes[1],Period1)[0]+">"+SMA(Closes[1],Period2)[0]);
EnterLong(DefaultQuantity, "");
}
}
and an identically inverse one for shorts using CrossBelow.
Bug #1: Looking at the trace in the output window I should see CrossAbove trigger true 3 times, and yield 3 print statements. On Wednesday my trades did execute (incorrectly), but only 2 statements appeared for the first long. This is not possible unless something is wrong. Is it CrossAbove? I have a theory it is actually related to Bug #2 below.
Bug #2: A logic bug in NinjaTrader
Much more serious is that my trades executed 1 bar late costing me 2 ES points.
My first trade was a long entry off of the CrossAbove which occurred in real time at the close of the 9:56 bar on SPY. The trade did not trigger until the close of the 9:57 bar of ES(submitted at 9:58:01 - i.e. near open of 9:58 bar).
What did I expect? I expect historial testing and reality to follow the same logic. In historical testing (at the E.O.D. on the same data recorded in real-time) the order would have been placed at the open of the 9:57 bar (i.e. 9:57:01 or so). Looking at my raw data, this is the expected behavior in real-time.
What went wrong? I have a theory. In historical backtests, all data is sitting on disk, and when BarsInProgress==0, the bar for the 2nd security is by definition closed. In real time, either by design or what I fear _randomly_, NinjaTrader may not recognize that the bar for the 2nd security is closed, and so is still working off of data that is off by 1 bar. This would certainly exhibit the behavior I have seen (and even explain Bug #1 above if by the 3rd bar things were re-synchronized).
If this is correct, then there is a design flaw in at least SOME part of NT. There should NEVER be random chance to see the data. Either OnBarUpdate should NEVER run until all bars are closed, or it should NEVER have access to the 2nd securities current bar -- INCLUDING in backtests.
Whatever the cause, the actual behavior needs to be documented clearly in anything written about multi-instrument strategies.
Possible work-around #3: Please let me know if the following should bypass any concerns raised above. Since orders are triggered off of the 2nd instrument, modify the code to be:
if (BarsInProgress == 1) {
// work at close of 2nd instrument bar
if (CrossAbove(SMA(Close,Period1), SMA(Close,Period2), 1))
{
EnterLong(0,DefaultQuantity, "");
}
}
This would guarantee the 2nd instrument bar was closed. Does the order get sent immediately?
Do I need to invoke full Advanced Order Handling to do this? i.e. keep track of IOrder objects, etc? or should this work on its own? I may move to AOH sometime anyway, but I was hoping to get a vanilla strategy correctly working first.
Please feel free to give a technical answer. I have extensive experience in scientific applications and algorithms.

[zstheorist]

Quick addition. The reason I suspect that access to the 2nd instument's completed bar may be random is that later in the day I had a short where all 3 print statements appeared in the output window, but the trade was still 1 bar late. This leads me to suspect that the 3rd bar for my original long saw the close of the 2nd instrument and decided the 3rd bar had passed, but when the short occurred it was just delayed by a bar.
This 1-bar offset/is the bar closed issue could also explain why I was missing trades when I only looked back 1 bar in my original post.

[NinjaTrader_JoshP]

Why would CrossAbove print 3 times? It will only print 3 times if the condition was met 3 times. Using a lookback of 3 just means you are checking if there was a cross 3 bars ago. If there was it will only fire off once.

When you use CalculateOnBarClose your trades always enter 1 bar after the signal bar. The only reason you got a signal from your signal bar is because that bar already closed. Since that bar is closed it is impossible to enter any position on that bar anymore so now you have to enter on the next bar.

Historical testing and realtime testing are never the same. Please understand that once the bar is closed (which is how all bars are in historical testing) you can't interact with that bar anymore. Your signals are all coming from closed bars. You can only trade on the next available bar. Thus if a cross happend on bar x you can only trade on bar x+1.

[zstheorist]

identified possible cause - how to avoid?

I have waited to reply until I had more information. I believe I have tracked down the cause of the strange entries. The question is what can I do to prevent this?

1. In response to Josh: Actually CrossAbove with a lookback of 3 correctly evaluates to true _3_ times beginning with the first bar that crosses (usually). This is both the documented and real time behavior. The successive orders are skipped because I limit entries to 1 per direction.

2. After printing out the world I have come to realize the failure is not stemming directly from CrossAbove, but from the values returned by the SMAs.

To keep it short, when calling CrossAbove(SMA(Closes[1],10), SMA(Closes[1],20), 3) from within the strategy applied to ES, the SMA's evaluate to different numbers than when calling CrossAbove(SMA(Close,10), SMA(Close,20), 3) applied directly to SPY.

An example from real-time trading (both evaluated on bar close with longer
periods):
in ES strat: SMA1_SPY=126.115633333332; SMA2_SPY=125.880506666667
in SPY strat: SMA1_SPY=126.027955555556; SMA2_SPY=126.032066666668
At no time on these charts did the numbers agree.

SPY chart says SMA cross should have triggered. ES cross does not happen for _4_ more bars... costing me 2 more ES points.

Where did the discrepancy come from? I have identified a clue: The real-time recorded charts and logs show that NT lost its connection to the IB data feed for a minute or two prior to the crossover.

The essential questions: 1) Why did it affect the two charts differently.

2) I am storing real-time data bars in Options. If ES is somehow requesting historical bars, and SPY is recording live, I can imagine that the numbers would differ. If I turn off store real-time bars, will this force the strategy to request data from IB before executing? It would introduce a delay. Would it backfill the missing minute before calling OnBarUpdate?

Figuring out how to fix my datafeed is an separate issue, but for now I am
trying to understand how to force NT to reconcile the prior data before
executing. I presume this would involve setting a flag if
ConnectionStatus.ConnectionLost occurred, and not allowing trading logic to execute until a data update request is fullfilled. How do I ask NT to reload data from within a strategy?

[NinjaTrader_JoshP]

I recommend you first strip down your prints into just printing out Time[0] for each instrument. This way you can see exactly the order the OnBarUpdate() handles the different bar series.

1. Don't know what you mean by affect the two charts differently. If you lost your data connection that is it for all your charts. Wherever your stream ends is where it ends.

2. Storing real-time bars and requesting historical bars are independent of each other. Once your strategy is done loading up it will proceed in updating the OnBarUpdate() of each instrument as data flows in. This data is never set in stone in ordering. It will come in sequence however the tick data flows. If SPY decides to update twice in a row without ES updating, it will do that.

If your strategy lost connection you would need to manually handle it. The general approach is to just cancel the strategy. You will want to reinstate it manually. NT will load data automatically. You won't be able to tell it to reload. Whenever it is running it will have the proper data.

What you are experiencing is likely sourced to a misinterpretation of the tick sequencing of the two bar objects. Please use my original suggestion to fully see the OnBarUpdate() in action.

[Nick88]

Hi,

I happened to have problems with a CrossOver strategy executing trades where, when I compared them to the position of an Average in my chart, they shouldn't have been executed.

It turned out that the problem was the amount of available data on which the Average was calculated. In my chart there was enough data (>15 days) for the Average to be calculated correctly. Since I started the Strategy from the Strategies tab, there was no more then 10 days to calculate from (default value for minute bars). Look at Tools - Options - Data, maybe that is causing your problem.

[zstheorist]

Nick88 thank you for the suggestion. I am running the strategy from a chart, but perhaps it is using the lookback period from Tools-Option_data anyway?

Josh, I did do exactly what you suggested. This is how I identified the problem.
I printed out Time[0], Close[0]_ES, Close[0]_SPY, SMA1_SPY, and SMA2_SPY on identical length RTH charts of both ES and SPY. Both were set to only display RTH and were started before RTH, so data SHOULD have been identical. Nevertheless, the output window showed different results.
Both strategies set to print were run On Bar Close.

Let me see if I understand your point 2. This appears to be related the question I asked before:
Let me imagine that when my ES bar closes, a) the SPY bar did not update yet. What bar for SPY is referenced in the SMA_SPY inside OnBarUpdate for ES? i.e. Does it wait for the SPY bar to close? Does it use the previous bar? Does it use the current value (tick), even though the strategy is set to only operate on bar closes? This could explain things.
b) Imagine ES bar closes, but SPY bar updated a while ago. Does ES see the bar that closed, or the current tick?
1) If I switch the entry routine for ES to be in BarsInProgess==1(SPY), will this only execute when an SPY bar closes, even though it is on an ES chart? And will it send the order immediately?
2) Should I apply the strategy to an SPY chart instead and add ES as the other instrument? I really don't care what ES price is when I enter/exit - just the SPY price.