Strategy didn’t execute live but works in Market Replay

[WallSt.]

Hello,

Last week was the first week that I ran my strategy live. I ran a scan each evening after the market closed to see if the live results matched the back testing results and unfortunately they didn’t. On May 31st a trade should have executed live but it didn’t.

I ran the EXACT same script that I run live in Market Replay on that symbol and the trade executed as intended so I don’t see how it could be a problem with the script. The strategy was enabled and green prior to the start of the session and stayed green the entire session. There were no connection losses at the time the trade should have executed nor were there any errors in the log tab. I’m using the recommended versions of IQ Feed and TWS and I have tested the connections by executing a number of live trades using a stripped down version of the script.

As it was my first week running live I kept a close eye on the hardware resources and the CPU utilization stayed below 100% approximately 98% of the time and I only used up 25% of my RAM so I don’t think I was overtaxing my system. It’s a multi-instrument script that had approximately 600 additional data series/symbols, 30 days of historical 1 minute bars loaded per symbol, set to calculate on each tick, total lines of code is approximately 900 although I make generous use of return commands so for all practical purposes it was only running about 450 lines on 98% of the symbols that day.

Unfortunately, I can’t share the script with NT for troubleshooting purposes although as I mentioned already it works fine in Market Replay so I don’t see it being a coding issue (I could be wrong).

I am hoping NT can provide some guidance on how to troubleshoot this issue.

Thanks in advance,

WallSt.

[NinjaTrader_ChelseaB]

Hello WallSt.,

Historical data and real-time data (Market Replay) are fundamentally different.

Please review the following:

http://ninjatrader.com/support/forum/showthread.php?p=491297#post491297

For a strategy to match the market replay the data must be exactly the same .

May I confirm that you have added 1 tick granularity to the script?

May I confirm no historical data is being included with the Playback test?

May I confirm the first processed bar in backtest in the Strategy Analyzer is also the first bar processed in Playback?

Have you added any prints to see what values in the conditions are different?

[WallSt.]

Hi Chelsea,

I am aware of the differences between historical and real-time data.

Yes, it is the same data from the same provider (IQ Feed). I used Market Replay with 1 tick granularity for the day in question and in real-time the strategy was set to calculate on each tick.

Not sure what you mean by no historical data included in Playback - I have to use historical data in PlayBack in order to replicate how the strategy calculates in real-time i.e. the real-time strategy loads 30 days of historical 1 minute bars and then calculates the real-time bars on each tick in the same way I load 30 days of historical 1 minute data in a chart and then playback the day in question tick by tick in Market Replay. Same data, same process.

The first processed bar is the same for Strategy Analyzer, Market Replay, and in the Strategies Tab (realtime).

The script I run in the back test is not the exact same script I run in real-time/Market Replay precisely because of what you mentioned in your first paragraph that historical data and real-time data are fundamentally different. I only need to flag a potential trade with the Strategy Analyzer (without using tick data for the sake of speed on a large number of symbols) and then I can drill down with Market Replay using 1 tick data. This is why I haven't gone the print route as I won't learn anything new as the scripts are different.

Make no mistake though, the code I ran in real-time and the code I ran in Market Replay are exact matches, the tick data I received in real-time and the tick data I used in Market Replay are from the same provider and the first processed bar is the same as well.

[sledge]

Are you using limit orders?

[WallSt.]

Hi Sledge,

Yes, the strategy enters with limit orders. On the day in question no orders were even generated live.

[WallSt.]

Possible Solution

I think I may have figured out what the issue is. On the first real-time bar of the session (for each symbol, i.e. looping through the BarsInProgress) I am using the IsFirstTickOfBar command and then running a number of calculations including a nested loop. I printed the results of the nested loop as it calculated in real-time and it only outputted for two-thirds of the symbols (I gave it until 5 minutes after the session open to account for any late openings). I disabled the strategy and then enabled it 5 minutes into the session so that the first bar of the session would now be a historical bar and the script would therefore ignore the IsFirstTickOfBar command and the nested loop outputted for all symbols.

I am guessing this is why the strategy didn’t execute live on May 31st – that symbol's nested loop calculation may have been dropped which would have definitely screwed the code in real-time but in Market Replay it didn’t get dropped so it worked as intended. I’ll have to do some more testing to really say definitively this is the problem but it likely is. I’ll simply not use that command and see what happens. I don’t see how anyone could rely on this command on a large number of symbols if it drops that many calculations.

[NinjaTrader_ChelseaB]

Hi WallSt.,

To find what exactly is different (either data or order fills), it is necessary to add prints to the script that print the values used for the logic of the script to understand how the script is evaluating.

In the strategy add prints that print the values of every variable used in every condition that places an order along with the time of that bar.

This will print to the output window. Backtest the script and when the output from the output window appears save this by right-clicking the output window and selecting Save As... -> give the output file a name and save -> then attach the output from both computers to your reply.

Below I am providing a link to videos that demonstrate adding prints to a script to get further information about the behavior of the script.
NT8 — https://www.youtube.com/watch?v=BA0W...tu.be&t=51m51s

Please add prints to your script and send the output from the script and I will assist in analyzing this output to understand the behavior.

[WallSt.]

Hi Chelsea,

I have isolated the issue to the first real-time bar of the trading session. My strategy works fine when enabled 5 minutes into the session so that the first bar of the session is a historical bar but screws up when the first bar of the session is a real-time bar. This is true even if I replace the Bars.IsFirstBarOfSession command with an alternative expression like ToDay(Time[0]) != ToDay(Time[1]). This is also why my strategy works fine in Market Replay ON the first bar of the session (because it’s not a real-time bar).

I have attached a script that is similar to how my actual strategy begins and I have also attached the output (Output_Realtime). It should count up to 504 which is the total number of symbols (S&P 500 stocks: 503 additional data series + symbol “A" as the primary) but it never counts more than two-thirds of the symbols and that is regardless of how many symbols I am running it on e.g. I run it on 600 it only counts 400, I run it on 400 it only counts 270, etc.

When I change the code to the below and enable the strategy 5 minutes into the session it works fine and I have also attached the output from this (Output_Historical):

protected override void OnBarUpdate()
{
if (ToDay(Time[0]) == 20170614)
{
for (int i = 0 ; i < 504 ; i++)
{
if (BarsInProgress == i)
{
if (Bars.IsFirstBarOfSession && IFBOS[i] != true)
{
IFBOSCount = IFBOSCount + 1 ;
Print(IFBOSCount) ;
IFBOS[i] = true ;
}
}
}
}
}
}
}


Thanks for your assistance,

WallSt.

[NinjaTrader_ChelseaB]

Hello WallSt.,

I'm not quite following.

Are you saying that when backtested, historical data is not loading for all instruments that have been added?

Are you saying that Bars.IsFirstBarOfSession is not true on the first bar of a session in historical data for that BarsInProgress?

I'm trying to understand the logic. On every bar for every data series you loop through all data series and see if the currently processing BarsInProgress is the item in the loop and then check to see if that data series is the first bar of the session, is this correct?

[WallSt.]

Hi Chelsea,

Please see my comments in red:

Are you saying that when backtested, historical data is not loading for all instruments that have been added?

Historical data IS loading for all instruments when backtesting. The script that I attached in my prior post is being run in the strategy tab in real-time.

Are you saying that Bars.IsFirstBarOfSession is not true on the first bar of a session in historical data for that BarsInProgress?

It is true on the first bar of a session for that BarsInProgress IF that bar is a historical bar. If however, that bar is a real-time bar then it is not true for all BarsInProgress only about two-thirds are true.

I'm trying to understand the logic. On every bar for every data series you loop through all data series and see if the currently processing BarsInProgress is the item in the loop and then check to see if that data series is the first bar of the session, is this correct?

Yes, that is correct.

Regards,

WallSt.

[NinjaTrader_ChelseaB]

Hello WallSt.,

I would like to further clarify.

When you mention " If however, that bar is a real-time bar then it is not true for all BarsInProgress only about two-thirds are true."

You are saying that Bars.IsFirstBarOfSession is not evaluating as true on the first bar of a session in real-time, is this correct?

Can you create a sample with only a few instruments that can demonstrate this?

[WallSt.]

Hi Chelsea,

Yes, that is correct.

I'll try running it on only a few instruments on tomorrow's open and let you know.

Regards,

WallSt.

[WallSt.]

Hi Chelsea,

I ran the script that I attached in my prior post on only 10 symbols on today's first real-time bar of the session and it only counted to 8. All 10 symbols had the same first bar time (931 EST).

[NinjaTrader_Jim]

Hello WallSt.,

I am responding on behalf of my colleague Chelsea who is out of the office today.

I have tested your script with some modifications so I could test with the Playback Connection and Market Replay data. I am not running into an issue with the 10 futures contracts I have chosen with Market Replay data.

Here are the data series I tested with on Market Replay Data.

AddDataSeries(	"6B 09-17"	, Data.BarsPeriodType.Minute, 1, Data.MarketDataType.Last);
AddDataSeries(	"6E 09-17"	, Data.BarsPeriodType.Minute, 1, Data.MarketDataType.Last);
AddDataSeries(	"CL 08-17"	, Data.BarsPeriodType.Minute, 1, Data.MarketDataType.Last);
AddDataSeries(	"ES 09-17"	, Data.BarsPeriodType.Minute, 1, Data.MarketDataType.Last);
AddDataSeries(	"FDAX 09-17"	, Data.BarsPeriodType.Minute, 1, Data.MarketDataType.Last);
AddDataSeries(	"GC 08-17"	, Data.BarsPeriodType.Minute, 1, Data.MarketDataType.Last);
AddDataSeries(	"M6E 09-17"	, Data.BarsPeriodType.Minute, 1, Data.MarketDataType.Last);
AddDataSeries(	"NQ 09-17"	, Data.BarsPeriodType.Minute, 1, Data.MarketDataType.Last);
AddDataSeries(	"TF 09-17"	, Data.BarsPeriodType.Minute, 1, Data.MarketDataType.Last);
AddDataSeries(	"YM 09-17"	, Data.BarsPeriodType.Minute, 1, Data.MarketDataType.Last);
AddDataSeries(	"ZB 09-17"	, Data.BarsPeriodType.Minute, 1, Data.MarketDataType.Last);

Could you download Market Replay for those instruments and test with the Playback connection? If we can reproduce through the playback connection, could you give me a list of instruments that would show this issue?

I've included a video of myself testing the script: https://www.screencast.com/t/s5YWgNoU6h

I look forward to being of further help.

[WallSt.]

Hi Jim,

I was not able to reproduce the issue with those symbols in Playback. That is not surprising though as the code has always worked in Playback.

Has NT tested the code in real-time (preferably on a large number of symbols, such as the S&P 500 stocks) as that is where the issue lies?

Regards,

WallSt.