Can market replay be used to test a strategy in either backtest or real-time modes?

[bluelou]

I'm trying to use market replay to test a strategy in both backtest and real-time modes? Is it it possible to use both strategy modes with market replay data?

Presently, my strategy runs as expected while in backtest mode on historical Level I data. However, when I try to run the strategy while in backtest mode but on Level I market replay data the strategy seems to get stuck in a branch of code within OnBarUpdate and an entry signal is never generated. Any ideas?

[NinjaTrader_ChelseaB]

Hello bluelou,

I'm not quite sure I'm understanding what you are doing.

Are you connecting to the Market Replay and then trying to do a backtest with the Strategy Analyzer?

When you say both strategy modes, which modes are you referring to? Are you referring to testing with the Strategy Analyzer while connected to your live connection and testing on a chart while connected to and playing back Market Replay?

Are you referring to an actual parameter you have added to your strategy that changes the behavior of the strategy to work differently with the Strategy Analyzer vs running in real time/


Are you saying that your backtest in the Strategy Analyzer when connected to a live connection is not able to complete the test?

[bluelou]

Hello bluelou,

I'm not quite sure I'm understanding what you are doing.

Are you connecting to the Market Replay and then trying to do a backtest with the Strategy Analyzer?

When you say both strategy modes, which modes are you referring to? Are you referring to testing with the Strategy Analyzer while connected to your live connection and testing on a chart while connected to and playing back Market Replay?

Are you referring to an actual parameter you have added to your strategy that changes the behavior of the strategy to work differently with the Strategy Analyzer vs running in real time/


Are you saying that your backtest in the Strategy Analyzer when connected to a live connection is not able to complete the test?

Let's break the question down into smaller pieces.
Just give me the basics of what Market Replay is
1) Are there certain methods, objects, interfaces, etc that can be used in Strategy Analyzer but CAN'T be used in Market Replay?

1) Can I use BarsInProgress in Market Replay exactly the same as I can in Strategy Analyzer or are there differences?

[NinjaTrader_ChelseaB]

Hello bluelou,

There are no methods or interfaces that are not available when Market Replay is connected. Further, your strategy will not even know the difference between running live and running in Market Replay. The Market Replay is a recording of a real session and works the exact same way.

Yes, you can use BarsInProgress in a script when playing this back in Market Replay. This will work the same as if the script were running live.

There are differences in historical data and real time data that cause differences between backtest and real-time. Below is a link to the help guide that outlines these differences.

http://ninjatrader.com/support/helpG...ime_vs_bac.htm

[bluelou]

ChelseaB,
To be clear I meant are their any coding differences required between Market Replay and Strategy Analyzer?

[NinjaTrader_ChelseaB]

Hello bluelou,

The only differences would be to add intra-bar granularity to your script so that when testing with the Strategy Analyzer the data has tick resolution.

But to directly answer your question, the code does change or behave differently between backtest, market analyzer, and running real-time.

[bluelou]

Hello bluelou,

The only differences would be to add intra-bar granularity to your script so that when testing with the Strategy Analyzer the data has tick resolution.

But to directly answer your question, the code does change or behave differently between backtest, market analyzer, and running real-time.

ChelseaB,
Yes, of course. I typically use 1-tick bars when using Strategy Analyzer PLUS a slower periodicity bar for certain Indicators.

The problem I seem to be running into is when moving from Strategy Analyzer to Market Replay or Real-time. It seems like there's multiple code conflicts when trying to move strategy and indicator mode across these 'modes'. It appears that you've just verified this; i.e., that the necessary code does change among these modes. I don't know what the hell to do given that's the case. Any suggestions?

[NinjaTrader_ChelseaB]

Hello bluelou,

No, the code does not have to change and the existing code's behavior will not change.

You can choose to add intra-bar granularity to a script so that when testing with the Strategy Analyzer there is tick resolution so that you get the same fills in back-test as you do in real time.

This is not required. The data will be different if intra-bar granularity is not added. NinjaTrader will still process your code the same though.

[bluelou]

Hello bluelou,

No, the code does not have to change and the existing code's behavior will not change.

You can choose to add intra-bar granularity to a script so that when testing with the Strategy Analyzer there is tick resolution so that you get the same fills in back-test as you do in real time.

This is not required. The data will be different if intra-bar granularity is not added. NinjaTrader will still process your code the same though.

ChelseaB,
Well, that's good news if true. I'll explain my use case and maybe you can tell me what the problem might be.

1) My code consists of 1 indicator file and 1 strategy file.
2) The indicator requires 2 data streams: a) 1-tick 'Last' 'bars' and b) n--volume bars (e.g., n = 1000)
3) The strategy requires 4 data streams c) 1-tick 'Last' Level 1 data, d), 1-tick 'Bid' Level 1 data, e) 1-tick 'Ask' Level 1 data, and f) the same size n-volume bar used in the indicator
4) I'm using Level I data to get the most realistic fill assumptions possible.

5) Since the indicator is called into the strategy data streams a) and b) must be available in the strategy file for use in the indicator

6) Managing 4 data streams requires many instances where BIP or BarsArray are used.

7) Since the strategy and indicator always use bars, even if it's a 1-tick bar, then CalculateOnBarClose = true is required.

8) With the Strategy Analyzer the strategy runs smoothly and I can control my fill assumptions at Level 1 detail.

9) When I try to use Market Replay I can see my algos producing the correct output w/the Output Window but the code seems to be stuck in one branch since certain Print statements are never hit when in Market Replay or Real-time.

Let me know if you have any suggestions or suspicions. I'm going to try a few more changes tomorrow.

Thx,
Lou

[NinjaTrader_ChelseaB]

Hello Lou,

You will need to debug to find the issue.

If you add Print(Time[0]); to OnBarUpdate() outside of any conditions do you see that print?
(If yes, the script is running)

If you add Print(BarsInProgress); to OnBarUpdate() outside of any conditions do you see that print?
Do you see the value of BarsInProgress changing?
(If yes, you know its going through bars in progress normally)

[bluelou]

Hello Lou,

You will need to debug to find the issue.

If you add Print(Time[0]); to OnBarUpdate() outside of any conditions do you see that print?
(If yes, the script is running)

If you add Print(BarsInProgress); to OnBarUpdate() outside of any conditions do you see that print?
Do you see the value of BarsInProgress changing?
(If yes, you know its going through bars in progress normally)

ChelseaB,
I'm in the middle of doing exactly what you've just suggested.

1) There was an easy solution to fix the BIP issue. The strategy behaves much differently if I launch it from a chart (a 1-tick 'Last' chart) than if I launch it directly from the Strategies tab. I forgot about that quirk in NinjaTrader - I thought it only applied to live trading.

2) Strategy still is still running different in Strategy Analyzer vs Market Replay. Market replay only generating short trades. But, I consider this progress since at least I have trades now. Working on debugging now.

Thanks for your help so far.

~Lou

[NinjaTrader_ChelseaB]

Hi Lou,

Yes, the settings need to be the same to get the same behavior.

However, adding prints that print the values of all of the conditions should show you what is different.