Misma Estrategia misma configuraion DIFERENTES RESULTADOS (REAL vs REPLAY)

[JesusSanchez]

hello i've been designing strategies for a long time, improving and perfecting them until i get very good results, but when i get real i put my strategy in motion and it goes with negative results, obviously i downloaded the data of the day i put it in the replay the same strategy with the same configuration and the results are completely different nothing to see just hit the first 2 or 3 entries,

To clarify that my strategies have a success rate of more than 98% tested in REPLAY during more than a month and a half in a row, they work in each basic TICK point in 5 minutes graphs

I put up a few captures of the performance of the REAL and REPLAY strategy.

I the only explanation that I find that the data downloaded for the ninjatrader PLAYER is manipulated or that the data received from the market is delayed, but check it yourself please... who knows why this happens?

Translated with www.DeepL.com/Translator (free version)

[NinjaTrader_Kate]

Hola JesusSanchez,

Gracias por escribirnos hoy.

Tenemos capacidad limitada para proporcionar soporte de plataforma en español, sin embargo, hemos encontrado que este sitio https://www.deepl.com/es/translator puede traducir el inglés al español con buena calidad. Por favor copie nuestra respuesta y luego navegue al sitio que le colocamos para pegar la respuesta en el cuadro hacia la izquierda.

I'm translating your original message into English below:

hello i've been designing strategies for a long time, improving and perfecting them until i get very good results, but when i get real i put my strategy in motion and it goes with negative results, obviously i downloaded the data of the day i put it in the replay the same strategy with the same configuration and the results are completely different nothing to see just hit the first 2 or 3 entries,

To clarify that my strategies have a success rate of more than 98% tested in REPLAY during more than a month and a half in a row, they work in each basic TICK point in 5 minutes graphs

I put up a few captures of the performance of the REAL and REPLAY strategy.

I the only explanation that I find that the data downloaded for the ninjatrader PLAYER is manipulated or that the data received from the market is delayed, but check it yourself please... who knows why this happens?

It would be expected that there would be differences in results between real time live trading and simulation over that same data in the Playback connection using Market Replay data.

During simulation using real-time market data or Playback, the fill price is based on incoming market data and volume, you may receive better or worse fill prices dependant on where the bid or ask price is and what volume is available at this market prices.

During real-time brokerage trading, orders are filled according to market dynamics, which may vary. The simulator tries to get as close as possible to mimicking market dynamics, but it would not be exactly the same as trading live with a real account.

Also, depending on your data provider for real time data, the data for market replay may be recorded from a different data provider who filters ticks differently - who is your real time data provider?

Thanks in advance; I look forward to assisting you further.

[JesusSanchez]

Hola JesusSanchez,

Gracias por escribirnos hoy.

Tenemos capacidad limitada para proporcionar soporte de plataforma en español, sin embargo, hemos encontrado que este sitio https://www.deepl.com/es/translator puede traducir el inglés al español con buena calidad. Por favor copie nuestra respuesta y luego navegue al sitio que le colocamos para pegar la respuesta en el cuadro hacia la izquierda.

I'm translating your original message into English below:



It would be expected that there would be differences in results between real time live trading and simulation over that same data in the Playback connection using Market Replay data.

During simulation using real-time market data or Playback, the fill price is based on incoming market data and volume, you may receive better or worse fill prices dependant on where the bid or ask price is and what volume is available at this market prices.

During real-time brokerage trading, orders are filled according to market dynamics, which may vary. The simulator tries to get as close as possible to mimicking market dynamics, but it would not be exactly the same as trading live with a real account.

Also, depending on your data provider for real time data, the data for market replay may be recorded from a different data provider who filters ticks differently - who is your real time data provider?

Thanks in advance; I look forward to assisting you further.

I opened an account with ninjatrader brokerage in Philip capital,
I am new to these issues, I only know how to design effective strategies, but if it doesn't help me, is that because they are in basic points?
In a closed bar, will it happen to me? because I also have some created which do not generate so many benefits.

[NinjaTrader_Kate]

Hello JesusSanchez,

Thank you for your note.

As a note, I'm removing the duplicate post you made regarding this same issue since we're already discussing this here.

"I only know how to design effective strategies, but if it doesn't help me, is that because they are in basic points?
In a closed bar, will it happen to me?"

I'm uncertain what you mean here - could you clarify?

Thanks in advance; I look forward to assisting you further.

[JesusSanchez]

Hello JesusSanchez,

Thank you for your note.

As a note, I'm removing the duplicate post you made regarding this same issue since we're already discussing this here.

"I only know how to design effective strategies, but if it doesn't help me, is that because they are in basic points?
In a closed bar, will it happen to me?"

I'm uncertain what you mean here - could you clarify?

Thanks in advance; I look forward to assisting you further.

if you'll excuse me.

How could I do so that this does not happen to me again?

Will the same thing happen to me if I put my strategies under sail?

I want that if I put a strategy and I finish the day and I put it to reproduce that it gives me the same result.

If not, what is the use of creating a strategy that is only effective in a simulator?

what do you advise me to do?

[NinjaTrader_Kate]

Hello JesusSanchez,

Thank you for your reply.

Are you using market orders in your strategy? If so, slippage could be a factor in why you're seeing different results. You can specify a certain amount of slippage be applied to market orders in the Strategy settings when the strategy is running on Market Replay. If you are using market orders this may help you more accurately model real time results.

Thanks in advance; I look forward to assisting you further.

[JesusSanchez]

Hello JesusSanchez,

Thank you for your reply.

Are you using market orders in your strategy? If so, slippage could be a factor in why you're seeing different results. You can specify a certain amount of slippage be applied to market orders in the Strategy settings when the strategy is running on Market Replay. If you are using market orders this may help you more accurately model real time results.

Thanks in advance; I look forward to assisting you further.

hello Kate and thank you for the response and interest.

if i use market orders without displacement because i didn't know i was doing that function.

i have strategies with close0 crossover setup by a high0 EMA with an offset.

and others that close0=high0.


The idea of the strategy is to put a market order in a support or resistance in a high0 and low0 of a lot of price fluctuation with a profit of few ticks,

EXAMPLE in CL = 10 entries x 10 profit of 100 = 1000$

if the next candle touches support the same in a moment you can get enough profit depending on support or resistance that this price, the strategy works well but there is a moment of analysis that the player if it goes crazy counting and begins to tell me operations of more creating a statistic even better but at some point stops being real.
(example) ABC's make me 30 entries, act as if it had been the condition close0=high0 30 times, when in reality in that candle only happened the condition 6 times.


Should I make my strategies on tick charts or make them on closed candles?

[NinjaTrader_Kate]

Hello JesusSanchez,

Thank you for your reply.

So in this case the difference you're looking at between your backtesting and realtime is definitely slippage if you're using market orders during volatile periods - you're almost certain to experience slippage in that case of at least a bit.

From what I'm understanding of your current question it's that you're seeing more entries than you would expect watching the price movement of the candle, is that correct?

What I'd recommend there to understand why this may be occurring then is to turn on the Order Trace function:

Strategy Builder > Default Properties > More Properties > Trace Orders, or:

if (State == State.SetDefaults)
{
TraceOrders = true;
}

Once you then recompile the strategy, you can open a new NinjaScript Output window under New > NinjaScript Output. This will print a log of any orders submitted by the strategy during while it's running, along with any ignored orders. You can then look through and see what may be occurring.

Here is a link to our help guide that goes into more detail on tracing orders:

https://ninjatrader.com/support/help...aceorders2.htm

Trace orders alone may not give you the full picture of whether or not a trade should have been entered on a given bar, so adding prints to your strategy that will show in the NinjaScript Output window, with information on what the variables you're using for your conditions are on a particular bar at a particular time, can be helpful.

This forum post goes into great detail on how to use prints to help figure out where issues may stem from — this should get you going in the correct direction. You can even add these using the Strategy Builder.

https://ninjatrader.com/support/foru...ns-not-working

If you run into issues like we saw here, the above information will allow you to print out all values used in the condition in question that may be evaluating differently. With the printout information you can assess whether the condition is being hit more times than you would expect.

Please let us know if we may be of further assistance to you.

[JesusSanchez]

Hello JesusSanchez,

Thank you for your reply.

So in this case the difference you're looking at between your backtesting and realtime is definitely slippage if you're using market orders during volatile periods - you're almost certain to experience slippage in that case of at least a bit.

From what I'm understanding of your current question it's that you're seeing more entries than you would expect watching the price movement of the candle, is that correct?

What I'd recommend there to understand why this may be occurring then is to turn on the Order Trace function:

Strategy Builder > Default Properties > More Properties > Trace Orders, or:

if (State == State.SetDefaults)
{
TraceOrders = true;
}

Once you then recompile the strategy, you can open a new NinjaScript Output window under New > NinjaScript Output. This will print a log of any orders submitted by the strategy during while it's running, along with any ignored orders. You can then look through and see what may be occurring.

Here is a link to our help guide that goes into more detail on tracing orders:

https://ninjatrader.com/support/help...aceorders2.htm

Trace orders alone may not give you the full picture of whether or not a trade should have been entered on a given bar, so adding prints to your strategy that will show in the NinjaScript Output window, with information on what the variables you're using for your conditions are on a particular bar at a particular time, can be helpful.

This forum post goes into great detail on how to use prints to help figure out where issues may stem from — this should get you going in the correct direction. You can even add these using the Strategy Builder.

https://ninjatrader.com/support/foru...ns-not-working

If you run into issues like we saw here, the above information will allow you to print out all values used in the condition in question that may be evaluating differently. With the printout information you can assess whether the condition is being hit more times than you would expect.

Please let us know if we may be of further assistance to you.

hello kate
thanks to you for your help and quick answers.

1) If, as you can see in the screenshots I sent the same strategy, same day, same stop and profit and how you can see it difference is noticed.

2) I share with you some screenshots that I made a new analysis of the same day, with the ORDER TRACK activated, SCROLL 1 (I didn't know what value to put on it and I put 1) and with the ninjascript output and the monitor, it still gives me positive results, but it has nothing to do with how it was in real.

3) when the monitor puts date up and date down the value it indicates are the times my condition (input close0=low0) paints me date (counting the input arrows, profit and stop????

4) with the ninjascript output and the monitor, you could count the "EXACT" times that close0 crosses an Ema in a candle of 5min??


player = 241 entry
actual = 109 entry

[NinjaTrader_Jim]

Hello JesusSanchez,

This is Jim responding on behalf of Kate who is out of the office at this time.

Historical backtesting, realtime simulations, Market Replay and live trading are all different scenarios.

Further information comparing these can be found below.

Discrepancies between realtime and backtesting — https://ninjatrader.com/support/help...ime_vs_bac.htm

Comparing real-time, historical, and replay performance — https://ninjatrader.com/support/foru...mance?t=102504

Market Replay uses the simulation engine just like trading on the Sim101 account with realtime data, so as long as the strategy is started at the exact same time with the exact same settings used, we can expect the same results as long as the strategy is not adding timers for time based operations and the strategy is not using DateTime.Now (PC clock time) instead of Time[0] (bar time.)

Market Replay data is a snapshot of Level 1 and Level 2 data as it is recorded. This is more granular than single tick data and will have a more accurate representation of the market movement than just using single tick data.

If you want to count how many times Close crosses over EMA, you can use a variable. For example, create an integer variable "CrossesCount" at the class level, and then when yur cross over condition becomes true, increment that integer. (CrossesCount++

Understanding the differences further would involve looking at which trades are different and then to check the logic to see why the trades have not taken place. Debug Prints can be used with Market Replay to see how your logic allows the order submission methods to be reached, and TraceOrders can give hints for why an order has been ignored if the logic does hit the order submission methods. If you want to log the logic actions taken by the strategy as it is running live, I would suggest using a StreamWriter to write logs of the strategy logic so you can compare with your Market Replay tests.

SampleStreamWriter - https://ninjatrader.com/support/help...o_write_to.htm

We look forward to assisting.

[JesusSanchez]

Hello JesusSanchez,

This is Jim responding on behalf of Kate who is out of the office at this time.

Historical backtesting, realtime simulations, Market Replay and live trading are all different scenarios.

Further information comparing these can be found below.

Discrepancies between realtime and backtesting — https://ninjatrader.com/support/help...ime_vs_bac.htm

Comparing real-time, historical, and replay performance — https://ninjatrader.com/support/foru...mance?t=102504

Market Replay uses the simulation engine just like trading on the Sim101 account with realtime data, so as long as the strategy is started at the exact same time with the exact same settings used, we can expect the same results as long as the strategy is not adding timers for time based operations and the strategy is not using DateTime.Now (PC clock time) instead of Time[0] (bar time.)

Market Replay data is a snapshot of Level 1 and Level 2 data as it is recorded. This is more granular than single tick data and will have a more accurate representation of the market movement than just using single tick data.

If you want to count how many times Close crosses over EMA, you can use a variable. For example, create an integer variable "CrossesCount" at the class level, and then when yur cross over condition becomes true, increment that integer. (CrossesCount++

Understanding the differences further would involve looking at which trades are different and then to check the logic to see why the trades have not taken place. Debug Prints can be used with Market Replay to see how your logic allows the order submission methods to be reached, and TraceOrders can give hints for why an order has been ignored if the logic does hit the order submission methods. If you want to log the logic actions taken by the strategy as it is running live, I would suggest using a StreamWriter to write logs of the strategy logic so you can compare with your Market Replay tests.

SampleStreamWriter - https://ninjatrader.com/support/help...o_write_to.htm

We look forward to assisting.

Hello Jin.

I understand and was aware of it, I just need to know how real my backtests are. I forgot to put in sim001, and the backtests are made from the same day and time as it started.

Now, to get some more real data, I first leave the live data with sim001 account and at the end of the day I pass it through the replay, to make my statistics apart.

my idea was to make the strategy in 5min graphics, I thought that if I activated each mark would abate tick by tick, but even that this in speed x1 aps is put the REPLAY make operations and I do not see how many or what it does, I only see that the counter goes up and up, but (it usually happens to me when there is much volatility)

I am working on a strategy for the CL, which in a 5min bar at 7am central european time, takes me out with $80.000 which that day went out of the market with $202.000, I leave the strategy in REPLAY a whole month and I reach almost $500.000

But I think it is because I have too tight a profit that the REPLAY goes crazy, but I do the analysis again and it is always the same amount even if the speed is at MAX.

That's why I need to know the EXACT number of times that close0 crosses through ema or is equal to low0, high0, I'm going to try the variable to see if it works.

if i adjust a tick chart as close as possible to a 5min graph, will the backtest be real? or could it also vary a bit?


PS: the attached exel is from January 02, 2019, (I intend to analyze a whole year), the date of the resuem don't pay attention, and the photo there is the replay almost at the end of January.

Translated with www.DeepL.com/Translator (free version)

[NinjaTrader_Jim]

Hello Jesus,

if i adjust a tick chart as close as possible to a 5min graph, will the backtest be real? or could it also vary a bit?

If this is an ATM strategy, it will operate off of current market data and would not be associated with bar data on the chart. NinjaScript strategies operate off of bar data, so a NinjaScript strategy will behave differently on a 5 min chart versus a tick chart. For an ATM strategy, the data series on the chart will not matter.

Please note however, that if you are using Alerts in Replay, you should only playback at 1x since the alerts are checked on a 250ms interval. If you playback at higher speeds the alerts which fire your entries may be missed.

Using Alerts - https://ninjatrader.com/support/help...ing_alerts.htm

That's why I need to know the EXACT number of times that close0 crosses through ema or is equal to low0, high0, I'm going to try the variable to see if it works.

If you are using an ATM strategy, and not a NinjaScript strategy, you will need to create an indicator (or NinjaScript strategy) that checks the cross overs and increments your variable.

We look forward to assisting.