Live Simulation Vs Playback Results

[ntbone]

Why would a strategy trade differently with recorded playback data compared trading via a simulation account with the same live data feed?

I executed a strategy during a live data feed, that was also being recorded for playback. When I executed the strategy during playback, while most of the behavior was the same, there are some orders/executions that end up behaving differently. The end result ends up being significant enough that the playback is proving to be unreliable.

Please note: this is using playback and then running through the entire day, not using the strategy analyzer.

[NinjaTrader_Jesse]

Hello ntbone,

Without knowing what specific differences you saw it would be difficult to say why that may happen. It is possible to see differences when comparing any two tests even if they share the same data depending on the specifics of those tests. What specifically was happening different in your comparisons?

If you recorded data and later go back to playback the test, it is possible for the strategy to start at a different point now. Are you certain the strategy was executed for the exact same bars during the testing periods? Tick based charts are a good example for this case, if the two tests start even one tick different it is possible for bars to be built differently leading to differences in tests.

This could also relate to the logic you have used, have you tried doing this comparison with a more simple strategy like the SampleMACrossOver to see if you get the same differences?

I look forward to being of further assistance.

[ntbone]

During live trading I may only get filled a quantity of 100 for the instrument. During playback I would get filled the full amount. This was the most drastic of the differences. The exact same code is executing in both cases, on the same instrument with all the same parameters. The only difference is that one executes during the day on the live feed, and the latter executes after the day is done using the market replay data recorded.

Yes the strategy executes for the exact same bars.. My strategy only uses completed bars to perform its evaluation. The currently building bar is not taken into account.

[NinjaTrader_Jesse]

Hello ntbone,

From the given details this still would not fall into any specific category that I am aware of, do you have a situation which you can easily reproduce this happening? For example are you able to run the script in realtime on a smaller timeframe like 10 seconds, see the event then disconnect and recreate it? If you have not yet tried to make a simple example of what you have seen, that may be the next step or to take one of the very simple test strategies like SampleMACrossOver and modify it in a way that produces this result. If you are able to provide a sample of the situation we can look more closely at the logic involved and explore that further. Based on your comment it sounds like you may be getting partially filled, or not the full quantity in realtime, are you otherwise seeing a similar result when not partially filling or using less quantity which always fills?


I look forward to being of further assistance.

[ntbone]

I cannot recreate this specifically because I would never know during a live trading day when it is going to happen. I could collect the orders/executions from the order history for a live event and the orders/execution history from playback together and send those along.