TradeStation vs. NinjaTrader backtesting discrepancy

[ell2230]

Hi,

I started testing trading strategies on NinjaTrader vs. TradeStation. For parallel comparison, I imported Historical Data from TradeStation.

So, what I've found is that there are cases where trade signal Triggered on TradeStation but not triggered on NT. One of the reason I have confirmed is that on TradeStation, the backtesting assumes you can double entry in the same bar, e.g: when signal reverses TS proceeds as first cover the previous Long/Short and then reverse Short/Long.

In NT, it doesnt allow signal reversing inside same bar. for the above case, when signal reverses, it would cover position in this bar, and enter a reverse buy/sell on next bar.

I am not sure which one is more realistic.

However, there is another big discrepancy in indicator value calculating and Bar constructing.

So, assume that I have perfectly copy my strategies from EasyLaguage to C#. I would say it is 100% correct, as I tested some of the strategies on both platform and generated say 95% similar results. Plus I re-assured the trade being executed are all match-up.

The problem is, okay all the trades are signal based, when the indicators are plotted correctly, the trades are correct trades. However, I happened to do some due diligence, and found out that there are 5% of chance that some indicators plotted on TS are different than that on NT, sometimes its slighlty different, sometimes its BIG.

I understand this feels like an inevitable issue as these are two different platforms, but I just want to know if this is what cannot be resolved. Also, NT has really fast optimization speed, and given that we are able to optimize a lot of parameters, which could be a double sword. (overfitting)...

Sorry it is a long thread, there should be an explanation from NT, or even TS. Its about reliability, and consistency. I love NT, more than TS, but when you see something performed great on NT but moderate on TS, you start to be skeptical.

Thanks,
Steve

[NinjaTrader_Matthew]

Hello Steve,

Please review the information in the following link which explains why there can be discrepancies in back testing:

http://www.ninjatrader.com/support/helpGuides/nt7/discrepancies_real_time_vs_bac.htm

If you're looking for Intrabar granularity in your backtesting, this can be done by adding a lower time frame to your script:

http://www.ninjatrader.com/support/forum/showthread.php?t=6652

Regarding the different indicator values, can you verify the start/end time for the bars you're using in NT vs TS? It could be that you are using a session template in NinjaTrader that is looking at a different start/end period than you seeing your TS charts.

[ell2230]

Thanks for the quick response and references.

Yea, the time sessions seem to be the problem. Even tho charts are almost the same but the time stamp is completely different, What do you mean the time session template? How do I synchornize them?

Should I used instrument default in the template setting?

im testing on FX futures btw

[NinjaTrader_Matthew]

Hello,

The session template is set by right clicking on the chart--> select "Data Series"--> session template

You will also have this option when backtesting, or starting a strategy.

You can verify the session times under Tools--> Session Manager

More information using the Session Manager and Session Templates:

http://www.ninjatrader.com/support/helpGuides/nt7/index.html?session_manager.htm

Please also review our Help Guide article on 'How Bars are Built':

http://www.ninjatrader.com/support/helpGuides/nt7/how_bars_are_built.htm

[ell2230]

Hello,

The session template is set by right clicking on the chart--> select "Data Series"--> session template

You will also have this option when backtesting, or starting a strategy.

You can verify the session times under Tools--> Session Manager

More information using the Session Manager and Session Templates:

http://www.ninjatrader.com/support/helpGuides/nt7/index.html?session_manager.htm

Please also review our Help Guide article on 'How Bars are Built':

http://www.ninjatrader.com/support/h..._are_built.htm

I read all the references and could not solve the problem I have.

I did some investigation, finding that the time stamp on Ninjatrader lagged by 5 hours. So as of the time 7:35am on TradeStation which is the correct time, it shows 2:35 on NT.

I am testing on Canadian dollar, I tried both default template and CME FX Future ETH template. Neither work correctly.

[NinjaTrader_Matthew]

What time zone are you located in? What time zone is your local PC clock set to? What operating system are you using?

[ell2230]

Im in Chicago so, Central Time Zone,my local machine is set to Central Time GMT-6:00. OS: Windows XP

[NinjaTrader_Matthew]

Hello,

I see - NinjaTrader will always time stamp your bars based on your local PC clock, however if this data was imported from TradeStation, there may be an issue with how the data was imported.

When you exported the data from TradeStation, did you have the Display: Time Zone setting set to Local time, or another?

When importing in NinjaTrader, did you have "Time Zone of Imported Data" set to UTC as per default, or did you set it to the same settings you used when exported from TradeStation?

[ell2230]

I did not modify any time settings when both importing and exporting data between TS and NT, that means all by default.

I clearly remember what I did was:
1) load the 1min Chart on TS (correct time stamp) ;
2) insert NTDataDownload Strategy
3) Import the txt file on NT, with UTC left as default

Should I change any time settings? how?

[NinjaTrader_Matthew]

Hello,

By default, TS will export as local time, however our default settings are UTC.

To correct this, you will need to delete your historical data in NinjaTrader and reimport making sure to set the Time Zone in the Historical Data Manager/Import tab to use the same time zone that TradeStation is using.

[ell2230]

Thanks. That seems to work. I am trying to redo everything.

Just a quick question regarding backtesting vs. TradeStation. In TS, we have this check box to make sure limit order gets triggered on when price is traded through. Also, intra bar double entries could happen on TS. I wonder if this is the case on NT? I am more care about "Trade Through" setting on NT, I heard that the default fill type is similar to "Trade Through"? right?

[NinjaTrader_Matthew]

Correct - Default fill type will only fill if the price was penetrated.

Liberal would fill if the price was touched.

[ell2230]

What if we have Profit Target and Stop loss would happen all the in the same bar. Would NT stop it out first or take profit first? suppose we have a very small target?

[NinjaTrader_Matthew]

Only one event will be processed and this occurs the the close of the bar on a single data series strategy. There is no way to predict which event (profit or stop) will be executed.

If you would like intrabar granularity, you would need to at a higher resolution by adding a tick series for example. In this case, it would hit which ever price was penetrated first.

[ell2230]

Only one event will be processed and this occurs the the close of the bar on a single data series strategy. There is no way to predict which event (profit or stop) will be executed.

If you would like intrabar granularity, you would need to at a higher resolution by adding a tick series for example. In this case, it would hit which ever price was penetrated first.

I see. That makes sense. Thank you for your explanation.