Result of the Strategy Analyzer

[Fernand0]

Hi, i've been testing severals strategies and optimizing them.

The results, even with High Order fill resolution or Tick Replay... i don't see the same result, the analyzer and the Strategy Performance don't give a single real number.

Other simple platforms make the backtest really fast, without buying or selling on a price that was never touched.........

And for some reason Renko and other BarTypes don't have the Tick Replay... so..... I'm kinda tired of testing and testing.... any help? what's going on? why NinjaTrade has to do such a big deal to get the right numbers?

Thanks for your time.

[NinjaTrader_Jesse]

Hello Fernand0,

Thank you for the post.

Regarding Renko and other bars types that use RemoveLastBar(), these BarsTypes cannot be used with TickReplay specifically due to how their logic works. So what you are seeing in regard to these types would be expected.

As far as the results you are seeing, this would come down to a combination of your logic and also the nature of how backtesting works in NinjaTrader. It is expected that there can be discrepancies between realtime performance reports and backtesting, you can find some of the common differences here: https://ninjatrader.com/support/help...lightsub=discr

If you are seeing that the overall logic is happening as expected but the fills are not as expected, this may be due to the backtest using its fill logic, You would likely need to review the step by step of the fill logic here and compare it against your fills to confirm.
If you are instead seeing the logic of your script execute at different times, this can be caused by differences in data or differences in how the logic is being executed overall. Most commonly you would need to step through your code to address these items or to find what specifically is happening differently to see if that can be mitigated. Prints often work well for this type of debugging or to see that in one situation the script was executed the same as another.

Please let me know if I may be of additional assistance.

[Fernand0]

Sorry Jesse, I was gonna let this issue pass... but.... I started to use the Analyzer again and "bugs" are all over the place...

I do not understand the purpose of having those BarTypes, even using High Fill Resolution, with Heiken-Ashi it does not work.
Sometimes the entry and exit point are not between a High and the Low. They are in the air, price was never in that entry/exit point.
I tried with Tick Replay (for Heiken-Ashi Tick 150) shows an error of some sort... "data series is not available"...

So... why NinjaTrader has these "features" if they only mislead users(at least me..)? The backtest feature has to be one of the best things on the platform... and i would dare to say that Metatrader(the most basic platform) has a better backtester than NinjaTrader...

Several backtesters have their own option of precision(e.g. High Fill Resolution, Tick Replay, Level II) ENABLED BY DEFAULT. And with NinjaTrader you have to enable them, and only 1, 2 is too much apparently.
You just have 2 options: 1 of them doesn't work well, the Fill Resolution, and the other, Tick Replay, it's extremely slow...... 2 HOURS for 500 Iterations...

Please... tell me if you are working on this... like.. yesterday...
Stop providing features like Oder Flow + for Lifetime license... and do something with your backtester.... otherwise your Platform is no different than the rest. Why? Because the features that you are offering can't be backtested properly!

[NinjaTrader_PatrickH]

Hello Fernand0,

Thank you for your response.

I would like to break down each item you are reporting so we can get to the root cause.

Now aside from the items my colleague posted in our first response to you I do want to clarify on a few items:

• Renko Bar backtesting is generally not recommended in any trading software as Renko Bars re-build themselves on real-time data and therefore a historical backtest is not accurate to how the bar formed in real-time.
• Tick Replay is not enabled by default as it requires specific development to utilize it and Tick Replay does not factor into fills directly. Tick Replay is intended for historical bar processing to build tick by tick. Now if you are using Tick Replay you need to configure your NinjaScript object to utilize it as detailed in our documentation at the following link: https://ninjatrader.com/support/help...ick_replay.htm
• High Fill Order Resolution is also not enabled by default as it is a specific setting based on available historical data. If you do not have historical Tick data then the Tick option would not work for you for example.

On the subject of historical data and instruments, can you advise the data feed provider you are using in NinjaTrader for your historical data?

You mentioned your fills are outside the bars with and without the High Order Fill Resolution enabled. And these are Limit and Stop orders, correct? Do you have a simplified strategy that demonstrates this behavior that we may test on our end?

You can export your strategy by going to Tools > Export > NinjaScript Add On > Add > select your strategy > OK > Export > name the file 'NTsupport' > Save.

The file will be located under Documents\NinjaTrader 8\bin\Custom\ExportNinjaScript. Please attach the file to your response.

You stated Heiken Ashi does not work on your end. Can you detail this a bit further? Can you provide screenshots of the settings used in the Strategy Analyzer and then the results that you are seeing?

To send a screenshot with Windows 7 or newer I would recommend using Window's Snipping Tool.

Click here for instructions: http://windows.microsoft.com/en-us/w...#1TC=windows-8

Alternatively to send a screenshot press Alt + PRINT SCREEN to take a screenshot of the selected window. Then go to Start--> Accessories--> Paint, and press CTRL + V to paste the image. Lastly, save as a jpeg file and send the file as an attachment.

Click here for detailed instruction: https://www.java.com/en/download/faq/screenshot.xml

For the error message "data series is not available" can you send me your Log and Trace files?

You can do this by going to the Control Center-> Help-> Mail to Platform Support. Ensuring 'Log and Trace Files' is checked will include these files. This is checked by default.

Please include 'ATTN: Patrick H' in the subject line and reference this thread in the body of the email.

I look forward to assisting you further.

[Fernand0]

Thanks Patrick, but i'm really tired... I'll show you some images.. and that's it.... i already have other problems with the Strategy Builder that if you have an open position(let's say short)... and you have less than $1000... and you execute EnterLong... it throws errors of margin excess like crazy.... Paul is already helping me with that..

According to this image.. i will earn money.. not lose it...
Analyzer

These are the results and... some... more (check the big purple arrows to see entry/exit prices..)... and BTW... i lose money... so...
Chart

I downloaded that data for Market Replay from CQG. How NinjaTrader buys and sells at prices that don't match with the database...? How in the hell is that possible?
If we talk about coding, how..? how is this possible? That database has to be reflected in the charts... let's suppose that there is no problem there... then how NinjaTrader buys/sells at a nonexistent price in that database at that time??!!!!!

After that Chart... i start to wonder if i made a mistake leaving forex and cTrader to go for futures with NinjaTrader...

Few hours ago i went to sleep because there was nothing i could do (i was making a new backtest)...
The Strategy Analyzer with the Tick Replay enabled slows down the program to the point where you can't click a button (the rest of the programs of the pc work perfectly)
So... 500 iterations... of 5 days... on Heiken-Ashi Tick 150... took more than 5 hours to check... this is a disaster.......

[Fernand0]

I tried with Tick Replay (for Heiken-Ashi Tick 150) shows an error of some sort... "data series is not available"...

I had to restart the pc, for some reason it was not enough to reconnect and close and reopen the program. This was fixed.

[NinjaTrader_PatrickH]

Hello Fernand0,

Thank you for your response.

The Strategy Analyzer is analyzing Historical Data on the close of the bar. Playback is playing back Market Replay Data tick by tick as if in real-time.
Differences here are two fold; Historical and Market Replay data are not the same data, and backtesting uses fill logic for historical data where Market Replay data is using the simulation engine.

Executions are occurring outside the price level for a Heiken Ashi bar in Playback as the Heiken Ashi is not a representation of where the ticks were. Now in a backtest as the historical fill processing is using the Heiken Ashi bar to determine where the order filled you would in fact see it fill within the bar, but add a 1 Tick High Order Fill Resolution and now they will fill outside of the bar again.

Please let me know if you have any questions.

[Fernand0]

The Strategy Analyzer is analyzing Historical Data on the close of the bar.

Ok, so, no tick data... I wonder how this is useful... seriously. Anyway, in my examples i used Tick Replay, so if Tick Replay doesn't give me the real results.. what does..?

Differences here are two fold; Historical and Market Replay data are not the same data, and backtesting uses fill logic for historical data where Market Replay data is using the simulation engine.

But if i put the Tick Replay ON it should be precise..? Please... tell me or link me, what i have to do to get the REAL RESULTS on a backtest... that's the only thing that matters to the algo traders

Executions are occurring outside the price level for a Heiken Ashi bar in Playback as the Heiken Ashi is not a representation of where the ticks were.

So, what you are saying is that NinjaTrader can't handle Heiken-Ashi BarType.... Otherwise the prices would be in the parameters of the candlestick....
Your Heiken-Ashi BarType doesn't respect the real values and just renders whatever your algo for this BarType dictates... WOW... I will check the Heiken-Ashi code, but there is a public algorithm for the Heiken-Ashi where every bar respects the High and Low...
Developers didn't think that the way how we see the Candlesticks have an impact on our strategies..?
In other words, your Heiken-Ashi is not smoothing data, is rendering an ILLUSION..

[NinjaTrader_PatrickH]

Hello Fernand0,

Thank you for your response.

Historical backtesting discrepancies are detailed at the following link: https://ninjatrader.com/support/help...ime_vs_bac.htm

There is no means to get the exact results as in real-time trading in backtesting.

You would want to review Heiken Ashi bars tye so you have a better understanding of how they are built: https://ninjatrader.com/support/help...HeikenAshiBars

Please let me know if you have any questions.

[Fernand0]

I know how Heiken-Ashi works, that's not the problem, the problem is that the entry/exit prices are being displaced few bars on the chart because they are trying to be based on time as well... that's certainly a problem...

10 tick chart, 20 ticks happen at the same time, 2 bars of 10 ticks each with the same initial time.
Let's say that your target is in the 2nd bar, the mark will be placed in the air, above or below the first bar, but it should be placed at the 2nd bar, where the target really is...
After all these years, why this issue is still around?

Running a Strategy at the Close of a Bar or Tick by Tick
• During backtest, strategies can ONLY be processed at the close of each bar.

Ok, i will, just, let it be...

Thanks for everything

[NinjaTrader_PatrickH]

Hello Fernand0,

Thank you for your response.

As you stated it is due to the timestamps for the Tick charts. I can forward a feature request if you have a suggestion for the execution markings on matching timestamps.

Please let me know if you have any questions.

[Fernand0]

Hello Fernand0,

Thank you for your response.

As you stated it is due to the timestamps for the Tick charts. I can forward a feature request if you have a suggestion for the execution markings on matching timestamps.

Please let me know if you have any questions.

Yes please, I'm new on this, I don't know if I have to say it in this thread or in a special part of the forum.

It's not a suggestion actually, it's more like fixing a bug. The mark should be where the price is, not where the first Bar was drawn.
So the mark it's been drawn at the right Y(price), but not at the right X(bar). It only has to find the right X, the right bar.

[NinjaTrader_PatrickH]

Hello Fernand0,

Thank you for your patience.

This feature request has been assigned the internal tracking id of SFT-3257.

Please let me know if you have any questions.