Is NT has a chronic bug?

[aytacasan]

Hello,

Previous release of NT have same issue. But i didn't report it before and i solved my issue in various ways. But with NT8 i don't want to do anymore. My opinion its a bug but of course i'm not sure.

Reproduce issue below steps (Also i will attach sample codes and 2 screenshots);

1. Write an indicator that plot some data overlay on price panel and need bars number below 20. Only for example indicator can be calculate support and resistance levels with only to use current bar values. This means this indicator's "BarsRequiredToPlot" property can be 0 (like i said below 20) because of there is no need to use any previous bar's data.

2. Write a strategy that doing nothing in terms of trading because there is no need for reproduce issue. Only do below steps;

2.1. Create private variable for indicator.
private SupResLevels supResLevels;

2.2. Add/Change below lines to OnStateChage -> SetDefaults.
BarsRequiredToTrade = 1; // There is no need bigger value about it
AddPlot(new Stroke(Brushes.Blue, 2), PlotStyle.Hash, "Support");
AddPlot(new Stroke(Brushes.Red, 2), PlotStyle.Hash, "Resistance");

2.3. Add below lines to OnStateChage -> Configure.
AddDataSeries(Instrument.FullName, new BarsPeriod() { MarketDataType = BarsPeriod.MarketDataType, BarsPeriodType = BarsPeriod.BarsPeriodType, Value = BarsPeriod.Value * 10}, Instrument.MasterInstrument.TradingHours.Name, null);

2.4. Add below codes to OnBarUpdate.
if (BarsInProgress != 0)
return;

if (CurrentBars[0] < BarsRequiredToTrade ||
CurrentBars[1] < BarsRequiredToTrade)
{
return;
}

if (supResLevels == null)
{
supResLevels = SupResLevels(BarsArray[1]);
}

Support[0] = supResLevels.Support[1];
Resistance[0] = supResLevels.Resistance[1];

3. Connect to market replay and open a chart for X instruments X TimePeriod. I used "ES 03-17" and 4 ticks renko chart. So our secondary data will be "ES 03-17" and 40 ticks renko. You will see these is NaN and -0- plots on a chart. Warning!!! you have to load data on a chart below 20 bar number for higher time frame for see the problem. It does not matter market replay is at first available date or working or not.

Shortly i'm saying that to NT8; For Indicator calculation you can start from first bar, for strategy calculations you can start from second bar. But NT don't worry about these values and always trying to use 20 and because of we don't have enough bar for 40 ticks renko chart it will plot nonsense values to a chart.

How can i solve this issue? I'm adding below code to a indicators OnStateChange method and set it BarsRequiredToPlot to zero again like State.SetDefaults;

else if (State == State.Historical)
{
BarsRequiredToPlot = 0;
}

Question is why i have to set BarsRequiredToPlot to zero in many states, even i set it at state SetDefaults? If i set it to zero in SetDefaults, so my opinion it's default for all states (no matter real time, historical etc.) and can not be change/override in any stuation(even NT can not) and only programmer can be override defaults.

Regards,
Aytac

[aytacasan]

Support has not noticed my potential bug report? Or working on it?

[NinjaTrader_JessicaP]

Hello aytacasan, and thank you for your message. I am currently looking to the NaN values you were able to get and will return when I have more information.

[aytacasan]

Hi Jessica,

Thank you for your effort. I'm waiting news from you about this issue. Also i think i found new issue (more terrible issue) and i'm working on it, i have to do a little bit more tests and when i'm sure about it, i will report it from here too.

Thanks,
Aytac

[NinjaTrader_JessicaP]

I have some more information.

First, I do have some scripts attached that meet your specifications and do not produce NaN. However, you will see some descriptive warning text instead under the circumstances you specified.

The configuration you are currently trying is unsupported. If we review the help guide configuration for Instrument, we see

Warning: The properties in this class should NOT be accessed within the OnStateChange() method before the State has reached State.DataLoaded

If you run AddDataSeries with a BarsPeriodType.Renko, you will receive a request to use AddRenko instead of AddDataSeries.

If we review the documentation for AddRenko, we see

Syntax

AddRenko(string instrumentName, int brickSize, Data.MarketDataType marketDataType)
AddRenko(string instrumentName, int brickSize, Data.MarketDataType marketDataType, string tradingHoursName)
AddRenko(string instrumentName, int brickSize, Data.MarketDataType marketDataType, string tradingHoursName, bool?isResetOnNewTradingDay)

Warnings:
• This method should ONLY be called from the OnStateChange() method during State.Configure

Therefore, since we must specify an instrument name during State.Configure, and we can not use Instrument.FullName until State.DataLoaded, I have submitted a feature request to the product management team for the following feature :

This user would like an overload for AddRenko that does not require an instrument name

I will follow up with more information as soon as it's available. If the feature requests already exists, a vote will be added to it.

Please let us know if there are any other ways we can help.

[aytacasan]

Jessica,

I'm ready to share new issue. I'm not sure this issue is related first issue or not. Anyway please change SupResStrategy's BarsRequiredToTrade to zero and OnBarUpdate method with below code (this is needed because of simplicity);

protected override void OnBarUpdate()
{
//Add your custom strategy logic here.
if (BarsInProgress != 0)
return;

if (CurrentBars[0] < BarsRequiredToTrade ||
CurrentBars[1] < BarsRequiredToTrade)
return;

if (supResLevels == null)
supResLevels = SupResLevels(BarsArray[1]);

Support[0] = supResLevels.Support[0];
Resistance[0] = supResLevels.Resistance[0];
}

After that please put the strategy these charts; 1&10,2&20,3&30,4&40 tick renko charts. Also I'll attach my screenshots you can clearly see what i mean. Go to far left (first candle of higher time frame) and put vertical line on that candle and check that is it sync with indicators on lower time frame but calculating vs higher time frame data. You will see after 2 tick renko chart there is some strange behavior. NT don't paint some plots. Actually there is no any calculation error, problem is there is holy grail number is 20 for all NT versions. And with NT8 nothing change again holy grail 20. I set indicator's required plot value ZERO, strategies required trade to ZERO but NT changing this values to 20. Why don't understand? Maybe you think why this is so important you'll miss 0-20 bars on lower time frame for trade and most of the time it'll be historical data. This is not important, important think is i want to control this but NT does not allow and i'm worry about why? So if NT decide that why i'm setting BarsRequiredToPlot to 0?

Thanks,
Aytac

[NinjaTrader_JessicaP]

Debugging user code is beyond the scope of the support we may provide. Additionally, as mentioned previously, the strategy you wrote calls Instrument before State.DataLoaded. Please revise your strategy in accord with the message I posted earlier. Once your strategy no longer uses unsupported code it will be much easier to answer questions as to how it is functioning.

[NinjaTrader_JessicaP]

Hello aytacasan,

With respect to the new AddRenko overload, this feature is being reviewed by the product management team and has been assigned the following unique tracking ID


SFT-1979


Please let us know if there is any other way we can help.

[aytacasan]

Hello Jessica,

My English is not so good and i can not follow your posts. You sent many posts when i try to answer yours. I sent my previous post without seeing yours. You are understanding me completely wrong because of that consistently talking about renko or NaN values. I want to clarify;

1. I didn't request debug my code. Very ridiculous if you are thinking like that. Instead Please answer my question... When i set BarsRequiredToPlot to ZERO, even so NT drawing my plots after TWENTY bars later, why???

2. I'm not requesting new feature about renko. This is your understanding. I'm not using renko bar type in my original strategies, i'm using custom bar type which is developing for my private customer. Only my samples using renko charts. So why using renko charts? Because my customer didn't give permission to share their ideas (custom bar type) here and renko is nearest bar type. So actually i'm using custom bar type but samples using renko bar type.

3. You said that "The configuration you are currently trying is unsupported. If we review the help guide configuration for Instrument". So this warning valid under which conditions? When i debug my code i see all properties are accesible and valid in configure state of strategy. But i don't care about which state, if you are recommend to use AddDataSeries in DataLoaded state so i'll.

4. You said that "I do have some scripts attached that meet your specifications and do not produce NaN." I didn't say my problem is NaN values, i said my problem is zero values. But i'll explore your scripts to understand what are you mean.

5. You said that "If you run AddDataSeries with a BarsPeriodType.Renko, you will receive a request to use AddRenko instead of AddDataSeries." Request from who? are you mean you'll receive a warning on output window of NT? If so i'm not. But again i don't use renko because of that i'm not using AddRenko method. I'm using AddDataSeries for my custom bar type.

6. You said that "Therefore, since we must specify an instrument name during State.Configure, and we can not use Instrument.FullName until State.DataLoaded, I have submitted a feature request to the product management team". If i can specify instrument name during DataLoaded state when i use AddDataSeries, then i don't have any feature request... If i can't, yes i have a request about it for AddDataSeries.

So Lets start begining...
If i want to add custom data series i can call AddDataSeries method. I can call it in both states; configure (according to your help guide AddRenko & AddDataSeries samples) or dataloaded (you are saying that if you call any overload that reqesting instument name you have to call method it this state). But if i want access instrument class properties i have to call it from only dataloaded state. Am i understand right?

I'm using overload of AddDataSeries with instrument name because i want to load secondary data series for same as primary data series according to same instrument, same trading hours and same EOD setting. If i write like below;

AddDataSeries(new BarsPeriod() { MarketDataType = BarsPeriod.MarketDataType, BarsPeriodType = BarsPeriod.BarsPeriodType, Value = BarsPeriod.Value * 8});

Is this mean secondary data series will be same instrument, same trading hours and same EOD settings with primary data series?

Thanks,
Aytac

[aytacasan]

Jessica,

Read yours and mine posts again and again and decide we don't understand each other correctly. I'm trying to develop a strategy that working multi-time frame. This strategy will use custom bar type and a few custom indicators. Custom bar type not time based like renko, indicators will calculate for both data series, primary and secondary.

FIRST STEP. I develop custom bar type and it's working correctly.

SECOND STEP. I develop indicators which don't need to access previous bar data. All plots calculating current OHLC. So I set this indicators BarsRequiredToPlot property to ZERO during state SetDefaults (for see plot values from the first bar, currentbar=0) and in OnBarUpdate can not put any code for control bar number. Because i want to calculate and show plots from currentbar=0.

In this step i see a problem that NT don't draw my plots on a chart sometimes (when i use 8&64 tick there is problem but when i use 10&80 tick there is not). Below you'll see steps that produce a problem;

* I connect to market replay data and select first available date.
* Open new chart and select 2 data series which are 8 tick and 64 tick renko. Both data series all properties exactly same, like i said only bar type period different.
* Open add indicator dialog and add my custom indicator for both data series. Again like i said indicators all properties exactly same only data series different. One is calculating/ plotting on smaller time frame other on larger time frame.
* When i look at chart for smaller time frame there is no problem. But on larger time frame i see indicator's plot price markers drawing and data box showing some values but on a chart there is no any plot. Also higher time frame indicator values must be change close to close bar but indicator values changing between bars.
* I do test on my custom bar type and renko. There is same problem but when i test it for time based bar type (for example 5 min for smaller time frame & 50 min for larger time frame) there is no any problem. I think this is a bug, but like i said this happening sometimes and maybe problem can be related market replay, don't know please check it. For clarification look at my attachments (minute.png & renko.png). In attachments i point the problems, also see on renko attachment for higher time frame indicator plots, there is values and NaN values. I know it's so complex and confusing. Sorry about that i try to clarify as possible as.

THIRD STEP. Next step is develop strategy. I'll use two time frame and both will be use my custom bar type and both all settings must be same except period. So for custom bar types i can only use AddDataSeries and according NT's user guides i can call this method from during configure state. Plus like i said all properties must be same (instruement, trading hours, break EOD etc.) except period i used this overload of AddDataSeries; AddDataSeries(string instrumentName, BarsPeriod barsPeriod, int barsToLoad, string tradingHoursName, bool? isResetOnNewTradingDay). And via Print command i check all values is correct or not. I see all values instrument name, trading hours etc. are correct. Next step add indicators to my strategies etc. But i see the problem which i report to you my first post. But because of we have confidentiality agreement with my customer. I create all samples with renko bar type which has same problems. When i debug my code i see my indicators BarsRequiredPlot value is zero at SetDefaults state but during Historical state its values changing because of that i set again at that state to zero and my problem is solved. Yes i know this is so illogical. You concentrate NaN values but my opinion most important thing is where does it come from zero values?

So Jessica maybe we have to go step by step and first try to understand/solve my second step. Because at third step i'm plotting indicators values on a chart when i do market replay back-test. Maybe third step problem will be solve after solve second step problem. After that maybe we can talk about AddDataSeries overloads and which state is right to call it etc. But for now i have to say that if i try to call AddDataSeries at during DataLoaded i receive an error that "Error on calling 'OnBarUpdate' method on bar 0: Index was outside the bounds of the array." for line if (CurrentBars[1] < BarsRequiredToTrade) return;

Thanks,
Aytac

[NinjaTrader_JessicaP]

I appreciate your patience as we work through our misunderstanding. What happened when I tried to use your code, is that I saw calls such as this in the test code you sent me. I will emphasize the important parts.

            else if (State == [B]State.Configure[/B])
            {
                AddDataSeries([B]Instrument[/B].FullName, new BarsPeriod() { MarketDataType = BarsPeriod.MarketDataType, BarsPeriodType = BarsPeriod.BarsPeriodType, Value = BarsPeriod.Value * 10}, Instrument.MasterInstrument.TradingHours.Name, false);

Since the help guide specifies Instrument should not be accessed before State.DataLoaded, we can not use the code you sent me to prove things about NinjaTrader. I was attempting to change the code that you sent us so that it could be used in the scenario you presented, and this is where I ran into a second obstacle regarding Renko bars, which could only be resolved through a feature request. Until the feature request is granted, we can not use the test scenario, or the test script, you presented. Otherwise, it is possible that the way our test code is written could impact our results.

So I will keep the feature request I have already submitted, with the vote coming from myself instead of you, so that it is easier to resolve future queries like this one, as the overload (which I will demonstrate in a code block later on in this post) that allows you to use the currently traded instrument does not allow you to use renko bars and tells you in the log window to use AddRenko instead.

This said, when you say

I'm using overload of AddDataSeries with instrument name because i want to load secondary data series for same as primary data series according to same instrument

The way to do this while having a value 8 times the value of your primary series is like this

[FONT=Courier New]AddDataSeries(BarsPeriod.BarsPeriodType, BarsPeriod.Value * 8);[/FONT]

All this said, to simplify our situation, would it be possible to come up with a stripped down version of the code which demonstrates what you would like us to look into, and nothing else? Please use the overload in the code block from this post, and for simplicity, please use BarsPeriodType.Tick when testing if this shows up with Tick bars, or if Renko bars are required (with the understanding that they are a stand-in for a bar type you are creating), please create these bars with "AddRenko("ES 03-17", ..." .

In the meantime, since I believe what you are referring to is bars inside an indicator plotting earlier than expected when called from a hosted strategy, I will look into this on my end. I would like to ask that if I am correct about this, you make one more change to code you send our way. In your strategy, rather than calling AddPlot and then assigning plot values from the indicator, please consider using AddChartIndicator, as this will contain some "under the hood" code specific to managing indicators, and may by itself resolve your query.

[aytacasan]

Jessica,

Really try to understand you and what i understand is;

You are saying that : We can not access instrument class and it's properties during configure state (which is previous state of DataLoaded) like user guide said.

>>> Again i want to say that when i try to access i don't have any problem about class and it's properties and they are available for me. Also this type limitation is not good. I'll explain below why it's not good? But let say we accepted this limitation.

You are saying that : Plus because of this limitation also you can not add secondary data series with AddDataSeries overload method that require instrument (name property). But also you are saying that if really need to use this overload method, let write instrument name constant; i mean use AddDataSeries("ES 03-17", ...) instead AddDataSeries(Instrument.Name, ...) or use AddDataSeries(BarsPeriod.BarsPeriodType, BarsPeriod.Value * 8) that will load secondary data series for instrument that same as primary data series but with other period value.

>>> Use AddDataSeries("ES 03-17", ...) approach so bad i have to change this constant before every test for each instrument(Maybe we can add InstrumentName parameter to a strategy and use it inside strategy). Second approach AddDataSeries(BarsPeriod.BarsPeriodType, BarsPeriod.Value * 8) is not working (at least for not time based data series like renko) because we can not set break EOD settings.

You are saying that : Because of above problems you requested new feature from development team that write new overload method of AddRenko that allow to add renko data series (same instrument of primary data series) inside configure state.

>>> I don't understand what are you mean with this new feature? Exactly what will it do? But again i want to say that i have a custom bar type and it's logic something like renko and it's not time based like renko. Could you please explain more detail about new requested feature?

You are saying that : I believe what you are referring to is bars inside an indicator plotting earlier than expected when called from a hosted strategy.

>>> I don't understand what are you mean. I'm saying that sometimes, after higher time frame's first bar close, shorter time frame can not plot it's value or ploting wrong value (zero). Please explore my attachments more closely. Instead use custom plots inside strategy i'll try to use AddChartIndicator like you said.

PS : NT's market replay have so serious bugs and i capture video for the most important. It's size is almost 72 MB how can i deliver video to you? I can upload to onedrive and send you a link.

Thanks,
Aytac

[NinjaTrader_JessicaP]

Briefly, what I am saying first, is that we deeply appreciate the report and the effort you have made to help improve NinjaTrader 8. This said, the test you gave us is invalid for the reasons I outlined. I have tried to fix the test. I put in a feature request which would enable me to fix the test, so that we could use it to prove things to ourselves about NT8.

This is aside from your query. I believe we should focus on your query rather than the test.

I would like to help you get charts that plot when we desire them to plot, and from what you have mentioned, your BarsRequiredToPlot setting is being ignored.

A first step, to reduce the complexity of the situation we are in and focus on exactly what is occurring, so that we can either repair NinjaTrader or your code, is to use AddChartIndicator, rather than adding plots and assigning values to them. Please let us know what the results of this first step are.

[aytacasan]

Ok i see Jessica i'll try to use AddChartIndicator first and will inform you about the result in a few minutes but i recommend to take care of my screen capture about market replay. How can i deliver it to you?

UPDATE : After add higher time frame indicators via AddChartIndicator like your said. I remember that before i do same thing and after see plots not plotting every bar i changed my code and add plots to strategy and set this plots for every lower time frame's bar. At that point i want to ask that why NT don't plot indicator values to all sub bars?

[NinjaTrader_JessicaP]

Thank you for this additional information, aytacasan. I will be reviewing the information in this ticket and I will return when I know exactly why you are not getting sub-indicator bars to plot when their BarsRequiredToPlot is reached.

As far as sending a 72 MB video, I recommend using the publicly available service SendSpace,

https://www.sendspace.com/

Please set your from address to [email protected] if you use this method.