Ag Commodity Charts showing bad data?

[stosh]

I just switched my charts from september to december for wheat and corn (from ZC 09-17 & ZW 09-17 to ZC 12-17 & ZW 12-17). The current bar on the charts correctly shows the new dec info but all of the historical bars show the september data. The trades that show on the wheat chart are even september contract trades?? I have tried creating new charts fro scratch, reloading ninjascript, releoading historical data, etc and nothing seems to work. Any suggestions? I've attached a couple screen shots. i am a lifetime subscriber using Phillips and continuum. My market analyzer shows the correct prices as do the bars since I created the chart. It's only the historical that are screwy. Perhaps there is a database command i need to run somewhere. Thanks for looking at this.

[NinjaTrader_ChelseaB]

Hello stosh,

The ZC and ZW 12-17 rollover dates are August 24th, 2017.

What is the merge policy you have selected?

http://ninjatrader.com/support/forum/showthread.php?p=470622#post470622

[stosh]

Thanks for the quick reply. Perhaps that is the problem. Currently, I want to look at both september and december chart so I dont want a merge polic. But when I run strategy analyzer for backtesting I do want history to merge. My global merge policy is currently "merge back adjusted". Is there a way to set merge policy only for backtesting and not have merge for anything else? How do people handle this that are trading calendar spreads?

[NinjaTrader_ChelseaB]

Hello stosh,

No, the merge policy is the same globally either for all instruments or for a specific instrument.

You can change it and then reload all historical data, however, all windows use that setting.

[stosh]

Thank you.

[Harry]

I have not started a new thread as this one deals with agriculturals.

Preset rollover contracts for soybeans (ZS), soybean oil (ZL) and soybean meal (ZM) are false. NinjaTrader 8 has illiquid contracts preset.

If you open a 1000-tick chart with a lookback period of 365 days, you will easily notice the problem with the false contracts.

For ZS: Contracts for August and September are illiquid. Please delete them under instrument settings and adjust the rollover date for the November contract.

For ZL & ZM: Conctracts for August, September and October are illiquid. Please delete them and adjust the rollover date for the December contract.

As a result tick charts will now show up properly and not thin out between June and September.

[NinjaTrader_ChelseaB]

Hello Harry,

Thank you for reporting this.

I am looking into to see if there is an issue and if so, I will request to have it corrected.

Once I have more information I will update this thread.

I appreciate your patience.

[NinjaTrader_ChelseaB]

Hello Harry,

I apologize for the delay on a response here.

I've been deliberating with our development on the proper rollover date for these instruments.

The rules NinjaTrader is using for the rollover date is to roll 8 exchange days before the end of the month prior to contract month.

The contract month of July for instance would end 8 business days before the end of month prior to July (June), placing the end of this contract and the start of the August contract on June 20th.

They've chosen to continue using that pattern moving forward, however, I am happy to assist you in adjusting your rollover dates to use any pattern you would like.

[Harry]

Why do you maintain illiquid contracts?

Hello Chelsea, I just wanted to help. But it seems your developers are particularly stubborn, as they insist on false contracts and false rollover dates.

(1) You mention a rule of 8 working days prior to the end of the contract month.Generally this would not be a bad choice, if it was applied at all.

If I look at ZS, the preset rollover dates are

ZS 08-17: June 16, 2017
ZS 09-17: July 17, 2017
ZS 11-17: August 18, 2017

All of these rollover dates do not comply with your own rule, but are too early.

(2) As I mentioned the selected contract months August and September are illiquid. When I apply contracts and rollover dates as preset, there is a liquidity gap from June 16 (first day when data from contract month August is used) until August 17 (last day when data from contract month September is used). Please see first chart attached.

When I skip the contract months August and September, directly switching from July to November, the false liquidity gap disappears.

I am now applying the 8 working day rule to the July contract. The resulting rollover date would be June 21, which I use as the roll date for rolling to the November contract.
Please note that there is no liquidity gap when the correct contracts are chosen. Please have a look at the second chart.

I do not understand,

- why you try to defend maintaining contract months that are illiquid
- why you do not apply your own rules (8 exchange days) for determining the roll dates

I have had the same argument over gold futures for years, before you finally removed the illiquid contract months. This is something that can be fixed in 10 minutes. Why don't you simply fix it? It would benefit your customers!

[NinjaTrader_ChelseaB]

Hello Harry,

That was my mistake. I was addressing the 2018 contracts which do use this rule set for rollover dates ('8 exchange days before the end of the month prior to contract month') such as 08-18, 09-18, 11-18, etc.

With the 2017 contracts, the rule set was '4 weeks before termination'.

I am happy to submit a feature request on your behalf for the Development department to consider a different rule set for these instruments.

It is up to the NinjaTrader Development to decide if and when they will implement this request.

What specific rule set would you like used?

[Harry]

Hello Chelsea,

I think that the 8-day-rule is fine.

My concern was about using the illiquid contract months of August and September for building the mergebackadjusted chart. These contracts have low volume and cannot be used for backtesting.

Directly switching from the July to the November contract would fix the problem of false volume, as I have already shown in my previous post. Just look at the left chart with the big hole in it, which is caused by the August and September contracts.

Just take them off.

[NinjaTrader_ChelseaB]

Hello Harry,

I have submitted a feature request on your behalf to remove the August and September contract months from the ZS future moving forward to our development.

Once I have a tracking ID for your request I will forward these to you for future reference.

[Harry]

Hello Chelsea,


Thank you very much for your consideration.

I have already fixed the issue on my own machine by simply selecting the same roll dates for the August, September and November contracts. I have further set the roll dates to 8 working days prior to the end of June - applying your rule for 2018.

I have just posted here to make things easier for other users.

[NinjaTrader_ChelseaB]

Hello Harry,

Changing the rollover dates to be the same on multiple contracts may lead to unexpected behavior down the road.

If you are wanting to remove these, I would recommend selecting the contract-month and removing it with the remove button.

[Harry]

Chelsea,

Thank you for this information.

When I delete contract months, they are sometimes reinserted by NinjaTrader. However, when I leave them and just change the rollover dates to match the prior contracts, the intermediate contract months will be ignored. The rollover offsets for the contracts with the identical roll dates will be the same.

I have been doing this for years without any problems.

What type of unexpected behavior would be down the road?