error in Nat Gas chart

[Tasuki]

Ninjatrader never ceases to amaze me (but not in a good way).
Please see attached chart of front month of Natural Gas.
Fresh chart, front month, and over a month and a half of data
is just plain missing from the middle of the chart. WTF?
Using Kinetick EOD data. Is this this problem? Is Kinetick just
plain garbage? Or is there something else wrong?
Thanks,
Taz

[NinjaTrader_PatrickH]

Hello Tasuki,

Thank you for your post.

Likely there is corrupt data. Please shut down NinjaTrader > then go to the following directory on your PC: (My) Documents\NinjaTrader 7\db > then delete the 'day' and 'cache' folders > restart NinjaTrader and test the chart again.

Please let me know if I may be of further assistance.

[Tasuki]

Patrick,
Followed instructions. no change. Realized I'm seeing this data gap in two workspaces (both NG) and two front months (April and May) and three timeframes (day, week and month). So, there's something else wrong, obviously. Any other ideas?
Thanks,
Taz

[Harry]

Have loaded a daily chart for NG 04-15 via Kinetick data. No problem, everything is correct, see chart attached. Problems like this can be caused

-> by erroneous cache data (see comment by NinjaTrader Patrick)
-> gaps in historical data (please reload all chart data when connected)
-> false rollover settings (please check contract months, rollover dates and offsets in the instrument manager)

[NinjaTrader_PatrickH]

Hello Taz,

Thank you for your response.

Harry is correct here. Please go to Tools > Instrument Manager > search for the NG > then select the NG in the search results > select Edit > Misc tab > scroll down to the Contract Months> left click on Contract Months and then the '...' button > ensure all the rollover dates are correct for the contracts you are viewing.

For a video on rollover dates please visit the following link: https://www.youtube.com/watch?v=zzXBE2Jheog

[Tasuki]

no luck yet

Thanks Harry, Patrick,
So, I've been down this road before, and I was aware of the rollover issue. Note that this is a gap in the middle of a chart, where a month and a half of data simply isn't there. Rollover problems, at least as I understand them (and have experienced them on my own charts), have to do with a chart stopping, not missing data in the middle.
No, this is a different problem. Thanks for your input, but you NT tech folks need to figure out how a fresh chart on a new contract with Kinetick EOD data can be simply missing a month and a half in the middle of the chart. BTW, to confirm this, I just created a new workspace with a new chart and got the same bizarre result.
As scary as this is, I've been reporting errors in Ninjatrader for pretty much as long as I've been looking at this software, and I keep finding errors where data is just missing. For example, I found a chart some time ago (I've forgotten the specifics) where an entire month was simply not there. This was eventually confirmed by your techs, and corrected, but the point is, this keeps happening. Harry, I'm glad your chart looks good for NG, but if I'm seeig a problem, I'll bet others are too. How on earth can a month and a half of data simply not be there on the middle of a chart? Please somebody figure this out.
Thanks,
Taz

[NinjaTrader_PatrickH]

Hello Taz,

Thank you for your response.

Unfortunately you would be incorrect. Incorrect rollover dates would cause gaps in the chart when the merge policy merges the historical data for contracts - if the rollover dates overlap. You can test for this by going to Tools > Options > Data > Merge Policy and setting it to Do Not Merge > OK. Then reload historical data on your chart.

Kinetick EOD has the data you need, and it should be downloaded when you request it. You can confirm the data is present by going to Tools > Historical Data Manager > Edit > select your instrument > Last > Day.

Other causes of missing historical data can be corrupt cache, corrupt data, or even using the Tools > Options > Data > Save Chart Data As Historical - enabled, but Get Data From Server - disabled.

Please let me know if I may be of further assistance.

[Tasuki]

no, fortunately!

Patrick,
Thanks for your reply. No, actually, I'm, really glad that I'm wrong, because this would explain the gaps, and thanks very much for the explanation! Not exactly sure why I never encountered this with other trading platfforms, but OK, I"ll just have to deal with it in Ninjatrader. Obviously this requires some more research on my part.
Taz

[bltdavid]

Did you try right click in the chart and select "Reload All Historical Data"?

Here is the Nuclear option that I'd use in such a situation.

[Make a full disk image backup, I use Macrium Reflect Free Edition,
I make one every weekend, so I never really worry about this step]

Tools -> Options -> Data tab
Do Reset DB
Do Repair DB
Do Reset Instruments

Exit NinjaTrader

Goto folder "<Documents>\NinjaTrader 7\db"
Delete all 5 subfolders:
cache data day minute tick

[Leave Windows Explorer open, you'll quickly see
NinjaTrader recreate these 5 folders when it starts]

[Note that the "data" folder represents market replay data,
so you may want to save this data to a spare drive, but I never
do, I just re-download using http://marketreplaydata.com, but
I am currently not a big M/R user, either.]

[Note that just renaming the "db" folder, say to "db.001" should
also work, I coulda swore a NT tech did that on my PC on a
remote support call, only do this when NT is not up, but you'll
lose ATM strategies, since they're stored in the NinjaTrader.sdf
database file, which will be re-created from scratch.]

Restart NinjaTrader
Reconnect to your data feed.
In all charts, right click and "Reload All Historical Data"

But in a true Nuclear situation, I'd probably delete the file
"workspaces\_Workspaces.xml" and start with a fresh
"Untitled1" workspace. Then reload or rebuild your
desired workspace after the data issue is completely
resolved.

See attached for my settings for historical and real-time data.

[Tasuki]

nuclear option

To anyone,
The heart of the problem seems to be what Patrick described---incorrect rollover dates. So, I've found something odd that might lie at the heart of the problem, namely "Offsets". Ring a bell? Here's where you find the "Offset" option:

Tools
Instrument Manager
Scroll down until you find "NG" under "Future", and highlight it
Edit
Misc
Contract months--highlight
hit the [...] ellipsis on the right side
click each of the Contract Months and look on the right side fo the box under "Misc". There are three lines:

Contract month
Rollover date
Offset

It's this last one that's the problem I'm thinking.
This last line, "Offset" is blank for some of the contract months and filled (various values) for others. Is this relevant to the problem?
Most of the contract months in the past do have offset values except those for 07-14 thru 01-15. These past contracts do NOT have offsets. Not sure why that is.
The contract months in the future don't have offsets either, but that might be because they haven't happened yet!
So, could this be the problem? Should the contracts for 07-14, 08-14, 09-14, 10-14, 11-14, 12-14 and 01-15 have offsets? If so, how do I fix this?
Thanks,
Tasuki

[NinjaTrader_Kyle]

Hello,

Offsets for these contracts are included based on what I see when testing here.

Which version of NinjaTrader are you using currently? To check please click Help > About. Version is shown at the top of the menu in a 7.0.1000.X format.

[Tasuki]

Hello,

Offsets for these contracts are included based on what I see when testing here.

Which version of NinjaTrader are you using currently? To check please click Help > About. Version is shown at the top of the menu in a 7.0.1000.X format.

Thanks for weighing in, Kyle,
Version I'm using is the latest, I think, update 27, so:
NinjaTrader 7.0.1000.27 (Direct Live)

You said something about "Offsets for these contracts are included"....included? where? Am I suppose to see something? Was there an attachment I can't see?

[NinjaTrader_PatrickH]

Hello Taz,

Thank you for your response.

The later contract months will not have offsets yet as the closing price on the rollover date is not yet known for the preceding contract and the contract that we would be rolling into on that date.

[Tasuki]

Halleluliah, and a tip of the hat to bltdavid for his excellent explanation of the "nuclear" option (see his reply on 03-19-2015). I followed the steps he described and my Nat Gas charts are now showing the missing data.
Still very mysterious how a month and a half of data just vanished.
Tasuki