Please outline steps to repairing historical data gaps for chart

[NinjaTrader_ChristopherJ]

When creating a Forex chart with the Price based on Last within NinjaTrader the price displayed will be automatically based off the Bid therefore this data is stored in the Last folder.

If you would like you may set the Price based on Bid or Ask if you would like to see this data in those specific folder.

[adamus]

OK, I have an admission from IB that it is their bad data.

It is one bad tick containing an erroneous bid price.

So what do I do to stop it being randomly reloaded from IB by Ninjatrader? I must have corrected it about 10 times already.

[NinjaTrader_ChristopherJ]

After you have made edits to the data can you confirm you are clicking Save. Also can you tell me what steps you are taking when the edit is not recognized and is overwritten with the original data containing the bad tick.

[adamus]

Yes I am definitely clicking 'save'. I save the data and get the correct data displaying and often it stays like that for days and weeks even. So the edit is always recognised.

However for some reason at some stage NT7 decides to re-fetch it, despite my edit.

[NinjaTrader_ChristopherJ]

I will need to attempt to reproduce this on my end and will follow up as soon as I gather additional information.

[Crassius]

Yes I am definitely clicking 'save'. I save the data and get the correct data displaying and often it stays like that for days and weeks even. So the edit is always recognised.

However for some reason at some stage NT7 decides to re-fetch it, despite my edit.

One of the irritating and often complained about behaviors of NT7 is the overwriting of data in the database, including any edits you have made without any notice or warning to the user.

This is on the list of customer feedback.

It is so bad and annoying that several of us that have complained about this behavior in the past either have separate machines for development where we keep our cleaned data for back-testing so it doesn't get corrupted by NT overwriting our cleaned data in the database, or store our cleaned historical data for back-testing under fake symbols that can't be overwritten by inadvertently re-loading historical data.

Since there is no notice to the user, and no switches to fine tune the control of when NT re-loads historical data.... and in so doing overwriting the local data, INCLUDING any corrections to it, like your bad tick edit, you have to always be thinking about what you are doing that could cause a re-load.

Good software design does not make the user think. You need to think all the time with NT7 because any of the actions listed in Help>>>Operations>>>Historical & Real-Time Data>>>When does NinjaTrader load historical data? can cause Ninja to not only re-load the historical data, but overwrite any corrections or edits you have made in the database.

I think what is happening to you is just that you are causing a re-load of the historical data, and that is overwriting your bad tick edit.

One other place you should be aware of that NT can pull data from is the DataBase cache.

This makes charts load faster as the data is cached, however, after you edit something in the database, there are situations when opening a new chart after the database edit will pull data from the database cache instead of the database you just edited. The cached data will not contain the edit you just made to the database. This can lead to confusion.

On my computer it is located in NinjaTrader7>>>db>>>cache>>>specific folders for your computer follow.

Another thing that is confusing is having data in bid and last at the same time, but only editing the last or only editing the bid. To eliminate this I only keep Last data... I would like to have ASK and BID, but the current overwrite issues make the overhead of keeping the database correct too high. It takes a lot of work and a lot of thinking to always have a clean local database of Last data, I don't the extra work of having to manage the Bid and Ask as well.

For all its user interface design flaws, I still find the locally stored and managed Historical database to be one of NinjaTrader strongest features. It isn't offered in many competing packages at all. I could not use V6 because it lacked this functionality. It gives the user great control over the historical data, and frees him to manage the data as he sees fit.

[NinjaTrader_ChristopherJ]

When testing on my end I am not able to reproduce the issue when reloading historical data however I will continue testing to see if I can force this behavior.

I have also included a link below that goes over when NinjaTrader loads historical data.

http://www.ninjatrader.com/support/helpGuides/nt7/historical_data.htm

[adamus]

Christopher, I think Crassius is correct - I am just doing something that causes NT7 to reload the data, and it wipes the edit.

Can you confirm that this is expected?

[NinjaTrader_ChristopherJ]

This is not expected therefore I will need to narrow down what specific steps I can use to reproduce this on my end.

If you also see this occur again after something specific is done please let me know as well.

[adamus]

OK. So you would say this looks like a similar issue to the one raise by Crassius and others?

Presumably there must be an issue number for it.

I will let you know if I see a definite action causing it.