Best way to refresh chart data if internet drops?

[forrestang]

Howdy.

Simple question, and that is what is the best way to refresh data if you're internet connection drops for some number of minutes?

I do not want to use the "Refresh all HIstorical Data," Option as I have a lot of data that I would like to keep.

But say for example if I wanted to refresh only the current day's data, or even the past 30 minutes?

I know I can go into Historical Data Manager->Download, then select the instrument, select the current date, and download the tick, minute data I want.

But is there a quicker way to accomplish this when I know I will have some missing data upon the internet reconnecting?

[NinjaTrader_PatrickG]

Howdy.

Simple question, and that is what is the best way to refresh data if you're internet connection drops for some number of minutes?

I do not want to use the "Refresh all HIstorical Data," Option as I have a lot of data that I would like to keep.

But say for example if I wanted to refresh only the current day's data, or even the past 30 minutes?

I know I can go into Historical Data Manager->Download, then select the instrument, select the current date, and download the tick, minute data I want.

But is there a quicker way to accomplish this when I know I will have some missing data upon the internet reconnecting?

There are no other ways other than the ways you've mentioned to reload historical data. Right-clicking on your chart and selecting "reload all historical data" will replace your locally stored data with data from the historical server you connect to. This will only extend out as far as your chart's "Days to load" setting. It would not replace or reload data beyond this amount of historical days.

[forrestang]

So just to be clear...

Let's say that I have a 10 year cache of minute data for a particular symbol.

If I have a chart open that only has say 10 days of data on it... if I use the "Reload All Historical Data" from the right-click context menu, it will ONLY reload the last 10 days of that symbol from my data provider?

I was under the impression that this option would erase everything stored, then download the max allowed from your data provider?

If it does only download the number of days open on the chart, will this be affected if you have that same symbol open on another chart that has the full 10 year history loaded? What if that chart is minimized?

Apologies if these questions are nonsensical, I just always thought the "Reload All Historical Data" was a sort of nuclear option.

[bltdavid]

Apologies if these questions are nonsensical, I just always thought the "Reload All Historical Data" was a sort of nuclear option.

It is ... but only for the date range (and data type) being display on that chart.

Inside the Data Series dialog, assuming "Load data based on" is set to "Days",

If you're showing a 1-day chart with 365 days in the "Days to load", then because that is what the chart needs to show you (in case you scroll backwards, for example) it will reload historical Day data for the last 365 days (no Tick or Minute data, just Day data) for the instrument the chart is showing you.

Even if you have 10 years of Day data for that instrument, Ninja only reloads the last 365 days of Day data -- because that is what Days to Load is set to.

Same with Tick or Minute based charts.

[NinjaTrader_PatrickG]

So just to be clear...

Let's say that I have a 10 year cache of minute data for a particular symbol.

If I have a chart open that only has say 10 days of data on it... if I use the "Reload All Historical Data" from the right-click context menu, it will ONLY reload the last 10 days of that symbol from my data provider?

I was under the impression that this option would erase everything stored, then download the max allowed from your data provider?

If it does only download the number of days open on the chart, will this be affected if you have that same symbol open on another chart that has the full 10 year history loaded? What if that chart is minimized?

Apologies if these questions are nonsensical, I just always thought the "Reload All Historical Data" was a sort of nuclear option.

When you choose to "Reload all historical data" for an instrument, it will reload all charts loading that instrument and that interval type. For example, a 15 minute CL chart and a 60 minute CL chart will both reload when you choose to reload on one of the charts. It will reload the data for the highest "days to load" chart, so in your example, yes, it would reload (or attempt to) all of the data according to your highest "days to load" chart.

In your case, it seems like manually downloading from the Historical Data Manager is the best solution.

[bltdavid]

If you're looking at a minute based chart (eg, 5-Minute chart) with Days to Load set to '10' then if you right click and select "Reload All Historical Data" then you are choosing the nuclear option for that chart's data only.

[Edit: Well, technically, all charts showing the same type of data (Tick,Minute,Day) for that same instrument will be forced to reload themselves if you reload any one of them. See Patrick's comment above.)

In other words, that chart needs to blow away all of its current set of data (10 days worth of minute data for that instrument) and reload all 10 days again.

[Edit: That's why it affects all chart windows showing the same type of data, because you've basically nuked all (or most) of their bar data and they all have to pause and wait for the reload to finish, so they can rebuild their historical bars from the last 10 days and refresh the candlesticks.]

That's it. The nuclear option is limited to 10 days ... aka, whatever the chart is set to show you, which is whatever you want to see if you want to scroll backwards in time.

[forrestang]

Thanks guys, those explanations help a lot!

[bltdavid]

Thanks guys, those explanations help a lot!

Conversely, the opposite is also important:

If you want to look at all your 10 years of data on a minute-based chart (say 240-Minute) and you set Days to Load to (oh, I dunno, maybe 3650, or you set the custom date range to be 10 years) ... this is a chart where you DO NOT reload historical data from that chart.

Because, if you do, prepare for a long coffee break ...

[Edit: Reloading other minute based charts with 10 Days to Load is ok, and should be relatively fast, just don't right click/reload from your 10 year chart, ever.]

[bltdavid]

[Edit: Reloading other minute based charts with 10 Days to Load is ok, and should be relatively fast, just don't right click/reload from your 10 year chart, ever.]

Hi Patrick, I think I may have learned something here:

You say "It will reload the data for the highest "days to load" chart, so in your example, yes, it would reload (or attempt to) all of the data according to your highest "days to load" chart."

Which kinda invalidates the advice of "ok to reload in other charts with 10 days to load even while viewing 10 years of a 240-Min chart" -- because if NT takes the highest days to load, won't it try to reload the 10 years (or maximum amount)?

I mean, it sounds like, whenever he is viewing a chart with his 10 years of minute data, he should never reload historical data in any minute based chart for that instrument -- because otherwise NT wants to reload the max minute data, because that doggone 10 year chart overrides all the others.

Is this correct?

[forrestang]

I mean, it sounds like, whenever he is viewing a chart with his 10 years of minute data, he should never reload historical data in any minute based chart for that instrument -- because otherwise NT wants to reload the max minute data, because that doggone 10 year chart overrides all the others.

Is this correct?

This is how I have understood it. That there is daily, minute and tic data. If you reload ANY chart that has tic data on it, it will erase and download the highest amount of data for that symbol that has tic data... while ignoring any non-tic data for that instrument.

So ANY "Reload all Historical Data" will occur on THAT data type, for the max data displayed on any of those charts.

I was wondering if having a chart MINIMIZED though would have an impact, as minimized charts don't use resources if I am correct? For example say you open a saved workspace that has a 10 year minute chart minimized, it never actually loads until you un-minimize it. So I was wondering if NT would reload 10 years of data even though it has been minimized since NT has been opened?

I'm curious but not willing to try it, as the replies have told me what I need to know to better manage reloads when necessary.

[NinjaTrader_PatrickG]

because if NT takes the highest days to load, won't it try to reload the 10 years (or maximum amount)?

I mean, it sounds like, whenever he is viewing a chart with his 10 years of minute data, he should never reload historical data in any minute based chart for that instrument -- because otherwise NT wants to reload the max minute data, because that doggone 10 year chart overrides all the others.

Is this correct?

Yes, this is correct. NinjaTrader reloads all charts of the same instrument and interval type when you choose to reload on one of those charts.

There are three fundamental data types to consider: Tick, Minute, and Daily. Reloading historical data in one chart of an instrument and data type will reload all charts of that instrument and data type. Another example other than my previous minute-based charts would be tick-based: A 4 Range CL chart and a 255 Tick CL chart. These are both tick-based and historical data will reload for both when one is reloaded.

As I recommended, if you are not wanting to overwrite your locally stored historical data, the best route would be to manually download using the Historical Data Manager.

I was wondering if having a chart MINIMIZED though would have an impact, as minimized charts don't use resources if I am correct? For example say you open a saved workspace that has a 10 year minute chart minimized, it never actually loads until you un-minimize it. So I was wondering if NT would reload 10 years of data even though it has been minimized since NT has been opened?

All windows, even if minimized (or in a different open workspace which is not the current active workspace) are actively loading and take up PC resources.Reloading will affect these charts as well.

[bltdavid]

This is how I have understood it. That there is daily, minute and tic data. If you reload ANY chart that has tic data on it, it will erase and download the highest amount of data for that symbol that has tic data... while ignoring any non-tic data for that instrument.

So ANY "Reload all Historical Data" will occur on THAT data type, for the max data displayed on any of those charts.

Remember:
"Reload all historical data" != "Download all historical data"

Reload happens from data on disk.
Only missing data is downloaded.

[NinjaTrader_PatrickG]

Remember:
"Reload all historical data" != "Download all historical data"

Reload happens from data on disk.
Only missing data is downloaded.

This is actually incorrect - my original statement was slightly incorrect in this regard. I've confirmed that it replaces locally stored data with downloaded data when you choose to reload.

I will edit my original post.

[forrestang]

Remember:
"Reload all historical data" != "Download all historical data"

Reload happens from data on disk.
Only missing data is downloaded.

I'm not sure about that? I think "Reload all historical data", (Cntrl+Shift+R), or the option available from the contextual right-click on chart will actually erase and downlaod from your provider for the # of max days, of that interval on the chart.

[bltdavid]

This is actually incorrect - my original statement was slightly incorrect in this regard. I've confirmed that it replaces locally stored data with downloaded data when you choose to reload.

I will edit my original post.

Ah, thanks. Then my original idea is still valid:

If he's viewing his 10 years of Minute based data in a chart window (even if that workspace is open but not active, even if that 10yr chart window is minimized), he should never reload from any other chart window with minute data for that same instrument. Well, "never" is a strong word there, let's just say "refrain" ...

Because, if he does, the max minute data will be downloaded (downloaded, not just re-read from disk), because that 10 year chart overrides all the others.

But, to be clear, there is no real penalty when he does this. No loss of any of his 10 years of prior minute data, the only "penalty" is time spent downloading and replacing (eg, refreshing) minute data he already has.

And, just to confirm, the minute data will be downloaded even if it already exists on disk -- such that newly downloaded minute data will replace the same minute data (if already present) on disk.

And finally, NT historical servers do not have 10 years of minute data for any instrument, so downloading max minute data is really only about 1 years worth, is that right?

Are all these statements correct?