Recent historical data dissapeared when I disconnect from IB

[pstrusi]

Hi NT,
So here a curious situation. I connect to IB as trading and historical data server. So, to be able to download historical data I set these as following:
- In connections: I set my IB connection, "Use IB as historical data server"
- In options:
- Data tab / Historical chart data / I select: Get data from server ( if available )
- Data tab / Real-time data / I select Save chart data as historical.

So, with all these options selected to be able to get and save all data from IB in my database, this happens: When I connect all data is updated and confirmed that is saved in my database. But If I disconnect, then I confirm that at least the last 2 days of data got deleted from my database, this is quickly solved by connecting back to IB.
So my question is: how should I set all data historical options, above commented, in order to have saved permanently all historical data received?

Edit:
Koganam made me a smart suggestion when following many instruments. Just open a Market Analyzer window, linked to one single chart, so whenever I want to see a chart of an instrument, just one click and done. Since IB connection is set to download from server any update for the data, my question is: How can I set all of above sets, in order to have my database updated automatically?



Thanks!

[NinjaTrader_ChelseaB]

Hello pstrusi,

The 'Save chart data as historical' saves the real-time stream data as historical tick data.

Are you showing that the recorded tick data is being removed?

Can you reproduce this by connecting to record a short amount, restarting, and then check to see if the tick data is still there?

[pstrusi]

Hi Chelsea,

Well at this point I've already updated the database not I couldn't perform that task now, but I'll try it this week. Meanwhile I got a question in regards to Market Analyzer and the saving data process. As you know in the Market Analyzer you can have many instruments, all of them receiving data. I have it even linked to a Chart, so whenever I click in the chart, I can see right away the instrument evolution, then 2 questions:

1. If I don't click on any particular instrument in the market analyzer, therefore not visualizing its chart, is its data getting downloaded and saved in my database ?

2. Would this set be necessary in order to get and save all data correctly? Data tab --> Historical chart data --> Selected: Get data from server ( if available )
If it turns out that this set is necessary: why would it be? If I'm getting all the data real-time through the Market Analyzer ?

Looking forward to the wisdom
Thanks!

[NinjaTrader_ChelseaB]

Hello pstrusi,

There shouldn't be any reason you couldn't test this the moment the market is open.

Yes, real-time data received in the Market Analyzer would be saved as historical tick data.

The 'Get data from server' option is for downloading data from the connected provider. It is not used to save real-time data as historical data.

[pstrusi]

edited 7-1-2019:

Chelsea, whenever I click on an instrument in the market analyzer, its chart is shown updated, however the forex instruments are the only ones that chart begin with a single tick that if you leave that chart, it continues to build that daily bar from that point. My question, why only the forex ? and how to address this so forex instruments charts update as the rest ?

Thanks!

[NinjaTrader_ChelseaB]

Hello pstrusi,

I'm unsure. If the data is not delayed data (not showing a yellow dot in TWS or showing the delayed tool tip when hovering the mouse) and the chart is a minute or daily chart, TWS is version 960.2g, I would be expecting the chart to load.

May I confirm the data is not delayed data?

What version of TWS are you using?

What is the bar type of the chart?

[pstrusi]

1. My receiving data is always real-time from its original exchange
2. Tws 960.2g
3. OHLC

[NinjaTrader_ChelseaB]

Hello pstrusi,

I'm not sure what this means. Does that mean you have a real-time subscription for absolutely all instruments with Interactive Brokers?

Below is a video that demonstrates the expected behavior.

https://drive.google.com/file/d/1f7smCkQM_sQttLRSiP9KKLLU5a42SBFy/view?usp=drivesdk

Can you show a screenshot of the instrument row in TWS so we can confirm that you have paid for a subscription for that instrument?

[pstrusi]

- Chelsea, I have real-time data only on instruments that I follow, obviously.
- In regards to you asking me a screenshot to "confirm" real-time data status, sounds silly knowing that I have years working with IB and NT.

Please don't ask more silly questions and investigate something that is real. You could have seen this in your very same frontend by now.

[NinjaTrader_ChelseaB]

Hello pstrusi,

I am able to load historical minute and day data for forex instruments on my end as demonstrated in the video.

Perhaps this means the issue is on your end?

Unfortunately, I am only able to use a process of elimination with involved items to identify an issue and resolve it.

Involved are the subscription, data type, symbol map, and version numbers.

[pstrusi]

Below you can find an example. This situation occurs when you don't have selected Get data from server' option, because allegedly all the data is saved in the database, which is being received in the market analyzer. In order to prevent that, I came back to select Get data from server' option and hope not to have the original issue of this post.

[NinjaTrader_ChelseaB]

Hello pstrusi,

The chart is a daily chart and is not a tick chart.

Real-time data is only saved as tick data, it is not saved as minute or daily data.

If 'Get data from server' is unchecked and is not enabled, historical data will not download.

With that option enabled, are you finding historical daily data is not downloading?

[pstrusi]

Chelsea,

Obviously, a daily chart is built only on daily data bar, not tick data. What I meant by the attached file was that you might notice how instead seeing a regular daily bar, you see only a daily bar that begins to build itself after you've called that instrument in the market analyzer.

I'm aware that real-time data is saved as tick data. Did I say the contrary?

These are the non-excluding ways to have saved data in your database:
1. You have an opened chart for each one of the instruments that you follow
2. You have a Market Analyzer window with a linked chart and selected the option "Get data from server".

No need for further help.