How to make NT save historical data from a external data feed?

[ntdev]

So I've implemented a custom external data feed but it only shows live data in my chart.

Coming back to the same symbol starts again with empty screen and only newly generated ticks are displaying.

I've looked through the .\Documents\NinjaTrader 8\db and can't find any .ncd files for my new symbol but it did create directories.

.\Documents\NinjaTrader 8\db\day\TEST
.\Documents\NinjaTrader 8\db\minute\TEST

Is there any trick to send the data in a way to make Ninja Trader actually save the data as historical so that it will be available later?

https://ninjatrader.com/support/helpGuides/nt8/?external_data_feed_connection.htm

int Last(string instrument, double price, int size)

Sets the last price and size for the specified instrument. A return value of 0 indicates success and -1 indicates an error.

So far, I have only called Last()

[NinjaTrader_Kate]

Hello ntdev,

Thank you for your note.

The DLL interface does not allow for sending historical data to NinjaTrader. However, if you go to Tools > Options > Market Data, there is an option to record live data as historical, which would allow you to save the data as historical and then when you reload the chart, it would be able to use that data you recorded to populate the chart. It's worth noting here that data would only be recorded when NinjaTrader is open and the live data is coming in.

Please let us know if we may be of further assistance to you.

[ntdev]

Thanks Kate.

I then see the Warning, "...Enabling this option while also using a historical data provider is not recommended as it may result in data gaps."

So, that leaves me more questions....

Q1. Will turning this on then cause gaps for historical data in other instruments that come from Continuum? Or will it only come into play for instruments without historical data, like the new external data that my DLL creates?

Q2. Is there a way to preload the data for past time?

One option that I see there is to write my own .ncd files.

I already have working code that parses the .ncd files so it theoretically possible to write them as well, even if Ninja Trader doesn't share that format.

Q3. Will NT automatically look for the .ncd files for the given hour for my newly added instruments?

Q4. Or can I just fake that there's data and handle myself in OnBarUpdate()?

Another option is to load my own historical data from my own cache + the Internet and then have OnBarUpdate() get the data from my own object but how to cause OnBarUpdate() to be called for historical data on an externally added instrument? My understanding is that OnBarUpdate() would only be called when there's data, not to get data.

So how to make NT think that there is data and therefore call OnBarUpdate()?


Do you have any other suggestions?

[NinjaTrader_Kate]

Hello ntdev,

Thank you for your reply.

Turning this option on will record incoming data as historical for all instruments. This will keep NinjaTrader from querying the Continuum historical Data server for all instruments when set to record.

You could import historical data for the instrument, that would be an option for you, however, you'd need to consistently be importing it each day. This would eliminate the need to write .ncd files as these would be created when you import the data.

NCD files represent data that is already on the platform. None of that is documented so we would recommend creating historical data files and then importing them.

There would not be a supported method to fake that there's data and trigger OnBarUpdate.

Since you're already pumping data to NinjaTrader from your own application, we would recommend building historical data files from your app and manually importing the data.

Please let us know if we may be of further assistance to you.

[ntdev]

Turning this option on will record incoming data as historical for all instruments. This will keep NinjaTrader from querying the Continuum historical Data server for all instruments when set to record.

So, that's a non-starter.

There would not be a supported method to fake that there's data and trigger OnBarUpdate.

Is there an unsupported method? Or a hint on the function name and I'll find the signature.

===========

https://ninjatrader.com/support/help...HistoricalData

I didn't remember see "Import for historical data" before... That might work if....

2 1/2 questions.

#1. Can this import historical feature be controlled programatically? Is there an api or function name?

#1b. If not, can I automate NinjaTrader from the outside? Has anyone taken control by clicking and sending keystrokes to it?

#2. Does import do a merge or an overwrite? For example, say that I had 10 minutes of historical data where 5 is already there in the .ncd file and 5 of it is new?

[NinjaTrader_Kate]

Hello ntdev,

Thank you for your reply.

I am not aware of an unsupported method to trigger OnBarUpdate with no data.

The import function is not available through the API, no. You may be able to use an external program to automate mouse clicks, but that would also be unsupported and I would not have any suggestions on a program to use for that.

Importing new data would overwrite the old data, yes.

Please let us know if we may be of further assistance to you.

[ntdev]

OK. So, I'll be done in a few hours with the ability to write .ncd files and that works great for historical data.

BUT I have a new problem with External Data that I overlooked before -- real time data.

Is it true that turning on an external data source is a blocking operation?



It must be possible to use an external connection along with my other connections, right?

If not, then what is the point of the external connection? Or of listing multiple connection mappings for each instrument?

Thanks in advance for your help with any suggestions.

EDIT: Note that I am only connecting to the external connection for data and not for trading. Maybe there's a read-only external connection setting which is allowed to connect at the same time as Continuum?

[NinjaTrader_Kate]

Hello ntdev,

Thank you for your reply.

It would not be possible to connect to additional data connections if you are connected to the External connection. I don't have any information as to why that is the case other than it was not built to handle multiple connections, but my suspicion is that it's due to the wide array of data connections you could theoretically make with the External connection. Generally most users would use this connection to receive real time data from an outside source or program via the Ask and Bid DLL interface functions.

I have added a vote for you to a current feature request to allow other connections when connected to the External connection. This request is being tracked under the number SFT-2510.

As with all feature requests, interest is tracked before implementation is considered, so we cannot offer an ETA or promise of fulfillment. If implemented, it will be noted in the Release Notes page of the Help Guide.

Release Notes — https://ninjatrader.com/support/help...ease_notes.htm

Please let us know if we may be of further assistance to you.

[ntdev]

I was afraid of that -- External connections are almost entirely (actually entirely) useless for anyone using a NinjaTrader broker to make trades.

BUT I am so close!

>> Can you think of any workaround? Please help me again think outside the box.

Also, please, can you tell me how I can have 50 other people contact you to make SFT-2510 move up in the priority list?

One work-around, since I can write .ncd history files, could be to just force a reload of the chart and that gets the data from history but all my volume profiles don't calculate so quickly....

>> Can I pragmatically call for an update in just one indicator? It's a little dirty but I could update the history files every minute and reload the indicator and it would be almost like real-time.

[NinjaTrader_Kate]

Hello ntdev,

Thank you for your reply.

I apologize for the delay in my reply as our forum was down for a few days.

I am unaware of any supported or unsupported workaround for this.

Forum users can reply to this post or make their own post to ask that their vote be added to SFT-2510, or users may email their vote in to platformsupport [at] ninjatrader [dot] com and reference the SFT-2510 number. Please be aware that many factors go in to whether a particular feature request is granted, and there is no specific number of votes that guarantee a request will be added to the platform.

It wouldn't be possible to just refresh one indicator manually, but you could programatically try something like this example one of my colleagues posted on the forums to refresh the values the indicator is using:

https://ninjatrader.com/support/forum/forum/ninjatrader-8/indicator-development/96415-reloading-indicator-from-code?p=783500#post783500

Please let us know if we may be of further assistance to you.