Historical Data Manager Metastock Asci Files

[Iceman03]

Hi, Any help would be much appreciated

I currently have a esignal data feed which I am starting to use for Ninja Trader. At this stage I'm only using end of day data. However when i use the esignal or Kinetic feed for the market analyzer ( the market analyzer loads two indicators per stock) it can take hours to load the data when i have 8000 stocks present. Even after reducing the list to 1600 it can still take a good hour and sometimes the list does not load properly.

I though one way to get around this would be to import metastock or ascii data into Ninja Trader7 using the historical data manager.

However it seems that to load Metastock data, one needs to load each folder for each letter of the alphabet individually. This also takes a long time and would not be practical on an everyday basis.

My question is there any way to automatically update and import daily metastock data on an everyday basis without having to import the whole history of data and without having to do it by individual folder of the alphabet?

If not is the same possible using ascii data?

Lastly the data i import seems to be overridden by kinetic or esignal data that i have used in the past. I think this is because some of that data is stored by Ninja Trader in cache if understand this correctly. Is there any way to stop this?

[NinjaTrader_Matthew]

Hello,

Thank you for your post.

There is not a way I'm aware of to load MetaStock data automatically each day. However, when you load eSignal or Kinetick data, you should only see the data that is missing loaded at this time. Please see our Help Guide article on 'When does NinjaTrader load historical data?' for more information:

http://www.ninjatrader.com/support/helpGuides/nt7/index.html?historical_data.htm

You can also change the way the data is loaded by going to Tools--> Options--> Data tab--> uncheck "Get Data from Server (if available)". Disabling this option will prevent the automatic fetching of historical data from the servers and should be loaded from your local database first. In in order to fetch the most recent data from the servers, you would either need to reload this from a chart, or by using the Tools--> Historical Data Manager--> Reload tab.

Are you sure it is the loading of data that is taking long for these symbols to populate? Do you have any custom or system indicators loading in the market analyzer? If so, what indicators do you have loaded?

In addition, please right click on the market analyzer--> select "properties"--> set the Days to load to a lower value and see if this improves.

[Iceman03]

Hi Matthew,

Thanks for your fast response
Yes I'm loading two proprietary indicators and that is most likely reason slowing things down. ( i need to load at least 30 bars for this indicator)

But the market analyzer seems to be very unresponsive for me

Even when i just put in a list of symbols in the market analyzer and press F5 to refresh i can't seem to get it load the last prices when connected to esignal on demand

I'm wondering if I import ascii files on an everyday basis will Ninja Trader know to only update the last bar?

or will it import the whole history everyday?

Also would there be a need for a new ascii file for each stock each day or should each days data be added to the existing historical ascii file for each stock?

[NinjaTrader_Matthew]

Hello,

It will import the whole history from the file you are importing.

I would suggest using each days data to the existing file for each stock.

[Iceman03]

Thanks Matthew,


I was wondering if you can think of any other reason why my market analyzer is unresponsive.


Like I said when I load the market analyzer with symbols only I can't even get the last prices to upload.


Also when I re open Ninja Trader on a new day the market analyzer appears with all the symbols and indicator values instantly, however these values are from the previous trading day(not the one just completed). As soon as I connect to esignal I get the same problem of a real slow download.


Any ideas would be appreciated.

[NinjaTrader_Matthew]

Hello,

If you are loading your Market Analyzer with indicators, you may want to see these indicators to 'Calculate On Bar Close = True' from the Indicators Column Parameters.

This will ensure your indicator values are only updating once per bar rather than continually updating which can drain CPU resources.

[Iceman03]

Unfortunately this does not solve my problem
But thank you for your help.

[NinjaTrader_Jason]

Hello,

Can you please set the amount at '# of bars to look back' for the indicator columns to the minimum number of bars required to properly calculate the indicator. It will take longer to load data and use more RAM if you are using a high number.

If this will not help, I suggest to load less instruments or columns in the Market Analyzer.

[Iceman03]

Hi Jason I've got the number of bars to look back at 30 which is the amount i need for my indicator.


CPU and Ram should not be a problem for me as I'm running a 2nd generation i7, 3.4ghs with 16 GB of ram


Setting Calculate on Bar Close = True ( what Mathew suggested ) won't work as i need the value for the last bar


I ran the market analyzer today and it took around 50 minutes to fetch the data for 1600 symbols which i can live with.


However i can't find the reason why the Last Price field dose not upload?

[NinjaTrader_Matthew]

Hello,

Were you testing this with imported historical data, or were you connected to your data provider?

If so, who were you connect with at that time? Do you have any symbol limits from these providers?

[Iceman03]

Hi Mathew
At the moment I'm still trying to see if I can use my esignal data feed, so i was connected to esignal on demand.


I noticed that I do get an error message from the Esignal Data Manager Notification
"Error
Symbol Limit Exceeded"


It does seem that esignal may have a 500 symbol limit all though on their website it says that there is no symbol limit for the end of day data package I'm subscribed to (esignal on demand). I'm currently speaking with esignal to clear this matter up.


In any case i don't think this will solve my speed issue for fetching data however it may allow me to receive the last price and other data points in the market analyzer.


The strange thing is, even when I get this symbol limit error in the esignal data manager i can still slowly fetch the data for the indicator points in the market analyzer but the last price and other columns e.g. high, low, net change, show nothing.

I'm wondering how does Ninja Trader fetch data from the esignal server? I see that when i connect to esignal on demand in the bottom right hand corner of Ninja Trader it says loading.



Does Ninja Trader load data for all my workspaces on intial connection?


If so, if i have a market analyzer window open in one of my other workspaces with more than 500 symbols or i have lots of charts saved in my workspaces, is Ninja Trader going to try to load the market analyzer and charts with data even though the workspaces are not currently open?

[NinjaTrader_Jason]

Indicators are calculated on historical data that is present and subsequently display their values. If you create a Market Analyzer without being connected and add an indicator column, you will see indicator values for instruments you have historical data present.

Yes, once connected to a data feed, it will load data in all windows you have opened including workspaces opened in the background.

Please go to Control Center-->File-->Workspaces and check what workspaces are listed below 'Save Workspace As'. The workspace with the checkmark is the workspace opened on your monitor. Any additional workspaces listed, are opened in the background.

[Iceman03]

Hi Ryan,
Ok so I've closed all the work spaces but one and loaded less than 500 symbols into the market analyzer (Nasdaq100 stock list) the open high low and volume seem to come up instantaneously, my indicators load one by one and take some time to load which is ok.

But the last price and net change does not load at all, is there any reason why this may be happening?

[NinjaTrader_Matthew]

Can you please email your current workspace file so I can test these results on my end?

To save a copy of your workspace, please go to File--> Workspace--> "Save Workspace As..." and Save this workspace with the name "Support".

Please locate and attach this workspace XML document in an email with 'Attn: Matthew' in the subject line and a reference to this thread.

[Iceman03]

Hi Mathew,

I followed your instructions and deleted all files in the cache folder. Unfortunately this did not help. Net change and Last Price still don't upload in the market analyzer.

Yes I'm running NT V7.0.1000.7

As for my PC I'm running an i7 3.4Ghs with 16Gb of ram.