problem loading historical data with MB Trading

[neptune]

Hello,


I use ninja with MBTrading data and I’m having trouble with historical data for two different forex instruments (NZDUSD AND EURGBP). I did a reload data on these instruments and noted missing data subsequent to the reload. At that time I went into historical data manager and deleted the entries in the tree for both of these instruments thinking that when I reload data a complete reload would be forced. In the Data Series dialog I have set up both instruments for 20 minute bars and 100 days to load and this has always worked fine. After deleting the entries in the data manager tree now I only get data loaded starting today (7/24/11) at the 2:00 PDT forex open. How do I force the complete data set for 100 days that I am requesting to be reloaded?

Thanks

[NinjaTrader_Jason]

Hello neptune,

Can you please restart NinjaTrader and create brand new charts for the instruments and check if the requested data is loaded.

If not, please check if historical data is enabled as per your MBTrading connection. Please go to Control Center-->Tools-->Account Connections, highlight your MBTrading connection and click Change. Click Next and check if 'Historical data' is enabled. If not, please enable it and finish the connection. Connect to MBTrading and check if the requested data is loaded.

If the issue remains, please tell me if you experience any error messages and tell me what they state.

[neptune]

Hello Jason,

I tried your suggestion of creating a brand new chart with no success. I checked the connection and historical data is enabled. Be advised that I have had charting working I believe correctly for quite some time and have been able to view historical data.

Also, I am noting that the problem is more extensive than first identified. I see it in many other instruments but related to timeframe. I have the following:

data looks correct for all instruments in the daily timframes
NZDUSD AND EURGBP all timeframes, data only loads from forex open 2:00 PM PDT (west coast US timezone), until present.
EURUSD - 20 min timeframe appears correct. all other minute timeframe data is missing from 7/14 - 7/21.
AUDUSD - data looks good for all minute timeframes.
USDCHF - 20 min timeframe appears correct. all other minute timeframe data is missing from 7/13 - 7/21.
USDJPY - 20 min timeframe appears correct. All other minute timeframes all data missing prior to 7/21. Data from 7/21 to present looks correct.
GBPUSD- data looks good for all minute timeframes.
USDCAD- data looks good for all minute timeframes.
EURJPY - 20 min timeframe appears correct. all other minute timeframe data is missing from 7/8 - 7/13.
EURCHF - All minute timeframes all data missing prior to 7/20. Data from 7/21 to present looks correct.

as you can see this is not usable in current state. I would be interested in completely clearing the historical data cache if that is possible. It seems that it might be messed up. Is there a way to do that? I have tried reload of individual charts / reload all in the historical data manager a number of times with no luck. Any other suggestions? Please advise.

thanks.

[NinjaTrader_Jason]

Do you experience an error message when historical data is not loaded? If so, please tell me what the error states.

Please send me your latest log and trace files. You can find these files at the following locations.

C:\-->Users-->*Windows user account*-->(My) Documents--> NinjaTrader 7--> Log--> log.20110725 for today. (log.yyyymmdd for any other date)

C:\-->Users-->*Windows user account*-->(My) Documents--> NinjaTrader 7--> Trace--> trace.20110725 for today. (trace.yyyymmdd for any other date)

You can send them to support[at]ninjatrader[dot]com. Please put 'Attn Jason' in the subject header and reference this forum thread.

[neptune]

Hello Jason,

I emailed attached trace logs as you requested. I do not experience any error message when historical data is loaded.

I also have noticed that there is an option in the control center under tools-> options-> data tab for Reset DB and Repair DB. I looked at the documentation for these, and there is indication that these "Should only be used at the request of a NinjaTrader technical support representative.". Are these worth a try? Thought I would ask before trying.

Thanks,

[neptune]

Hello Jason,

I neglected to send the trace in the first email I sent and only sent the log. I have sent a second email with both trace and log now. Thanks for any assistance you can provide.

[neptune]

I'd also be interested in knowing if other users are able to load forex historical minute data successfully. Anyone?

thanks.

[tgilligan]

I have experienced the same problem with MBTrading Forex minute chart data. Essentially it will now only download minute data for the current week. This is for all pairs that I have tried. It is also consistent across two PCs. On a quick test, daily data seems to be okay, and tick and range don't download any historical data all, which they used to.

Email exchanges with NinjaTrader Support suggested that something may have changed at MBTrading. I emailed MBTrading and just got a standard email back saying that NT is a third party product and MBTrading do not provide any support.

The timing of the issue arising seemed coincidental with the upgrade to NT7R6, so I downgraded one PC to 7R5. The issue was still there.

All this would indicate that the issue is at the MBTrading end. However, queries by this user have effectively been brushed aside. It would appear that what is required, is for whoever at NinjaTrader has a relationship with MBTrading, to contact them to resolve the matter.

Some progress would be appreciated. Thanks.

[NinjaTrader_Jason]

Hello tgilligan,

The MBTrading API recently contains limitations to help prevent users from making abnormally high numbers of requests for market data. You can only load so much historical data at a time with MBTrading. If you go over this limit, you will receive the 'errClientsideConnect' error in NinjaTrader when loading data.

MBTrading has limited the number of data requests you can have in one minute. The limit is currently set to 40. However, with tick data you can only download 25,000 ticks per request. NinjaTrader will make multiple requests of 25,000 to get all the tick data needed to build the tick chart.

What this means is that to build one tick chart you could have 40 requests depending on how many bars or days back you request. Therefor you could hit the 40 request limit with one single tick chart. When you hit the limit of 40 requests you will see an ErrClientSideConnect error. You will need to wait 1 minute and retry with a smaller request for historical data.

More information and an official response from MBTrading can be found in the public forum below.

http://finance.groups.yahoo.com/group/mbtsdk/message/8967

[tgilligan]

Hi Jason,

Thanks for the update. I haven't had any error messages - maybe they are just in a log somewhere.

I am not really concerned about tick data, though when I open a tick chart I get no histrical data whatsoever. So a long way short of 25,000.

I am more concerned with minute charts. If I start up NT and connect to MBT Forex, then open one new 1M chart I only get about 845 bars. Are minute bars being constructed from seconds? I thought not. In which case this is not much data.

I had a quick look at the public post from MBT. All too technical for me. The question that arises is, "Is NT now doing the correct thing to get historical data downloaded from MBT within their new constraints?" Based on what I am seeing, it doesn't look like it. Or are there some settings within NT that I need to change?

Thanks

[NinjaTrader_Jason]

Minute bars are created by minute data and not from seconds. Please see the link below under 'Understanding the underlying base data type required for constructing various chart bars' to see what type of data is used for each period type.

http://www.ninjatrader.com/support/helpGuides/nt7/how_bars_are_built.htm

Since you do not experience the error message, I do not think your issue is related to the limitation. Can you please send me your MBTrading login credentials, so I can test your connection on my end. You can send it to support[at]ninjatrader[dot]com. Please put 'Attn Jason' in the subject header and reference this thread.

[jokujoku]

About two weeks

About two weeks have been problem to get proper historical data from MBT
sometimes tick data burst, missing, sometimes minute data for EURUSD.
usually mondays are problem days but sometimes other days also.
quite often also GBPUSD.

I have deleted data and repair database sometimes works, but not help anymore.


These issues have been around more than half year
and its not limit of tick counts what mbt is giving, because i am not loading many days.

[NinjaTrader_Jason]

Hello jokujoku,

Can you please clarify what exact issue you experience. What exact data do you miss?

You mention you experience the issue about two weeks and more than half a year. Could you please explain?

How many instruments do you have selected in NinjaTrader once you connect to MBTrading. Could it be you have many instruments selected in NinjaTrader that cause 40 data requests? So when the EURUSD or GBPUSD requests data, you already had 40 data requests?

Try the following. Start NinjaTrader and do not connect to MBTrading yet.

Please go to Control Center-->File-->Workspaces and check what workspaces are listed below 'Save Workspace As'. The workspace with the checkmark is the workspace opened on your monitor. Any additional workspaces listed, are opened in the background.

Please select a workspace without the checkmark. Subsequently, go to Control Center-->File-->Workspaces-->Close Workspace. Repeat this procedure until only 1 workspace is opened.

Connect to MBTrading and check if all requested data is loaded. If not, please tell me how many windows the workspace contains that is left opened and how many instruments it consists.

[jokujoku]

I usually just use EURUSD, and sometimes work sometimes not.
and not loading more than one day.
and only one workspace

[NinjaTrader_Jason]

Do you notice any other consistencies when it does not work? Does the issue always occur on Monday?

Do you currently experience the issue? If so, please send me an email at support[at]ninjatrader[dot]com including your MBTrading username and password, so I can test it on my end. Please send me your workspace as well. You can find it at (My) Documents-->NinjaTrader 7-->workspaces. Put 'Attn Jason' in the subject header and reference this forum thread.