find historical data gaps

[ch9090]

Dear Support,

Here's my challenge of the day.

I've successfully imported 1800 stocks and 5 years of historical daily data. I know ca. 40 stocks do not have 2010 data, another 50 do not have 2009 data, etc.

I would like to identify these stocks and remove them from my instruments lists because they create problems during backtesting my strategy.

I can use the historical data manager and go one by one but I'll still be at it next week.

Do you have any idea how it could be done in a clever and quick way? The challenge of finding what is missing is not obvious to me.

Have a great Friday,
chris

[NinjaTrader_Jason]

Hello Chris,

Can you please tell me how you imported the historical data. Is there a possibility you can use the files, which you used to import the data, to identify these stocks?

Unfortunately I cannot think of a better way to identify these stocks within the software. Perhaps opening a chart and cycle through the instruments by typing in the stock symbols could be quicker, but I doubt it.

I am not a programmer, so I am not sure if it is possible by creating some custom indicator. Maybe other customers will have any suggestions in this regard.

[ch9090]

Hi Jason, indeed I can print the db\day directory content to identify the missing years but it is time consuming. There might be a code to spot something that is null but nothing comes to my mind right now.

I've searched the forum but did not found anything. Maybe a programmer will have a brilliant idea. Let's see.

Thank you for the quick reply. Chris

[koganam]

Dear Support,

Here's my challenge of the day.

I've successfully imported 1800 stocks and 5 years of historical daily data. I know ca. 40 stocks do not have 2010 data, another 50 do not have 2009 data, etc.

I would like to identify these stocks and remove them from my instruments lists because they create problems during backtesting my strategy.

I can use the historical data manager and go one by one but I'll still be at it next week.

Do you have any idea how it could be done in a clever and quick way? The challenge of finding what is missing is not obvious to me.

Have a great Friday,
chris

1. Write an indicator using GetBar() for dates between, say Jan 6 and Dec 20 of the years that you are testing for. If you take the difference between those barNumbers and it is zero, then it means that there were no dates in that period, and it is probably safe to say that there is no data.
2. Load the stocks and indicator into the MarketAnalyzer so that you can filter out the stocks that meet the criteria.
3. Manually confirm that the stocks have no data for the period.

[ch9090]

sounds like a good idea to me. I'll try it and post the code here if successfull. many thanks for pointing me in the right direction. chris