Seeking Information About the LookupPolicy Property of the BarsRequest Class

[NtFan]

I'm running NinjaTrader version 8.0.22.2 64-bit connected to Kinetick via IQFeed, and this inquiry is related to my use of the BarsRequest class in a custom Add On that I've developed. Note that for the purpose of this inquiry, I'm specfically interested in historical data for daily bars.

Here's what the Help Guide topic for BarsRequest states about the LookupPolicy property:

The lookup policies for the bars request. Possible Values are:
> Provider - Queries the provider. The repository is updated on provider's reply
> Repository - Looks up the local repository only

And here's a set of questions that showcase the information I seek.

Q1. What is the default value for LookupPolicy?

Q2. Does "Provider" mean the "Preferred connection" that I've set via "Tools > Options > Market data > Preferred connections - historical" for the instrument type of the BarsRequest? If not, what does "Provider" mean?

Q3. Does "Repository" mean the files that NinjaTrader keeps in the "Documents > NinjaTrader 8 > db" folder on my computer? If not, what does "Repository" mean?

Q4. If LookupPolicy is set to Provider, is it assured that the Repository will be bypassed completely even if it contains data for some or all of the bars within the FromLocal .. ToLocal span of the BarsRequest?

It's Q4 that really gets to the heart of the information I seek. I say this because my goal is to deal with stock splits that occur today or on any day in the future, and to find a foolproof way to query for historical data for daily bars that is split-adjusted for the full FromLocal .. ToLocal span of the BarsRequest.

Thank you very much in advance for any insight you might provide.

[NinjaTrader_Jim]

Hello NtFan,

Thanks for your questions.

Q1. What is the default value for LookupPolicy?

Through testing, I see Provider behaviors occurring when I do not specify a LookupPolicy. (If I delete data in Documents\NinjaTrader 8\db\tick\ and then make a BarsRequest for tick data, the historical data is redownloaded.)

Q2. Does "Provider" mean the "Preferred connection" that I've set via "Tools > Options > Market data > Preferred connections - historical" for the instrument type of the BarsRequest? If not, what does "Provider" mean?

Provider would mean that the request for data will go to the data provider. This would be your data provider, or more specifically, you you have configured as the preferred historical data provider you have for that asset type.

Q3. Does "Repository" mean the files that NinjaTrader keeps in the "Documents > NinjaTrader 8 > db" folder on my computer? If not, what does "Repository" mean?

Yes, this would refer to the historical data repository where you mention.

Q4. If LookupPolicy is set to Provider, is it assured that the Repository will be bypassed completely even if it contains data for some or all of the bars within the FromLocal .. ToLocal span of the BarsRequest?

Yes, the data retrieved from the data provider will overwrite the data in the repository.

Please let us know if there is anything else we can do to help.

[NtFan]

Thank your for your response, NinjaTrader_Jim. I've resumed work related to this issue, and my experimentation is leading to further problems/questions, as presented below.

The key part of the NinjaScript for my NinjaTrader AddOn Test procedure is pasted below.

string bTokSymIQ;
BarsRequest bBrRq;
DateTime bBrRqDtFr, bBrRqDtTo;

bTokSymIQ = "A";

bBrRqDtTo = DateTime.Now.Date;
bBrRqDtFr = bBrRqDtTo.AddDays(-10);

bBrRq = new BarsRequest( Instrument.GetInstrument( bTokSymIQ, false ), bBrRqDtFr, bBrRqDtTo );

bBrRq.BarsPeriod = new BarsPeriod { BarsPeriodType = BarsPeriodType.Day, Value = 1 };
bBrRq.LookupPolicy = LookupPolicies.Provider;
bBrRq.TradingHours = TradingHours.Get( "US Equities RTH" );

try
{

bBrRq.Request(new Action<BarsRequest, ErrorCode, string>((bars, errorCode, errorMessage) =>
{

for ( int i = 0; i < bars.Bars.Count; i++ )
{
mMsgLn = string.Format( "{0}: {1} {2:F2} {3:F2} {4:F2} {5:F2} {6:N0}", bTokSymIQ,
bars.Bars.GetTime(i).ToString("MM.dd.yyyy"), bars.Bars.GetOpen(i),
bars.Bars.GetHigh(i),
bars.Bars.GetLow(i),
bars.Bars.GetClose(i),
bars.Bars.GetVolume(i) );
NinjaTrader.Code.Output.Process( mMsgLn, PrintTo.OutputTab2 );
}

}
));

}
catch
{
mMsgLn = string.Format( "BarsRequest failed for {0}.", bTokSymIQ );
NinjaTrader.Code.Output.Process( mMsgLn, PrintTo.OutputTab2 );
}

Notice that my BarsRequest is for Instrument Name = A, it uses LookupPolicy = LookupPolicies.Provider, and it writes name, date, open, high, low, close and volume data to the Output window for each bar returned by my BarsRequest. The behavior that this Test procedure exhibits is unexpected and undesirable, as follows...

1. The attached file named "1HstDat_A_Before.PNG" shows what the repository contains for A before I run my Test procedure.
2. The attached file named "2HstDat_A_After.PNG" shows what the repository contains for A after I run my Test procedure.
3. The attached file named "3OutWndw_A.PNG" shows what my Test procedure writes to the Output window.

So, it appears that BarsRequest pulls data from the Repository for dates prior to today, and it pulls data from the Provider for only today. Can BarsRequest be made to pull data from the Provider for all dates and, if yes, how?

Thank you for any assistance you might provide.

[NtFan]

Please disregard the above; I've discovered what's preventing BarsRequest from pulling data from Provider for all dates, and I'll post an explanation shortly.

[NtFan]

Further experimentation demonstrates that BarsRequest will pull data from Provider for all dates when LookupPolicy = LookupPolicies.Provider, however, Provider data is overridden by Repository data for those dates for which Repository data is Edited.

For my earlier post (#3), I created the "Before" data using the Historical Data > Edit feature; notice that the attached "Before" and "After" images show the Status of the data is "Changed." Under these conditions, the data returned by BarsRequest is overridden by my Edits (except for the last day in the "After" data and the Output window, which I suspect is a BarsRequest bug).

I ran a similar experiment, this time creating the "Before" data using the Historical Data > Load > Import feature; as such, NinjaTrader considers the "Before" data to be unedited. Under these conditions, BarsRequest returns data for all dates, and it updates the Repository accordingly. The attached images demonstrate this behavior. Perfect.

Thanks much for your help with this inquiry.