Building My Own Historical Tick Database

[heyligerb]

Hi there...

I'd like to build my own historical tick-by-tick database for multiple futures symbols for backtesting purposes using the NT 7 replay feature.

What I'd like to do is have NT record each tick that come across the data feed over a period of days/weeks/years, and have that data placed in the historical database for backtesting at a later date.

How can I do this is a way that is as hassle free as possible? I'm familiar with how to record the data using replay, but how can I import the data into the historical database so that it is there for backtesting once I have recorded the data?

I'd like to set this up in a way that is as hassle free as possible, so that it does not require me to

Basically I just want to set it and forget it... I don't want to have to manually do anything each day/week etc...

Is this possible?

Thanks,

Brian

[NinjaTrader_Austin]

Brian, it is indeed possible to record both replay data and historical tick data. Please use the settings shown in the screenshot to do so.

As for how to actually get the data coming in, there is a strategy sample on the forums that goes something like this:

protected override void Initialize()
{
    CalculateOnBarClose = false;

    Add("ES 09-10", PeriodType.Minute, 1);
    Add("YM 09-10", PeriodType.Minute, 1);
    Add("TF 09-10", PeriodType.Minute, 1);
    Add("ES 12-10", PeriodType.Minute, 1);
    ....
}

The purpose of this strategy is to load up as many instruments as you want to record in one go, so you don't have to bring up a chart or anything for each instrument.

Another option would be to put all the instrument into the Market Analyzer, and then add an indicator column to the MA, which would start the recording of the data.

[heyligerb]

This is great! Thanks...

One more question... just to make this process a little more streamlined. Do you have any idea how I can get Ninja to capture the data on the ES ##-## contract rather than the front month? That way I won't have switch the contract every few months, and it will make back testing a little easier as well.

I'm sourcing the data in TradeStation from the continues contract (@ES), but NinjaTrader maps it to the front month (ES 09-10). If I keep it this way I have to manually adjust the front month at rollover, and I'm trying to avoid having to do this.

Anyway to accomplish this?

Thanks,

Brian

[NinjaTrader_Bertrand]

Hi Brian, unfortunately we're not aware of a workaround for this - the ##-## expiry is only used to access natively supported continuous contracts from 3rd party datafeed providers.

[heyligerb]

Okay... well... is there a way to have NinjaTrader always map to the same month no matter what front month is? For instance have it always map to ES 09-10, even when we roll to the next month?

[NinjaTrader_Bertrand]

You should be able to map the contiuous contract to the current front month by using just the TS base symbol map under 'External' (ES for example).

However, we normally don't recommend doing so as it would provide you with more control to explicitly map to the new front month as the rollover occurs as you would like to handle it.

[heyligerb]

I'm concerned primarily with the symbol the market data is recorded under in Ninja Trader, not the symbol sourced form the data provider.
.
What has happed in the past when I tried this, is when rollover is happens, Ninja stops recording the data because it now wants to record the data under the current front month.

For instance, I'm recording from the continues contract now (source:@ES to destination: ES 09-10 in Ninja). However, when we roll over, the data will stop being recorded, and will require me to manually add the following front month, Then for back testing, I have to manually merge everything into one contract (if that's even possible).

I'd just like to keep everything under one contract in Ninja Trader, and not have to worry about manual intervention when it comes time for rollover. That can become quite cumbersome when dealing with many different markets.

Is this possible?

[NinjaTrader_Bertrand]

Brian, unfortunately this is not possible - TS data would not feed the ##-## contracts in The best choice would then be using a data provider that support either support those natively or using the NT regular expiries and working with our automatic continuous contract features now available on NT7. For Market Replay data - you could download the recorded data from our servers directly without the need to record it yourself, please just go to File > Utilities > Download Market Replay Data in NT7's Control Center (no connection needed).

[heyligerb]

What is the " automatic continuous contract features now available on NT7?"

[NinjaTrader_Bertrand]

You would chart the front month and NT7 autoMerge's (according to merge policy setting for the symbol in the Instrument Manager) historical data by accessing the expired ones resting in the database / servers. The rollover dates are preset by us in the Instrument Manager (Misc section of the symbol), however those can be customized by the user as well offering the creation of custom continuous contracts as needed.