Accuracy of Tick data in Format (Second Granularity)

[mafon]

I use Future Historical Tick data 10 years back with Tick Format (Sub Second Granularity) described here https://ninjatrader.com/support/help...?importing.htm
to perform backtesting with Range bar Data Series and Historical fill processing High and 1 tick value.
Everything works fine so far.

I want to add and purchase more historical tick data that is older than 10 years.
However this is only available in Tick Format (Second Granularity) yyyyMMdd HHmmss;price;volume .

Can I expect the same accurate range bars and backtesting results in NT8 with Tick Format (Second Granularity) data for the time period before 2011 or are there any disadvantages?

[NinjaTrader_ChelseaB]

Hello mafon,

Some of the ticks may have shared timestamps. This can cause some ticks to be included with different bars than they would have in real-time.

Below is a link to the help guide on this. Please see the heading 'Shared Timestamps'.

https://ninjatrader.com/support/helpGuides/nt8/index.html?multi-time_frame__instruments.htm#WorkingWithMultiTimeFrameObjects

[mafon]

Hello Chelsea,

thank you for the link.
So if I only use a single Range bar data series or backtest with Oder Fill Resolution 'Standard' everything should be fine.

However if I backtest with Order Fill Resolution 'High' (with 1 tick second data series) 'Shared Timestamps' may lead to different fills than in real-time.


If I 'optimize' every 'Shared Timestamps' myself in assuming that in historical data file I want to edit/import every line corresponds with the correct sequence of the incoming ticks in real-time from oldest tick in the first line to newest in the last line.

That means
for Tick Format (Sub Second Granularity)

from
20061107 000431 1000000;1383.00;1
20061107 000456 1000000;1383.25;25
20061107 000456 1000000;1383.25;36
20061107 000456 1000000;1383.25;14

to
20061107 000431 1000000;1383.00;1
20061107 000456 1000000;1383.25;25
20061107 000456 1000001;1383.25;36
20061107 000456 1000002;1383.25;14


and for Tick Format (Second Granularity)

from
20061107 000431;1383.00;1
20061107 000456;1383.25;25
20061107 000456;1383.25;36
20061107 000456;1384.00;14

to
20061107 000431 0000000;1383.00;1
20061107 000456 0000000;1383.25;25
20061107 000456 0000001;1383.25;36
20061107 000456 0000002;1384.00;14


Or here the 'optimized' help guide example.




Would that optimization make backtesting results better or worse?

[NinjaTrader_ChelseaB]

Hello mafon,

As the range bars themselves are formed with tick data, they may include different different ticks and have slightly different volumes.

And yes, the tick an order fills with may be included with a different bar.

Usually it's very slight differences, but the page exists in the help guide to help explain why there may be differences.

Modifying the data may work, but you would need to modify it to match what actual timestamps were. Otherwise you also wouldn't know which ticks to include with which bar.

[mafon]

Hello again,

just to be 100% sure.

Please tell which of the four different following bar sequences will result to 'Shared Timestamps' whem using a secondary bars series and which not:

yyyyMMdd HHmmss fffffff;price;volume

1. 'Shared Timestamps' Yes or No?:

20061107 000431 4290000;183.25;1
20061107 000456 4300000;183.50;2
20061107 000456 4300000;183.25;4
20061107 000456 4300000;183.25;10

2. 'Shared Timestamps' Yes or No?:

20061107 000431 4290000;183.25;1
20061107 000456 4300000;183.50;2
20061107 000456 4310000;183.25;4
20061107 000456 4320000;183.25;10

3. 'Shared Timestamps' Yes or No?:

20061107 000431 4290000;183.25;1
20061107 000456 4300000;183.50;2
20061107 000456 4301000;183.25;4
20061107 000456 4302000;183.25;10

4. 'Shared Timestamps' Yes or No?:

20061107 000431 4290000;183.25;1
20061107 000456 4300000;183.50;2
20061107 000456 4300001;183.25;4
20061107 000456 4300002;183.25;10

[NinjaTrader_ChelseaB]

Hello mafon,

This question can only be answered if the bar type is known and the code in the bar type.

With standard minute bars, all of the timestamps will be within that minute bar no matter the millisecond.

With a standard range bar, the first tick with a shared time stamp may be included with the bar in real-time, in historical it may include all ticks with a shared time stamp. With a custom bar type it would be unknown.

[mafon]

MilliSecondYes, it is for standard range bars.

As I don't know the internal coding on what NT8 decides if a bar is a considered as a Shared timestamp, let me ask in code.

Does NT8 identify Shared timestamps figuratively for two consecutive bars
by doing this only till milliSecond accuracy:

private bool IsSharedTimestamp(DateTime barTime0, DateTime barTime1)
{
barTime0.Year == barTime1.Year &&
barTime0.Month == barTime1.Month &&
barTime0.Day == barTime1.Day &&
barTime0.Hour == barTime1.Hour &&
barTime0.Minute == barTime1.Minute &&
barTime0.Second == barTime1.Second &&
barTime0.MillSecond == barTime1.MillSecond
}

or this with whole tick accuracy ?:

private bool IsSharedTimestamp(DateTime barTime0, DateTime barTime1)
{
return barTime0.Tick == barTime1.Tick;
}

[NinjaTrader_ChelseaB]

Hello mafon,

A DateTime from historical data will have granularity down as far as .Millisecond.

https://docs.microsoft.com/en-us/dotnet/api/system.datetime.millisecond?view=net-5.0

The data will not be as granular as .Tick, one ten-millionth of a second.

https://docs.microsoft.com/en-us/dotnet/api/system.datetime.ticks?view=net-5.0

[mafon]

OK, this means that NT8 is theoretically capable of processing 100 nano second granularity values but at the moment the NT8 code only considers granularity down as far as .Millisecond !?

Does this also mean that data feed from IQFees/Kinetick which is offering for some exchanges microsecond timestamps granularity is only cosidered by NT8 as far as millisecond for real time and historical data ?

[NinjaTrader_ChelseaB]

Hello mafon,

Historical data from file is read up to millisecond granularity. Data received from any connected data feed will be received in millisecond granularity by the connection adapter.

[mafon]

Hello Chelsea,

typo or historical and real both the same granularity?

Historical data: read up to millisecond granularity
Real time: in millisecond granularity by the connection adapter

[NinjaTrader_ChelseaB]

Hello mafon,

Historical data from file will be granular up to millisecond granularity. It could be second granularity, It could be minute granularity. It could be day granularity. Depends on the file contents.

All data from a provider in NinjaTrader 8 will be millisecond granularity. Not less or more.

[mafon]

Thank you Chelsea,

everything clear now and good to know.

Are there any plans to consider finer granularity than milliseconds for historical and/or from a data provider in the future?

[NinjaTrader_ChelseaB]

Hello mafon,

I would not be able to discuss any future plans for the NinjaTrader Platform.