Tick data in Historical Data Export does not match data in Chart

[Derjan]

Hello,


I exported the last 3 days of Historical Tickdata (DataType: Last) of the FDXM 09-20 (Dax Mini Future), but when I compare the chart with exported data it seems either some ticks data is missing or the timestamps exported are not correct:

See my screenshot: In the chart there is a price of 12872 at 18:09:43 14.08.2020, but it is not included in the exported data. It should at least be found between 18:08:29 and 18:10:00.

Derjan.

[MojoJojo]

The price is also off, most likely the displayed data is constructed from a different contract month. I don't believe there is anything wrong.
Probably you're expecting to get merged data, which has to be implemented manually outside of NT unless some option exists. Another alternative is to write a data exporter.

[NinjaTrader_Kate]

Hello Derjan,

Thank you for your post.

Exported data will be in the UTC time zone. What time zone are you in?

Thanks in advance; I look forward to assisting you further.

[Derjan]

Hello Kate,

currently I'm in MESZ (summer time). I found the tick data is shifted by 2 hours. So this confusion is eleminated, thanks.

Now I want to export 2 years of tick data. But it seems I can only export/download data of the last 2 months: Although I select 01.01.2018 for "Start Date" the oldest data in the resulting file is from 15.06.2020. But when I open a chart and choose a start date in 2019, the tick data is loaded, so historical export should yield much more than 2 months, as the source tick data is definitely available!

Derjan

[NinjaTrader_Kate]

Hello Derjan,

Thank you for your note.

What's the end date for the chart you're loading when you start the chart in 2019? Do you see data for the full time period if you load a tick based chart from 1/1/2020 through today's date? Most data providers have limits on the amount of tick data you're able to request. For example, the NinjaTrader Continuum historical data servers hold roughly 90 days of tick data.

Thanks in advance; I look forward to assisting you further.

[Derjan]

The end date is 17.08.2020. My data provider holds tick data from the year 2002 until now.

[NinjaTrader_Kate]

Hello Derjan,

Thank you for your reply.

Are you exporting all contracts covered by the time frame you're attempting to export data for or just trying to export, for example, ES 09-20 from today back 2 years? If you export the necessary contracts individually do you get the appropriate data for the contracts within that time frame?

Thanks in advance; I look forward to assisting you further.

[Derjan]

I'm just trying to export FDXM 09-20 from today back 2 years. When choosing the prior contract FDXM 06-20 with Start Date for example 01.05.2020 and End Date 18.08.20, I get data from 04.05.2020 until 17.06.2020

[NinjaTrader_Kate]

Hello Derjan,

Thank you for your reply.

If you look under Tools > Historical Data > Edit, expand the Historical section, locate FDXM 06-20, expand that, click on last to expand that, and continue expanding down until you see Tick data listed, do you see data for all days in the time frame between 06-19 and the start date you are trying to export?

If not, please attempt downloading the same time frame of the same contract from your data provider via the "Load" tab at the bottom of the Historical Data window. After downloading, do you see the full amount of data?

Who is your data provider?

Thanks in advance; I look forward to assisting you further.

[NinjaTrader_ChelseaB]

Hello Derjan,

The historical data text files do need to be specific to the expiry contract month. This is also true when importing.

This is because each expiry contract month trades at different prices as if they were different instruments.

NinjaTrader uses a merge policy for charts and strategies, which is controlled by the rollover dates.

https://ninjatrader.com/support/forum/forum/ninjatrader-7/strategy-development-aa/96311-backtesting-choosing-contract-month?p=782986#post782986

When exporting you will need to export the expiry contract months separately, and I would recommend using the rollover dates for the start and end date.

[Derjan]

Ok, I understand. So this will be quite tedious if you need serveral years back.
Probably a better option would be to write some data exporter, as the user "mojoJojo" suggested above.
Anyway thanks for your support!

[MojoJojo]

Imo the most efficient way for a cutom data export is to use BarsRequest. It's not as straight forward as writing the data from an indicator's OnBarUpdate but processing a lot of ticks by loading them into a chart would waste a lot of resources(and might fail). It could be feasible though for less active products like the FDAX.

One trick to force BarsRequest to process synchronously if asynchronous processing is not ideal is to use something like AutoResetEvent and wait for it in the main thread.

[Derjan]

Thanks MojoJojo, I got it working using BarsRequest. It wasn't even harder than with OnBarUpdate. One problem remain regarding hwo to get the Ask Bid values, but I'll put this in another thread.