No data for first second on added time series

[gomifromparis]

Hi,

If I add a 1 sec time series
Add(PeriodType.Second,1);

And Print OnBarUpdates on a 1 tick chart
Print(BarsInProgress+"\t"+Time[0]+"\t"+Volume[0]);

Most of the time (for some bizarre reason sometimes it works) there is no 1 second data for the first second of the session

Example

0 09/05/2011 15:30:00 3
0 09/05/2011 15:30:00 2 (no BarInProgress=1 appearing as it should)
0 09/05/2011 15:30:00 4
0 09/05/2011 15:30:00 3
...blabla lots of ticks...
0 09/05/2011 15:30:00 3
0 09/05/2011 15:30:00 1
0 09/05/2011 15:30:00 1
0 09/05/2011 15:30:00 2
0 09/05/2011 15:30:01 20
1 09/05/2011 15:30:01 1692 here is the 15:30:01 data from the 1 sec time series
0 09/05/2011 15:30:01 1


The sum of those 15:30 missing ticks is 1990.


If we use a minute chart, here is what we get :

1 09/05/2011 15:30:01 1692
1 09/05/2011 15:30:02 20
1 09/05/2011 15:30:03 922
blablalostofticks
1 09/05/2011 15:30:58 15
1 09/05/2011 15:30:59 725
0 09/05/2011 15:31:00 10827

The sum of the second data is 8837, if we add the missing 1990 of the 15:30:00 second, we get the 10827 the one minute bar is showing.

If we use a 60 second chart, volume of first bar is 8837,meaning the first second ticks are not counted.

Whether I use 1 minute or 60 seconds, the volume of the first should be 10827, shouldn't it?

[NinjaTrader_Bertrand]

Thanks for the report gomi, we will look into it shortly.

[NinjaTrader_Bertrand]

Gomi, we ran tests here and cannot confirm your findings - the deal is that all ticks for example from 00.00 to 00.999 (the .999 representing the non accessible sub second tick timestamps) would be going into the 1 second bar, the next tick would already be then in the next bar.

We've ran a script cumulating the ticks and their volume here on a custom session to test the break situation you've reported issue with in, then output can be seen below - the tick vs second series values are all matching up as expected on our end.

*** SWITCHED TO LIVE ****
Tick: 5/12/2011 12:05:01 PM Volume: 1 Counter: 1
Tick: 5/12/2011 12:05:01 PM Volume: 1 Counter: 2
Tick: 5/12/2011 12:05:01 PM Volume: 1 Counter: 3
Tick: 5/12/2011 12:05:01 PM Volume: 1 Counter: 4
Tick: 5/12/2011 12:05:01 PM Volume: 1 [B]Counter: 5[/B]
Second: 5/12/2011 12:05:01 PM [B]Volume: 5[/B]
Tick: 5/12/2011 12:05:02 PM Volume: 1 Counter: 1
Tick: 5/12/2011 12:05:02 PM Volume: 1 Counter: 2
Tick: 5/12/2011 12:05:02 PM Volume: 1 [B]Counter: 3[/B]
Second: 5/12/2011 12:05:02 PM [B]Volume: 3[/B]
Tick: 5/12/2011 12:05:03 PM Volume: 2 Counter: 2
Tick: 5/12/2011 12:05:03 PM Volume: 1 Counter: 3
Tick: 5/12/2011 12:05:03 PM Volume: 3 [B]Counter: 6[/B]
Second: 5/12/2011 12:05:03 PM [B]Volume: 6[/B]
Tick: 5/12/2011 12:05:04 PM Volume: 1 Counter: 1
Tick: 5/12/2011 12:05:04 PM Volume: 2 Counter: 3
Tick: 5/12/2011 12:05:04 PM Volume: 1 [B]Counter: 4[/B]
Second: 5/12/2011 12:05:04 PM [B]Volume: 4[/B]
Tick: 5/12/2011 12:05:06 PM Volume: 3 [B]Counter: 3[/B]
Second: 5/12/2011 12:05:06 PM [B]Volume: 3[/B]
Tick: 5/12/2011 12:05:07 PM Volume: 1 [B]Counter: 1[/B]
Second: 5/12/2011 12:05:07 PM [B]Volume: 1[/B]
Tick: 5/12/2011 12:05:08 PM Volume: 3 [B]Counter: 3[/B]
Second: 5/12/2011 12:05:08 PM [B]Volume: 3[/B]
Tick: 5/12/2011 12:05:10 PM Volume: 1 Counter: 1
Tick: 5/12/2011 12:05:10 PM Volume: 1 Counter: 2
Tick: 5/12/2011 12:05:10 PM Volume: 1 [B]Counter: 3[/B]
Second: 5/12/2011 12:05:10 PM [B]Volume: 3[/B]

[gomifromparis]

Thanks for testing, but why always complicate things ? I gave an example with the ES on the 9th of May, with default session setting ? Why don't you test that ?

I also pointed that 60 sec charts and 1 min charts don't have same first bar volume. The 60 sec chart lacks first second volume. Why don't you test that as well ?

[NinjaTrader_Austin]

Gomi, I will have Bertrand get back to you tomorrow.

[NinjaTrader_Bertrand]

Gomi, have you tried the same testing after a reload of historical data? The second charts are build from tick data and your minute charts from minute source data. If I compare the first bar of session here on Kinetick or ZenFire I get exact matching volume for it reported.

[gomifromparis]

Gomi, have you tried the same testing after a reload of historical data? The second charts are build from tick data and your minute charts from minute source data. If I compare the first bar of session here on Kinetick or ZenFire I get exact matching volume for it reported.

That's really strange. To be sure, before I did my tests, I exported tick data, deleted minute data, reloaded tick data and checked "recreate minute data from tick data".
Strangely the 60 sec data volume is really missing the opening 00 second data to the tick, whereas it is included in the first minute bar.

[NinjaTrader_Bertrand]

Thanks for clarifying gomi, so you would only see this with imported tick data generating the rest of the barstypes and not when natively using the data from your data provider?

Would you mind sending me the tick data used via Help > Mail to Support as attachment or using a service like www.filexpressit.com?

Thanks in advance,

[gomifromparis]

here it is

https://rapidshare.com/files/3755456804/ES_06-11.Last.txt.zip

time zone is Paris, France

As for the provider issue,, I guess it's not the issue

If I look at yesterday's first bars, after "reload history data" to be sure I get clean data from the history servers..
1 Min is 15019 21151 5740
60 sec is 11845 21151 5740

[NinjaTrader_Bertrand]

Thanks gomi, will look into reproducing on our end - what provider is this data from? eSignal?

[gomifromparis]

yes, eSignal

[NinjaTrader_Bertrand]

Thanks for reporting this in gomi, we could reproduce and confirm this as an issue - development is looking into currently.

[gomifromparis]

Thanks .

[NinjaTrader_Bertrand]

Thanks again for the data and scenario gomi, this issue will be fixed with our NT7 R6.