DailyVolume jumps?

[pacoferre]

I'm using this code:

protected override void OnMarketData(MarketDataEventArgs e)
{

ZPFRegister.SaveEvent(e.Time, "Raw", Instrument.FullName, "L1 " + e.Time.ToString("yyyy-MM-dd HH:mm:ss")
+ " " + e.MarketDataType.ToString()
+ " " + e.Price.ToString()
+ " " + e.Volume.ToString());

}

And all is ok till a find this:

...
L1 2010-04-20 13:05:00 Last 2906 8
L1 2010-04-20 13:05:00 DailyVolume 0 564639
L1 2010-04-20 13:05:00 DailyVolume 0 565269
L1 2010-04-20 13:05:00 Last 2906 22
...

Where are 565269 - 564639 = 630 contracts?

Best regards,

Paco Ferre

[NinjaTrader_Tim]

Hi pacoferre,

It's possible that these are a result of OTC orders, however, so that I can test this on my end, what provider and instrument is this run on?

[pacoferre]

Hi, thanks for your attention.

The instrument is FESX with ZenFire.

[NinjaTrader_Tim]

Hi pacoferre,

I have tested this on my end since your last reply, and have not found an occurrence where a increase in volume is not explained by an equal volume of type last.

Have you seen this to happen again?

[pacoferre]

Hi, many times that day:

Time into [] is the PC time, and also i'm using Ninja Trader 7 beta 14.


[2010-04-20 13:09:28:994] L1 2010-04-20 13:09:41 Last 2904 5
[2010-04-20 13:09:28:994] L1 2010-04-20 13:09:41 DailyVolume 0 576278
[2010-04-20 13:09:29:044] L1 2010-04-20 13:09:41 Ask 2904 606
[2010-04-20 13:09:29:044] L1 2010-04-20 13:09:41 Bid 2903 430
[2010-04-20 13:09:29:625] L1 2010-04-20 13:09:42 Bid 2903 445

[2010-04-20 13:09:29:625] L1 2010-04-20 13:09:42 DailyVolume 0 576528
[2010-04-20 13:09:29:625] L1 2010-04-20 13:09:42 DailyVolume 0 576778
[2010-04-20 13:09:29:625] L1 2010-04-20 13:09:42 DailyVolume 0 577028
[2010-04-20 13:09:29:635] L1 2010-04-20 13:09:42 DailyVolume 0 577278

[2010-04-20 13:09:29:735] L1 2010-04-20 13:09:43 Bid 2903 448
[2010-04-20 13:09:29:795] L1 2010-04-20 13:09:43 Ask 2904 600
[2010-04-20 13:09:29:855] L1 2010-04-20 13:09:43 Bid 2903 450
[2010-04-20 13:09:29:885] L1 2010-04-20 13:09:44 Last 2904 1
[2010-04-20 13:09:29:885] L1 2010-04-20 13:09:44 DailyVolume 0 577279


[2010-04-20 13:09:32:669] L1 2010-04-20 13:09:50 Last 2903 1
[2010-04-20 13:09:32:669] L1 2010-04-20 13:09:50 DailyVolume 0 577296
[2010-04-20 13:09:32:679] L1 2010-04-20 13:09:50 Ask 2904 592
[2010-04-20 13:09:32:679] L1 2010-04-20 13:09:50 Bid 2903 472

[2010-04-20 13:09:33:420] L1 2010-04-20 13:09:51 DailyVolume 0 578171

[2010-04-20 13:09:33:440] L1 2010-04-20 13:09:51 Last 2904 1
[2010-04-20 13:09:33:440] L1 2010-04-20 13:09:51 DailyVolume 0 578172


[2010-04-20 13:09:32:669] L1 2010-04-20 13:09:50 Last 2903 1
[2010-04-20 13:09:32:669] L1 2010-04-20 13:09:50 DailyVolume 0 577296
[2010-04-20 13:09:32:679] L1 2010-04-20 13:09:50 Ask 2904 592
[2010-04-20 13:09:32:679] L1 2010-04-20 13:09:50 Bid 2903 472

[2010-04-20 13:09:33:420] L1 2010-04-20 13:09:51 DailyVolume 0 578171

[2010-04-20 13:09:33:440] L1 2010-04-20 13:09:51 Last 2904 1
[2010-04-20 13:09:33:440] L1 2010-04-20 13:09:51 DailyVolume 0 578172
[2010-04-20 13:09:33:440] L1 2010-04-20 13:09:51 Ask 2904 591
[2010-04-20 13:09:33:500] L1 2010-04-20 13:09:51 Last 2904 1
[2010-04-20 13:09:33:500] L1 2010-04-20 13:09:51 DailyVolume 0 578173


[2010-04-20 13:10:26:757] L1 2010-04-20 13:10:32 Last 2904 1
[2010-04-20 13:10:26:757] L1 2010-04-20 13:10:32 DailyVolume 0 579069

[2010-04-20 13:10:26:757] L1 2010-04-20 13:10:32 DailyVolume 0 579169

[2010-04-20 13:10:26:757] L1 2010-04-20 13:10:32 Bid 2904 262
[2010-04-20 13:10:27:017] L1 2010-04-20 13:10:33 Bid 2904 261
[2010-04-20 13:10:27:678] L1 2010-04-20 13:10:34 Last 2905 3
[2010-04-20 13:10:27:688] L1 2010-04-20 13:10:34 DailyVolume 0 579172
[2010-04-20 13:10:27:688] L1 2010-04-20 13:10:34 Ask 2905 634



[2010-04-20 13:10:51:600] L1 2010-04-20 13:10:51 Last 2904 1
[2010-04-20 13:10:51:600] L1 2010-04-20 13:10:51 DailyVolume 0 579451
[2010-04-20 13:10:51:630] L1 2010-04-20 13:10:51 Ask 2904 8
[2010-04-20 13:10:51:630] L1 2010-04-20 13:10:51 Bid 2903 628

[2010-04-20 13:10:51:721] L1 2010-04-20 13:10:52 DailyVolume 0 580091

[2010-04-20 13:10:51:751] L1 2010-04-20 13:10:52 Last 2904 1
[2010-04-20 13:10:51:751] L1 2010-04-20 13:10:52 DailyVolume 0 580092
[2010-04-20 13:10:51:811] L1 2010-04-20 13:10:52 Ask 2904 10
[2010-04-20 13:10:51:891] L1 2010-04-20 13:10:52 Last 2904 10
[2010-04-20 13:10:51:901] L1 2010-04-20 13:10:52 DailyVolume 0 580102
[2010-04-20 13:10:51:931] L1 2010-04-20 13:10:52 Last 2904 1
[2010-04-20 13:10:51:931] L1 2010-04-20 13:10:52 DailyVolume 0 580103

...



Next day the Last event come after de DailyVolume event, all ok.
[2010-04-21 14:00:07:014] L1 2010-04-21 14:00:21 DailyVolume 0 420577
[2010-04-21 14:00:07:024] L1 2010-04-21 14:00:21 Last 2893 17
[2010-04-21 14:00:07:024] L1 2010-04-21 14:00:21 Bid 2893 225
[2010-04-21 14:00:07:024] L1 2010-04-21 14:00:21 Ask 2894 578
[2010-04-21 14:00:07:104] L1 2010-04-21 14:00:22 Ask 2894 589
[2010-04-21 14:00:07:194] L1 2010-04-21 14:00:22 Ask 2894 593
[2010-04-21 14:00:07:254] L1 2010-04-21 14:00:22 Bid 2893 230
[2010-04-21 14:00:07:284] L1 2010-04-21 14:00:22 Ask 2894 592
[2010-04-21 14:00:07:375] L1 2010-04-21 14:00:23 Bid 2893 225
[2010-04-21 14:00:07:375] L1 2010-04-21 14:00:23 Ask 2894 599
[2010-04-21 14:00:07:385] L1 2010-04-21 14:00:23 Ask 2894 606
[2010-04-21 14:00:07:415] L1 2010-04-21 14:00:23 DailyVolume 0 421183
[2010-04-21 14:00:07:415] L1 2010-04-21 14:00:23 Last 2894 606
[2010-04-21 14:00:07:435] L1 2010-04-21 14:00:23 DailyVolume 0 421199
[2010-04-21 14:00:07:435] L1 2010-04-21 14:00:23 Last 2894 16
[2010-04-21 14:00:07:435] L1 2010-04-21 14:00:23 DailyVolume 0 421200
[2010-04-21 14:00:07:435] L1 2010-04-21 14:00:23 Last 2894 1
[2010-04-21 14:00:07:435] L1 2010-04-21 14:00:23 DailyVolume 0 421226
[2010-04-21 14:00:07:435] L1 2010-04-21 14:00:23 Last 2894 26
[2010-04-21 14:00:07:435] L1 2010-04-21 14:00:23 DailyVolume 0 421238
[2010-04-21 14:00:07:445] L1 2010-04-21 14:00:23 Last 2894 12
[2010-04-21 14:00:07:445] L1 2010-04-21 14:00:23 DailyVolume 0 421247
[2010-04-21 14:00:07:445] L1 2010-04-21 14:00:23 Last 2894 9
[2010-04-21 14:00:07:445] L1 2010-04-21 14:00:23 DailyVolume 0 421267
[2010-04-21 14:00:07:445] L1 2010-04-21 14:00:23 Last 2894 20


But there is an error again:
[2010-04-21 14:00:16:007] L1 2010-04-21 14:01:02 DailyVolume 0 423543
[2010-04-21 14:00:16:007] L1 2010-04-21 14:01:02 Last 2894 57
[2010-04-21 14:00:16:007] L1 2010-04-21 14:01:02 DailyVolume 0 423550
[2010-04-21 14:00:16:007] L1 2010-04-21 14:01:02 Last 2894 7
[2010-04-21 14:00:16:017] L1 2010-04-21 14:01:02 DailyVolume 0 423576
[2010-04-21 14:00:16:017] L1 2010-04-21 14:01:02 Last 2894 26
[2010-04-21 14:00:16:017] L1 2010-04-21 14:01:02 DailyVolume 0 423579
[2010-04-21 14:00:16:017] L1 2010-04-21 14:01:02 Last 2894 3
[2010-04-21 14:00:16:037] L1 2010-04-21 14:01:02 DailyVolume 0 423582
[2010-04-21 14:00:16:037] L1 2010-04-21 14:01:02 Last 2894 3
[2010-04-21 14:00:16:037] L1 2010-04-21 14:01:02 Bid 2893 506
[2010-04-21 14:00:16:037] L1 2010-04-21 14:01:02 Ask 2894 462
[2010-04-21 14:01:04:804] L1 2010-04-21 14:01:04 Bid 2895 279
[2010-04-21 14:01:04:814] L1 2010-04-21 14:01:04 Ask 2896 288
[2010-04-21 14:01:04:944] L1 2010-04-21 14:01:04 DailyVolume 0 424586

[2010-04-21 14:01:04:944] L1 2010-04-21 14:01:04 Last 2896 40
ˇˇˇ Should be 4 !!!

[2010-04-21 14:01:04:944] L1 2010-04-21 14:01:04 DailyVolume 0 424606
[2010-04-21 14:01:04:944] L1 2010-04-21 14:01:04 Last 2896 20
[2010-04-21 14:01:04:954] L1 2010-04-21 14:01:05 Bid 2895 295
[2010-04-21 14:01:04:954] L1 2010-04-21 14:01:05 Ask 2896 219
[2010-04-21 14:01:05:004] L1 2010-04-21 14:01:05 DailyVolume 0 424646
[2010-04-21 14:01:05:004] L1 2010-04-21 14:01:05 Last 2896 40
[2010-04-21 14:01:05:004] L1 2010-04-21 14:01:05 DailyVolume 0 424650
[2010-04-21 14:01:05:004] L1 2010-04-21 14:01:05 Last 2896 4


Best regards,

Paco Ferre

[pacoferre]

Also, may be a connection error in the last error.

[2010-04-21 14:00:16:037] L1 2010-04-21 14:01:02 Ask 2894 462
[2010-04-21 14:01:04:804] L1 2010-04-21 14:01:04 Bid 2895 279

But it doesn't explain why Last 40 is impossible in that situation.



Also i'm going to store OnConnectionStatus events to see if there is a reason by this way.


Thanks a lot for your support.


Best regards,

Paco Ferre

[pacoferre]

OK, error again.


There are 1004 contract from 423582 to 424606, not only 40


[2010-04-21 14:00:16:007] L1 2010-04-21 14:01:02 DailyVolume 0 423543
[2010-04-21 14:00:16:007] L1 2010-04-21 14:01:02 Last 2894 57
[2010-04-21 14:00:16:007] L1 2010-04-21 14:01:02 DailyVolume 0 423550
[2010-04-21 14:00:16:007] L1 2010-04-21 14:01:02 Last 2894 7
[2010-04-21 14:00:16:017] L1 2010-04-21 14:01:02 DailyVolume 0 423576
[2010-04-21 14:00:16:017] L1 2010-04-21 14:01:02 Last 2894 26
[2010-04-21 14:00:16:017] L1 2010-04-21 14:01:02 DailyVolume 0 423579
[2010-04-21 14:00:16:017] L1 2010-04-21 14:01:02 Last 2894 3
[2010-04-21 14:00:16:037] L1 2010-04-21 14:01:02 DailyVolume 0 423582
[2010-04-21 14:00:16:037] L1 2010-04-21 14:01:02 Last 2894 3
[2010-04-21 14:00:16:037] L1 2010-04-21 14:01:02 Bid 2893 506
[2010-04-21 14:00:16:037] L1 2010-04-21 14:01:02 Ask 2894 462
[2010-04-21 14:01:04:804] L1 2010-04-21 14:01:04 Bid 2895 279
[2010-04-21 14:01:04:814] L1 2010-04-21 14:01:04 Ask 2896 288
[2010-04-21 14:01:04:944] L1 2010-04-21 14:01:04 DailyVolume 0 424586
[2010-04-21 14:01:04:944] L1 2010-04-21 14:01:04 Last 2896 40
[2010-04-21 14:01:04:944] L1 2010-04-21 14:01:04 DailyVolume 0 424606
[2010-04-21 14:01:04:944] L1 2010-04-21 14:01:04 Last 2896 20
[2010-04-21 14:01:04:954] L1 2010-04-21 14:01:05 Bid 2895 295
[2010-04-21 14:01:04:954] L1 2010-04-21 14:01:05 Ask 2896 219
[2010-04-21 14:01:05:004] L1 2010-04-21 14:01:05 DailyVolume 0 424646
[2010-04-21 14:01:05:004] L1 2010-04-21 14:01:05 Last 2896 40
[2010-04-21 14:01:05:004] L1 2010-04-21 14:01:05 DailyVolume 0 424650
[2010-04-21 14:01:05:004] L1 2010-04-21 14:01:05 Last 2896 4


Now it is running also with this code, will see.

protected override void OnConnectionStatus(ConnectionStatus orderStatus, ConnectionStatus priceStatus)
{
ZPFRegister.SaveEvent(DateTime.Now, "Raw", Instrument.FullName, "CS " + orderStatus.ToString()
+ " " + priceStatus.ToString());
}


Best regards,

Paco Ferre

Hi, many times that day:


But there is an error again:
[2010-04-21 14:00:16:007] L1 2010-04-21 14:01:02 DailyVolume 0 423543
[2010-04-21 14:00:16:007] L1 2010-04-21 14:01:02 Last 2894 57
[2010-04-21 14:00:16:007] L1 2010-04-21 14:01:02 DailyVolume 0 423550
[2010-04-21 14:00:16:007] L1 2010-04-21 14:01:02 Last 2894 7
[2010-04-21 14:00:16:017] L1 2010-04-21 14:01:02 DailyVolume 0 423576
[2010-04-21 14:00:16:017] L1 2010-04-21 14:01:02 Last 2894 26
[2010-04-21 14:00:16:017] L1 2010-04-21 14:01:02 DailyVolume 0 423579
[2010-04-21 14:00:16:017] L1 2010-04-21 14:01:02 Last 2894 3
[2010-04-21 14:00:16:037] L1 2010-04-21 14:01:02 DailyVolume 0 423582
[2010-04-21 14:00:16:037] L1 2010-04-21 14:01:02 Last 2894 3
[2010-04-21 14:00:16:037] L1 2010-04-21 14:01:02 Bid 2893 506
[2010-04-21 14:00:16:037] L1 2010-04-21 14:01:02 Ask 2894 462
[2010-04-21 14:01:04:804] L1 2010-04-21 14:01:04 Bid 2895 279
[2010-04-21 14:01:04:814] L1 2010-04-21 14:01:04 Ask 2896 288
[2010-04-21 14:01:04:944] L1 2010-04-21 14:01:04 DailyVolume 0 424586

[2010-04-21 14:01:04:944] L1 2010-04-21 14:01:04 Last 2896 40
ˇˇˇ Should be 4 !!!

[2010-04-21 14:01:04:944] L1 2010-04-21 14:01:04 DailyVolume 0 424606
[2010-04-21 14:01:04:944] L1 2010-04-21 14:01:04 Last 2896 20
[2010-04-21 14:01:04:954] L1 2010-04-21 14:01:05 Bid 2895 295
[2010-04-21 14:01:04:954] L1 2010-04-21 14:01:05 Ask 2896 219
[2010-04-21 14:01:05:004] L1 2010-04-21 14:01:05 DailyVolume 0 424646
[2010-04-21 14:01:05:004] L1 2010-04-21 14:01:05 Last 2896 40
[2010-04-21 14:01:05:004] L1 2010-04-21 14:01:05 DailyVolume 0 424650
[2010-04-21 14:01:05:004] L1 2010-04-21 14:01:05 Last 2896 4


Best regards,

Paco Ferre

[NinjaTrader_Tim]

Hi pacoferre,

I will continue to try to reproduce this.

[pacoferre]

Hi again,

There is a Windows error saving the file: "The process cannot access the file because it is being used by another process".

I've added a lock block to avoid concurrent calls to the function, a maximun of 10 tries to save the data, with 5 milleseconds of Thread.Sleep between retries. Hope this work.

privatestaticobject objLockSave = newobject();

publicvoid SaveEvent(DateTime time, string name, string instrument, string data)
{
int tries = 0;
bool saved = false;
while (!saved && tries < 10)
{
try
{
lock (objLockSave)
{
StreamWriter sw = System.IO.File.AppendText("c:\\Datos\\"
+ time.ToString("yyyyMMddHH")
+ " " + name
+ " " + instrument + ".txt");
sw.WriteLine("[" + DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss:fff")
+ "] " + data);
sw.Close();
sw = null;
saved = true;
}
}
catch
{
tries++;
System.Threading.Thread.Sleep(5);
}
}
if (!saved)
{
thrownewException("Error saving Data: " + data);
}
}


Best regards,

Paco Ferre

[pacoferre]

OK, once again , two hours ago.

No windows file sharing errors, no connection lost, no antivirus...

[2010-04-23 12:06:42:933] L1 2010-04-23 12:06:42 DailyVolume 0 539762
[2010-04-23 12:06:42:933] L1 2010-04-23 12:06:42 Last 2838 100
[2010-04-23 12:06:42:933] L1 2010-04-23 12:06:42 DailyVolume 0 539773
[2010-04-23 12:06:42:933] L1 2010-04-23 12:06:42 Last 2838 11
[2010-04-23 12:06:42:963] L1 2010-04-23 12:06:42 DailyVolume 0 539798
[2010-04-23 12:06:42:963] L1 2010-04-23 12:06:42 Last 2838 25
[2010-04-23 12:06:42:963] L1 2010-04-23 12:06:42 DailyVolume 0 539812
[2010-04-23 12:06:42:963] L1 2010-04-23 12:06:42 Last 2838 14
[2010-04-23 12:06:42:963] L1 2010-04-23 12:06:42 DailyVolume 0 539818
[2010-04-23 12:06:42:963] L1 2010-04-23 12:06:42 Last 2838 6
[2010-04-23 12:06:42:963] L1 2010-04-23 12:06:42 Bid 2838 2
[2010-04-23 12:06:42:963] L1 2010-04-23 12:06:42 Ask 2839 723
[2010-04-23 12:06:43:233] L1 2010-04-23 12:06:43 Bid 2838 17
[2010-04-23 12:06:43:233] L1 2010-04-23 12:06:43 Ask 2839 652

[2010-04-23 12:06:43:293] L1 2010-04-23 12:06:43 DailyVolume 0 539928
(No Last **** 110 signal)

[2010-04-23 12:06:43:473] L1 2010-04-23 12:06:43 Bid 2838 47
[2010-04-23 12:06:43:473] L1 2010-04-23 12:06:43 Ask 2839 648
[2010-04-23 12:06:43:483] L1 2010-04-23 12:06:43 DailyVolume 0 539975
[2010-04-23 12:06:43:483] L1 2010-04-23 12:06:43 Last 2838 47
[2010-04-23 12:06:43:503] L1 2010-04-23 12:06:43 DailyVolume 0 539976
[2010-04-23 12:06:43:503] L1 2010-04-23 12:06:43 Last 2838 1
[2010-04-23 12:06:43:533] L1 2010-04-23 12:06:43 DailyVolume 0 539978
[2010-04-23 12:06:43:533] L1 2010-04-23 12:06:43 Last 2838 2
[2010-04-23 12:06:43:533] L1 2010-04-23 12:06:43 DailyVolume 0 539983
[2010-04-23 12:06:43:533] L1 2010-04-23 12:06:43 Last 2838 5
[2010-04-23 12:06:43:563] L1 2010-04-23 12:06:43 DailyVolume 0 539984
[2010-04-23 12:06:43:563] L1 2010-04-23 12:06:43 Last 2838 1


Well, not a big problem, but i'd like to know what's happening
żAnyone is storing data of instrument FESX from ZenFire?

Best regards,

Paco Ferre

[NinjaTrader_Tim]

Hi pacoferre,

I've completed my tests on this and have concluded that these appear to be explained by OTC orders.

The "missing trades" are....
1. Generally large numbers of shares
2. Generally "round" numbers
3. Do not appear with extreme frequency or concurrently with large CPU usage.

[pacoferre]

OK, perfect explanation.

Thanks a lot for your time

Best regards,

Paco Ferre

[balltrader]

I know this is an old thread.
Please explain or point to a url where I can understand what OTC orders are, and how it would affect this datastream?
Most searches reveal OTC = Over The Counter

Does that mean an order is placed outside the electronic system, without that order previously appearing in the DOM / queue, and the order processing "jumps" up the traded volume by the "round number" amount?

[NinjaTrader_PatrickH]

Hello balltrader,

Thank you for your post.

You can find an article going into detail on OTC at the publicly available link that follows: https://www.investopedia.com/terms/o/otc.asp

These orders are placed outside the exchange in OTC networks. Please refer to the link above for more details and information.

Please let me know if I may be of further assistance.