no correlation between backtest and real-time

[bugo11]

hello,
i am trading multiple instrument with a strategy which performs really good on backtest
but on real time strange things happening
after a long research i found out that the cause to this problem is due to bars with 0 volume
whenever there is a bar with 0 volume - ninjatrader will not update on real time, but when loading historical data there is a 0 volume bar

it is extremely important for my strategy to work as all my calculation are based on few indicators that need to be updated correctly

is there any way to fix this issue?
thanks!

[NinjaTrader_Jesse]

Hello bugo11,

Thank you for the post.

When you say a 0 volume bar it does not update, do you mean that you are not seeing OnBarUpdate called during that time? Can you provide more details on the specific situation you have explained?

Are you seeing 0 volume bars in historical but not in realtime, or are you seeing this in both but it is performing differently in those uses?

I look forward to being of further assistance.

[bugo11]

0 volume bars are only visible on historical data
in real time bars get updated only when there is volume
for exemple
Im using 1 minute bars:
in 15:00 a new bar created
after this there are 2 minutes with no volume,
next bar will be in 15:03 (only in real time)
on historical data there will be bars on 15:00, 15:01, 15:02, 15:03
(where 15:01, 15:02 are 0 volume bars)

[NinjaTrader_Jesse]

Hello bugo11,

Thank you for the additional details.

Can you tell me what instrument you are using and the specific time range where you see this? I want to explore the historical data and see if anything can be done in that situation but I would need the specifics of what you are testing here. What instrument is being used, what data provider and what time range shows the bars in question? Also are you specifically using the minute BarsType with a value of 1 or is this a different type with a base period of 1 minute? .


I look forward to being of further assistance.

[bugo11]

hey again Jesse
i'm using it for cryptocurrency,
it's very common to have a bar with no volume on some of the pairs
i'm using specifically 1 minute BarType.
i'm attaching 2 screenshots
1 is from live feed, the other is after reloading all historical data
as you can see, the first one have bars only when there is a trade,
the other will have bars for every minute even if there is no volume

thanks in advance for your help!

[koganam]

hey again Jesse
i'm using it for cryptocurrency,
it's very common to have a bar with no volume on some of the pairs
i'm using specifically 1 minute BarType.
i'm attaching 2 screenshots
1 is from live feed, the other is after reloading all historical data
as you can see, the first one have bars only when there is a trade,
the other will have bars for every minute even if there is no volume

thanks in advance for your help!

Refactor your code to ignore bars with zero volume, and also whatever the default volume NT assigns when the volume is zero.

[bugo11]

hey again
a few days have passed with no answer,
there is the way koganam suggested which i also thought but i think it's a bit complicated cause i need to adjust many things, and it might even hurt the current strategy, which as i said works really good under backtests, so i prefer an approach which works with what i currently have instead of changing everything and even might break it.

the way i see it, the other way is if i could create a bar based on a condition like - 'if there is no bar update in this minute', create a bar

so my question is, is there a way to draw custom bars?

[NinjaTrader_Jesse]

Hello bugo11,

It looks like I still need to know what data feed you are using and what instrument this is for so I can test this further. Can you provide those details for me to review?

I look forward to being of further assistance.

[bugo11]

my datafeed ia binance and the instrument is BTCTUSD

[NinjaTrader_Jesse]

Hello bugo11,

Thank you for confirming that.

[I]whenever there is a bar with 0 volume - ninjatrader will not update on real time, but when loading historical data there is a 0 volume bar[/I]

Going back to your original post, what you are seeing in realtime is correct we should not see bars generated if there was no data which also means OnBarUpdate will not be called during that time.

I would like to ask a few more questions about what you are seeing for the historical data. The general expectation for bars being built would be that some data for that time is required for a bar to be built. For most connections we would see missing time between two bars where there was no data. If the data feed is supplying historical data points for these times even though there is no data that may be the reason for that difference.

One item I would like to understand is how the data looks in the historical data manager for the points you had shown. Could you view the time you had shown in the historical data tool and see what values are reported for the bars with no data? From the control center -> tools -> historical data window and then use the left side to navigate to the instrument/date. For the bars displayed, what is the volume for those bars? Is it completely 0 or is there a small amount of volume that traded at that price? If you can take an image that would help.

I would also like to understand how the script sees this situation in both realtime and historical. Do you have a Print to display the difference from any tests you had done? If not, could you make a print like the following to print during both realtime and historical?

Print(State + " " + Time[0] + " " + Volume[0]);

If you run this in realtime during a period where you see this happen, and then can re rerun the script over that data again as historical that would help me to better understand what may need to happen for a solution.

I look forward to being of further assistance.

[bugo11]

Hello again Jesse,
thank you for taking your time to understand my problem and hopefully we'll find a fix for it.
looking at historical data manager, the report will have lines with 0 volume for every bar with 0 volume, i'm attaching a pic for you

for your second question - i chose an instrument where it's more common so i could supply you the details you need asap
( i'm posting only a few lines from each - if you need more information i will supply.
this is for historical data

Historical 9/10/19 10:20:00 PM 0

Historical 9/10/19 10:21:00 PM 0

Historical 9/10/19 10:22:00 PM 0

Historical 9/10/19 10:23:00 PM 0

Historical 9/10/19 10:24:00 PM 0

Historical 9/10/19 10:25:00 PM 0

Historical 9/10/19 10:26:00 PM 0

Historical 9/10/19 10:27:00 PM 0

Historical 9/10/19 10:28:00 PM 1138435000

Historical 9/10/19 10:29:00 PM 0

Historical 9/10/19 10:30:00 PM 0

Historical 9/10/19 10:31:00 PM 0

Historical 9/10/19 10:32:00 PM 0

Historical 9/10/19 10:33:00 PM 0

Historical 9/10/19 10:34:00 PM 0

Historical 9/10/19 10:35:00 PM 0

Historical 9/10/19 10:36:00 PM 0

Historical 9/10/19 10:37:00 PM 607477000

Historical 9/10/19 10:38:00 PM 15610893000

Historical 9/10/19 10:39:00 PM 7892201000

Historical 9/10/19 10:40:00 PM 545332000

Historical 9/10/19 10:41:00 PM 339217000

Historical 9/10/19 10:42:00 PM 476922000

Historical 9/10/19 10:43:00 PM 0

Historical 9/10/19 10:44:00 PM 9255000

Historical 9/10/19 10:45:00 PM 179552000

Historical 9/10/19 10:46:00 PM 20000000

Historical 9/10/19 10:47:00 PM 0

Historical 9/10/19 10:48:00 PM 0

Historical 9/10/19 10:49:00 PM 0

Historical 9/10/19 10:50:00 PM 0

Historical 9/10/19 10:51:00 PM 98495000

Historical 9/10/19 10:52:00 PM 0

this is for real-time

Realtime 9/10/19 10:54:00 PM 173660000

Realtime 9/10/19 10:55:00 PM 96579000

Realtime 9/10/19 10:57:00 PM 52000000

Realtime 9/10/19 10:59:00 PM 102153000

thanks again for taking your time to help!

[NinjaTrader_Jesse]

Hello bugo11,

Thank you for providing these details. It looks like this explains what is happening better.

I can see from the image that the volumes being reported in historical are integers so what you see is expected in this case. The 0 volume is just part of the data which was received, because there is data here the bar is going to be created. Generally data does not contain empty data points if there was no volume for a period of time because there was nothing happening. What you are showing seems to include extra datapoints either to fill in gaps or the volume is being reported at not a granular enough level for example a volume of 0.5 is just 0.

For this question you will need to contact the data provider to verify if this is how the data should normally be supplied through this connection. Binance is not a supported connection adapter for NinjaTrader so this is not something which our support can dive into further. You may be able to modify your script to account for this, however this would not be something the platform can change or would have a setting for specifically.



I look forward to being of further assistance.