OnAfterBarUpdate

[BarzTrading]

I want to run code once per bar, but only after all the data series have completed their individual OnBarUpdate calls.

Is there a good way to do this? Is there an event one can hook into or a technique to ensure all OnBarUpdate calls for that bar (DateTime) have completed?

Is the OnBarUpdate for the primary series (BarsInProgress=0) guaranteed to run first for each bar, assuming it has data for that bar?

If it simplifies things, we can assume all series have the same BarsPeriod.

Thanks.

[NinjaTrader_Jesse]

Hello BarzTrading,

This would be difficult to do considering there is no expectation that two instruments have identical data and close at the exact same time for the order to matter like this. You will very likely see the primary first but there is no guarantee of this in real-time.

There is an order to bar processing historical data but in realtime there is not really a guarantee that one series will have trades before another series to cause the bars to close in a certain way. It is likely for high volume instruments that you will have identical closes but there is nothing saying the primary has to close first here, the market is going to make a difference on this.

If you want to make sure that each series has had a close before completing this logic, you would very likely need to make logic to do specifically that. You could choose a point to reset your logic, such as the primary series and then have each second series set a boolean variable to denote it closed a bar since that time. Once all series have closed, you could do your other logic and repeat the reset process. You would likely need to use Prints here to identify if all of the instruments being used are closing bars at regular intervals to complete logic like this. If one of the series has low volume and doesn't update with the other series, you may miss events due to waiting for a bar to close on one instrument.

Here is an example of what I am talking about, this is a comparison between the ES and AAPL from this morning using the default 24/5 session template. This allows for some higher/lower volume bars:

Print(BarsInProgress + " " + CurrentBar);

0 24385
1 7471
0 24386
1 7472
0 24387
1 7473
0 24388
0 24389
1 7474

BIP 0 was called twice in a row during this time because AAPL didnt have any trades to close a bar during that time.


I look forward to being of further assistance.

[BarzTrading]

Thanks, Jesse, for the detailed reply. You've understood my question perfectly and the complications I ran into trying to do this.

I understand most of your reply. But what do you mean by "make sure each series has had a close?" What would you test or look at to verify that?

[NinjaTrader_Jesse]

Hello BarzTrading,

By "make sure each series has had a close"I simply mean that you make sure each added series has closed its bar during that time or the goal you were asking to achieve.

Let's break down your goal here:

I want to run code once per bar,

but only after all the data series have completed their individual OnBarUpdate calls.

Running code once per bar is easy but the requirement you have makes it difficult. If the requirement was simply to run the code once per bar, you could use IsFirstTickOfBar to simulate after bar close or OnBarOpen logic. However, because there is not a specific order with the live market, waiting for all series to complete an OnBarUpdate call may or may not be accurate. You would likely need to address this part of your requirement specifically. What is the specific need or why do we need to ensure all series have run OnBarUpdate before running this other logic? Also what series is being called after all series have run OnBarUpdate? At this point I would suspect a new bar is in progress of being built, are you trying to do something intrabar between the series you have added?


I look forward to being of further assistance.

[BarzTrading]

Thanks, Jesse. I'm trying to implement a kind of portfolio functionality. I have created code for "phantom" strategies. These strategies don't actually trade live, but they allow me to maintain stats on a set of strategies and instruments, AS IF they were trading live. These get updated with every bar update. But then I want to evaluate all of the signals from all of my phantom strategies and decide which of those signals to take for live trading based on a ranking of them.

I've done most of this work already. The final challenge is the subject of this thread. How can I update all of my phantom strategies on a bar and only then decide which of their signals to take for live trades?

If it makes this any simpler, I'm not interested in tick data. Also, CalculateMode=OnBarClose is just fine. I would probably not use bars smaller than 5 minutes.

My hope was that I could make the exact same trade live that would have been made in my phantom strategy.

My understanding is that when an order is submitted, it is processed at the open of the next bar. So trying to simulate after bar close logic using IsFirstTickOfBar means that an order submitted here would be delayed _another_full bar before processing. That would mean that my live trades would be one bar delayed from my phantom trades.

Maybe there are some subtleties about order submission and execution that would allow an order to be submitted in a simulated "OnBarOpen" event and executed in that same bar? Maybe not, I'm just grasping at straws.

Any ideas?

The only other thing I can think of is to use a smaller bar period and allow the live/portfolio trades to occur one bar delayed from the phantom trades. Honestly, this might work fine even though the phantom and live trades are not perfectly sync'd.

[NinjaTrader_Jesse]

Hello BarzTrading,

Thank you for the reply.

I've done most of this work already. The final challenge is the subject of this thread. How can I update all of my phantom strategies on a bar and only then decide which of their signals to take for live trades,?

Well this would be difficult. At any given time, they may update in a different order so I couldn't really suggest one place for this logic to happen. You would very likely need to have all instruments perform this logic for each bar so the overall script knows when all instruments have processed said bar. That would give the overview you need to pick signals out of the pool of signals generated.

You mentioned the phantom strategies, just so we are on the same page are all of these separate scripts or is this all within one strategy? That will also make a large difference on when everything is processing. If these are separate strategies, that will add another layer of separation which I would not really suggest. You very likely would need to use one script that adds multiple secondary series to achieve any kind of sync between the various instruments and processing, this will be important if you need information for more than one instrument to generate your signal.

My understanding is that when an order is submitted, it is processed at the open of the next bar. So trying to simulate after bar close logic using IsFirstTickOfBar means that an order submitted here would be delayed _another_full bar before processing. That would mean that my live trades would be one bar delayed from my phantom trades.

In this situation, to use IsFirstTIckOfBar you would also need to use OnEachTick so that would eliminate some of the waiting as you are not waiting for a bar to close here.

My hope was that I could make the exact same trade live that would have been made in my phantom strategy.

It is very likely that you could but you would need to identify how you want the overall logic to work in regard to signal generation. Being that the order of processing may change, you may need to make the signal processing logic smart to know when to wait for instruments still processing or when not to wait and use the information it currently has. This is really something that will depend on your goals so I can't really dive too deep into this topic without speculating.

Maybe there are some subtleties about order submission and execution that would allow an order to be submitted in a simulated "OnBarOpen" event and executed in that same bar? Maybe not, I'm just grasping at straws.

You can submit orders intrabar, so using IsFirstTickOfBar may be one way to accomplish a similar goal. I would really suggest making a simple test that includes only 1 or 2 instruments. From this, you can use Prints and also submit orders to more closely understand how the bars are being processed and when orders will fill. By doing this, you may extract other details which better help you form the signal generation logic.

The only other thing I can think of is to use a smaller bar period and allow the live/portfolio trades to occur one bar delayed from the phantom trades. Honestly, this might work fine even though the phantom and live trades are not perfectly sync'd.

This would also be something to test and see what the result is for you. If the result is acceptable, it may be worth your time to go down that road. Really this is going to be a question you will need to explore in NinjaScript to see what works best for your goal in real-time. Being that you want to use multiple instruments in this way, I would likely suggest going straight for the live modes such as playback, simulated data feed or a live data feed for testing.



I look forward to being of further assistance.

[BarzTrading]

Thanks for the detailed response Jesse. You've given me several things to think about and experiment with.

To answer a couple of questions:

It's all one NinjaTrader strategy script, the phantom strategies are my own code and have essentially nothing to do with NT, but they operate pretty much exactly the same way as an NT strategy except they only simulate trades. I update them with data and submit orders during the OnBarUpdate of the main NT strategy. In a basic NT strategy with one data series I can process historical data and submit identical orders to both NT and to a phantom strategy and I'll get exactly the same trades at the same prices, with the same performance stats, etc. That's redundant, of course. This is only really useful when you're dealing with a portfolio of strategies. And the phantom strategies allow it to all be under the umbrella of one NT strategy.

I use the term "phantom" strategy because I've heard John Ehlers use it. They're just simulated background strategies that never make real trades.

The biggest challenge, it seems, is what you mentioned in your first reply. Some data series may not have data for every bar. Therefore, we can't know which update is the last one for the bar. If we don't get an update for every data series, then we can't know for certain we've received all updates until the start of the next bar.

I might still be able to work with this, though.

I'll do some more testing and report back here with additional questions or successes.

Thanks,

[BarzTrading]

In the end I decided that I did not need an event to run AFTER all of the OnBarUpdate events for each data series.

I created an event OnBeforeBarUpdates which runs once per bar. We do this by checking the time on the bar, if it has changed then we run the event.

In this event, we can sort the existing phantom strategies based on their activity up to this point.

Then when the individual OnBarUpdate events execute for each series we look to see if it satisfies our criteria.

For example, we'll only trade the highest ranked instrument based on the sorting of the phantom strategies. In that case, as soon as the OnBarUpdate runs for that instrument we'll submit an order.

You could also say that we'll submit an order for any instrument that has a profitable phantom strategy. Then when an OnBarUpdate runs for a profitable instrument we submit the order. Then we set a flag to note that we've already submitted an order this bar so we don't submit another one. The flag gets reset for each bar in OnBeforeBarUpdates.

Of course, another way to do this is to set EntriesPerDirection=1.

NOTE:

There is a complicating issue with this approach. Real time updates (OnBarUpdate) execute in an unpredictable order while historical updates always execute in the same order.

So, the performance between real-time and historical could be vastly different depending on your criteria for whether you take an order or not.

I ended up shuffling the historical updates to more closely approximate the more random (not random, but more so) order of updates in real-time.

With historical updates we can actually run an event after all updates, something like OnAfterAllUpdates, which we can't do in real-time. For historical, on the first OnBarUpdate we can go ahead and check every data series to see if it has a bar at the current DateTime value. Then we know how many OnBarUpdate calls there will be for this bar. We can't do this in real-time because that data is not yet available at the first update of the bar.

So, we can collect information from each update, and then after the last update, shuffle the data and call some processing method in a more randomized order. This approximates the real-time behavior.

Another Approach:

If someone else is trying to implement something similar, I would suggest that you might be better off implementing a system that places trades one bar delayed. That is, do all of your processing in your custom OnBeforeBarUpdates event.

You'll rank your phantom strategies, then you'll look to see which of those generated order signals in the _previous_ bar. You apply some criteria and decide which orders to take, then submit them. These orders will then be executed at the start of the _next_ bar.

There are tradeoffs, obviously. You will not be exactly mirroring orders in the phantom strategies. You'll have to make sure that your strategy still works if you place your order one bar later.

But this approach allows your system to be more consistent between historical and real-time processing. When I have separate code for historical and real-time it makes me nervous.

And the coding is quite a bit simpler.


There is very likely a better approach than these, but there would still be some tradeoffs.

This got me what I needed for now.