How to ensure a time-based operation is performed when OnBarUpdate is not called

[digibob]

Hi.

Using either hard-coded time values or a SessionIterator in attempt to place an order and/or do some operation in code at a certain time, to begin with OnBarUpdate has to be called, which it doesn't when there is no market action during that bar. This is the case in Historical state.

1) Preliminary Q: is the behavior in real time similar to that in historical? If Calculate.OnBarClose, and there is no market action during a certain bar, would OnBarUpdate get called for that bar in real time or not?

I'm referring to a secondary series here, if it makes a difference. Primary series is daily, secondary is either 1-minute or 1-second.

2) Main Q: in both historical and real time states, how to make sure a time-based operation always takes place, given that On BarUpdate is not guaranteed to always be called at that time? Think instrument with less liquidity.

I see an example of using a timer in the BarTimer indicator but to place an order from within a timer callback doesn't strike me as safe or advised, if at all possible.

Thanks. Good day.

[NinjaTrader_ChrisL]

Hi digibob, thanks for your post.

1. OnBarUpdate will not be called when the time is out of session. If you want you can set the session template to Default 24X7 or 24X5 and manage the time aspect of the strategy yourself (not trying to enter the market when out of session)

2. Like I said already you can set a different session template and OnBarUpdate will be called. You can try implementing the timer and submitting an order in the callback, just test it in the Sim101 account of course. Note that your order will probably get rejected by your broker if you submit an order outside of the session.

If I can assist any further please let me know.

[digibob]

Hi Chris. Thanks for the reply. I didn't mean out of session. Both questions are in regard to in-session. Let's say I want to perform an operation in code 5 minutes before market close, but the instrument is not liquid enough, there is no market action at that time (not because out of session, simply because low volume), so no trade/tick data to trigger a call by the architecture to my OnBarUpdate implementation, so the code doesn't get executed.

This is the case in historical state as I can see.

I was wondering (1) if it's the same in real time, and (2) what I can do to ensure the code always runs at that point in time without relying on price change/incoming tick/market action.

Thanks.

[digibob]

By the way the more I think about it a timer would not help. How could I aim for a specific time with a timer? It would depend on the exact moment I click the button to start the strategy, and talking seconds here, it has to be accurate. I need to code to always run at exactly xx:xx:xx no matter what. How do I do that?

[bltdavid]

Huh? A timer sounds like the perfect solution.

Why can't your timer check wall clock time and execute its logic at exactly xx:xx:xx?

Even if you use a 1-second timer, and the timer fires every second, it is easy for your
timer code to skip all processing until it sees the exact time xx:xx:xx you're looking for.

The "exact moment" you start the strategy has nothing to do with it.

[EDIT: I'm referring to one of the many .NET timer classes. I see NT8's BarTimer
uses a DispatcherTimer. The idea is for the .NET timer to call your routine at an
interval you specify, where your code then checks wall clock time and only executes
at the exact time xx:xx:xx you specify.]

[bltdavid]

Good example of using a timer here,

https://ninjatrader.com/support/helpGuides/nt8/?triggercustomevent.htm

[bltdavid]

Good example of using a timer here,
https://ninjatrader.com/support/help...ustomevent.htm

Question for NT support:

The example in the above help guide page has this,

[COLOR=#008000][I]// Declare the WPF dispatcher timer[/I][/COLOR]
[COLOR=#0000ff]private [/COLOR]System.Timers.Timer myTimer;

And I see the NT8 BarTimer indicator actually uses a "DispatcherTimer" object
for it's timer needs -- which seems at odds of sort ---

In view of BarTimer using a DispatcherTimer, the code comment in the
help guide begs the question:

Is the help guide example code showing use of a "System.Timers.Timer" still correct?

I mean, if BarTimer indicator had to be updated to use a "Systems.Windows.Threading.DispatcherTimer",
does this code example need updating as well?

[NinjaTrader_ChrisL]

Hi bltdavid and digibob,

bltdavid I'm not sure about the difference between our help guide example timer and the bar timer's dispatcher timer. I will need to ask my colleagues. The DispatcherTimer runs on the UI thread where System.Timers.Timer is asynchronous.

digibob A timer that is called every X seconds or milliseconds would be a solution. The timer has millisecond resolution so it is very precise. If you set up the timer in one of the startup states then it will start counting from the millisecond "enable>OK" is clicked. The timer can capture any clock moment (hr.mm.ss.ms).

[digibob]

Hi Chris and bltdavid. Thank you both. A 1-second interval should obviously do the trick. I thought along the lines of starting the strategy six hours before the time in question with a six-hour interval timer.. a little thick in the head.. maybe thrown off by the needs and growing size of the strategy as well as my daily schedule and most of all the fact I've not worked with a timer before.

I'm not sure how to pass all the (many!) required variables to the custom handler, or if I need to pass them, or if I need a custom handler or just the TimerEventProcessor (which I understand runs on a different thread but where does the custom handler run?).

The link to the help guide is appreciated. If there are more examples out there I could certainly use that too. I'm currently searching and finding only System.Timers.Timer examples outside NT universe that look a little different, not sure what I can take from there. Will keep looking.

Thanks.

[digibob]

Ok so timer should solve for real time and I hope to figure out eventually how to implement it, but what about backtest? Performing a task at a certain time during historical session when there was no trading activity. I suppose a timer can't be the solution for that unless I'm out in the woods again.

[NinjaTrader_Jim]

Hello digibob and bltdavid,

You would be welcome to use any C# timer you would like but we should also be mindful of how that timer functions when incorporating it with NinjaScript.

System.Timers.Timer can be used freely in the script when System.Windows.Threading.DispatcherTimer will be tied to the UI thread and would work if started from State.Configure or when invoked from ChartControl.Dispatcher (as in the BarTimer script.) I would recommend System.Timers.Timer for simplicity.

digibob, you can access class level variables and objects/properties for your NinjaScript from your custom handler when using a timer. As for backtesting, a timer will not be of much use here because the historical data is processed all at once. (I.E. we do not wait 5 minutes for a 5 minute bar to close when backtesting.) You can test the strategy with the Playback Connection playing back at 1x speed to test the timer's functionality.

We look forward to being of further assistance.

[digibob]

Hi Jim. Thank you for your reply.

The information about timer helps.

As for backtest, the playback connection would not help me test 20 years back. Even if it did, playing back 20 years at 1x speed would take, well...

I'm a bit at a loss about this right now. Maybe one possible workaround is to use Bars.GetTime() in the tests, to check the next bar's time and if later than my scheduled task then the task should be performed immediately after handling the current bar, but then among other things the time signature would be wrong, I may not be able to use SessionIterator, and if it's more than one scheduled task and the time between them does matter then it would all fall apart, and in any case the version running in real time would be logically different from the one running in tests and I'm not sure I would be comfortable with that.

I think there may be room to add this functionality to NT - simulating timer in backtest, not a real timer of course but the ability to perform tasks at given time signatures in between historical bars. Because even if trade orders can't be executed without bars, it doesn't mean there should not be 'life' in between the bars. Having this functionality available in real time but I can't backtest it, how could I determine if I should use it.

Well I guess the availability of timer as it is now is not meant to be part of the algorithm but just for maintenance and utilities. Which means I will have to re-think some things.

Anyway thanks again.

[bltdavid]

I'm a bit at a loss about this right now. Maybe one possible workaround is to use Bars.GetTime() in the tests, to check the next bar's time and if later than my scheduled task then the task should be performed immediately after handling the current bar, but then among other things the time signature would be wrong, I may not be able to use SessionIterator, and if it's more than one scheduled task and the time between them does matter then it would all fall apart, and in any case the version running in real time would be logically different from the one running in tests and I'm not sure I would be comfortable with that.

No, same version. If the Account.Name is "Backtest", do not instantiate the timer.

Only one task is scheduled (it's your code, so you control this). Your code should
"trigger" your task at exactly time xx:xx:xx, there is no "scheduler" involved.

How do you do this?
Inside your timer proc, don't call TriggerCustomEvent until the time is xx:xx:xx.

I think there may be room to add this functionality to NT - simulating timer in backtest, not a real timer of course but the ability to perform tasks at given time signatures in between historical bars. Because even if trade orders can't be executed without bars, it doesn't mean there should not be 'life' in between the bars.

When your timer proc sees the exact time xx:xx:xx you will call the TriggerCustomEvent and
provide a callback method which represents your task to execute. This method is called with
a BarsInProgress context, so you have complete access to such things as Close[0], Time[0], etc.

The execution of your callback method provided to TriggerCustomEvent is immediately inserted
into the same flow as OnBarUpdate, OnPositionUpdate, OnOrderUpdate, etc.

Your calling CustomTriggerEvent from your timer proc is what creates "life" in between the bars,
that "life" is whatever code you provide in your callback method.

Having this functionality available in real time but I can't backtest it, how could I determine if I should use it.

Forget backtesting this feature.

Remember, if the Account.Name is equal to "Backtest", do not instantiate the timer.

Forward test with Sim101 using actual real-time data during market hours.

[digibob]

Hi bltdavid. Your input is appreciated. Let me elaborate that by two versions I did not mean two files. I meant two logical versions since in backtest there is no timer. The only way I can think of how to know in backtest whether a time-based task should be performed right now is by a 'cheating look-ahead' calling Bars.GetTime() with the next bar's index to see if one or more scheduled (and dependent) tasks fall in between the current bar and the next (when saying life in between bars I mean in backtest), then for one thing the use of SessionIterator would have to be altered, for another tasks would be performed in succession in a loop as opposed to once each time as it is in real time - that's already a logical difference - on top of that the strategy makes use of several series with different *trading hours* and time-frames, making the whole thing exponentially more complex.
For example that Bars.GetTime() call is not one call, it's many calls from each BarsInProgress to all the other series to figure out which BarsInProgress is next and when, and whether that next bar falls inside or outside of all the other series' sessions.
It's a mess, trust me
And it's quite a big difference in implementation between real-time/timer and backtest/no-timer. The two implementations may sit in the same file and as you say the timer instantiation will depend on the account name, but that's not the issue. You'd normally want the conditions in backtest to resemble those in real time as much as possible, and that won't be the case here.

In your notes you mostly address real-time and it does help me to understand and see the path to implement it. I'm grateful for that!

Currently my challenge is the backtest implementation. I *am* interested in backtesting it as these scheduled tasks as I call them are not a 'feature', they are the strategy itself. Testing only in real time over several days or weeks would not help me to know if there is 'meat' in it or what results to expect.

Cheers.

[bltdavid]

Currently my challenge is the backtest implementation. I *am* interested in backtesting it as these scheduled tasks as I call them are not a 'feature', they are the strategy itself. Testing only in real time over several days or weeks would not help me to know if there is 'meat' in it or what results to expect.

For backtesting, the only thing I can possibly think of is: add an extra 1-minute or 1-second bar series,
and setup your scheduling logic inside OnBarUpdate when BarsInProgress reflects these "timer" bars.

You mention worry about lack of bars arriving, due to low volume. Well, add this extra bar series for
the most liquid instrument you can for the hours you trade, so that bars on this BIP are effectively
"guaranteed" to always arrive.

But wait ...

What if you mandated that the primary bar series of the chart you start the strategy from must be a
1-minute or 1-second chart for that most liquid instrument? In fact, for whatever liquid instrument
you choose as the primary bar series, just ignore trading on BarsInProgress == 0, that way the
instrument chosen for your primary chart is immaterial to your trade logic.

What I mean is, setup your BarsInProgress == 0 code path to handle timing issues only. The
actual trading of your desired instrument(s) would come from other added bar series.

Regardless of which BarsInProgress becomes your "timer" bar series, such a change could require
fundamental code reorganization efforts, meaning lots of editing on your part. But if use of Strategy
Analyzer is that important and if these precisely scheduled actions constitute the bulk of your secret
sauce, well, a redesign using this kind of approach sounds like a worthwhile experiment.

Good luck!