Set strategy position without submitting order

[sjwelch]

What is the difference? If Historical is false, no historical bars are processed, meaning that only realtime bars are processed. As far as I am concerned, there may be a semantic difference: I fail to see it, and even then that would be irrelevant to me.

The difference is that if I filter out real-time bars the strategy would never do anything after initialization. Historical bars are what you really want to turn on/off. I was being ticky tacky and I apologize.

I do not know how else to say it. That is what I said. If the account is not flat, then process Historical data, so that when the Strategy starts it is going to sync to the Acoount position. That is why I use a parameter that I can specify as needed.

If I cannot communicate the below to you, then I have failed, and I will bow out of this discussion.

//pseudocode:
If account position is not flat, process [I]Historical [/I]bars.
If account position is flat, but processing [I]Historical[/I] bars makes the Strategy inMarket, then do not process Historical bars.

You've described the problem exactly, you just didn't realize it. Code: If my account is not flat, then process historical data. The problem: my account is long, but historical is flat.

Sorry, I know this is confusing, and I realize this is not a problem you would normally encounter. I would prefer not to flatten the account to sync the two. I would like the strategy to realize that it's still long and wait for the next sell signal to flatten the account. This comes back to finding a way to "trick" the strategy into being long/flat/short based on the account position. Again, if anybody has attempted this I would really like some pointers!

[koganam]

The difference is that if I filter out real-time bars the strategy would never do anything after initialization. Historical bars are what you really want to turn on/off. I was being ticky tacky and I apologize.



You've described the problem exactly, you just didn't realize it. Code: If my account is not flat, then process historical data. The problem: my account is long, but historical is flat.

Sorry, I know this is confusing, and I realize this is not a problem you would normally encounter. I would prefer not to flatten the account to sync the two. I would like the strategy to realize that it's still long and wait for the next sell signal to flatten the account. This comes back to finding a way to "trick" the strategy into being long/flat/short based on the account position. Again, if anybody has attempted this I would really like some pointers!

I have, and I have just described how to do it. I use it every time that I have to force sync my strategy to my account. I prefer to sync my Strategy to my account, not my account to the Strategy. I have given you a complete description of how I do it, so I am finding it hard to understand why you are asking for what I have already described.

[sjwelch]

I have, and I have just described how to do it. I use it every time that I have to force sync my strategy to my account. I prefer to sync my Strategy to my account, not my account to the Strategy. I have given you a complete description of how I do it, so I am finding it hard to understand why you are asking for what I have already described.

Ok, maybe I'm really not understanding what you say you're doing. Here's what I believe you are basically doing:

-You start up your strategy. Your account is flat, yet historical says you should be long. You don't want to enter the market late, so you shut down your strategy. Then you enable the boolean that filters out historical bars. Now, you start up your strategy again and since historical is filtered out the strategy is flat as it should be.

If this is wrong please let me know where I'm mistaken. This is what I've essentially done with my strategy, and it worked at first. But here's my problem again:

-I restart my strategy in the middle of a long position. My account is NOT flat, yet historical position is flat (for reasons I described earlier). Again, my logic is that if my account is NOT flat, then I DON'T filter historical bars. Therefore, my account position (long) and strategy/historical position (flat) are out of sync.

What's worse, even if I DO filter historical bars, they would still be out of sync since the strategy would still be flat (even though my account is long).

[koganam]

Ok, maybe I'm really not understanding what you say you're doing. Here's what I believe you are basically doing:

-You start up your strategy. Your account is flat, yet historical says you should be long. You don't want to enter the market late, so you shut down your strategy. Then you enable the boolean that filters out historical bars. Now, you start up your strategy again and since historical is filtered out the strategy is flat as it should be.

That is correct.

If this is wrong please let me know where I'm mistaken. This is what I've essentially done with my strategy, and it worked at first. But here's my problem again:

-I restart my strategy in the middle of a long position. My account is NOT flat, yet historical position is flat (for reasons I described earlier). Again, my logic is that if my account is NOT flat, then I DON'T filter historical bars. Therefore, my account position (long) and strategy/historical position (flat) are out of sync.

Exactly. So do not filter out Historical bars, so that the Historical bars processing will bring the Strategy into the state that it would be historically, which would match the account position. If you say this does not work correctly for you, there is nothing more to be said. Either I am lucky that my computer chooses to cooperate, or what we are describing is not what you are doing.

What's worse, even if I DO filter historical bars, they would still be out of sync since the strategy would still be flat (even though my account is long).

If you filter out historical bars while your account is not flat, you are guaranteed to have your Strategy out of sync with your account, because once you filter out historical bars, your strategy is guaranteed to start out flat. As a matter of fact, that is the whole point of this discussion: how to, at will, ensure that a strategy starts out flat or inMarket, depending on what the account position is.

[sjwelch]

Exactly. So do not filter out Historical bars, so that the Historical bars processing will bring the Strategy into the state that it would be historically, which would match the account position.

This is incorrect. Again, after I restart the strategy, my ACCOUNT is long, my STRATEGY is flat (historical). If I do NOT filter out historical bars as I have been doing, my ACCOUNT would still be long, and my STRATEGY would still be flat. Instead, I want my strategy to be long because my account is long.

I think I know where I'm not making it clear enough: my account was long PRIOR to starting the strategy. When I start up, the strategy is historically flat (again, for reasons I mentioned previously). So, I want the strategy to see that my account position is long, so the strategy position should be set to long as well.

What you (and I) have been doing in our strategies is the inverse of this problem: the account is flat, the strategy is long. Filtering historical will set the strategy flat. To my knowledge, there is no solution for setting the strategy long, which is what I'm trying to figure out.

[koganam]

This is incorrect. Again, after I restart the strategy, my ACCOUNT is long, my STRATEGY is flat (historical). If I do NOT filter out historical bars as I have been doing, my ACCOUNT would still be long, and my STRATEGY would still be flat. Instead, I want my strategy to be long because my account is long.

I think I know where I'm not making it clear enough: my account was long PRIOR to starting the strategy. When I start up, the strategy is historically flat (again, for reasons I mentioned previously). So, I want the strategy to see that my account position is long, so the strategy position should be set to long as well.

What you (and I) have been doing in our strategies is the inverse of this problem: the account is flat, the strategy is long. Filtering historical will set the strategy flat. To my knowledge, there is no solution for setting the strategy long, which is what I'm trying to figure out.

What you are saying is exactly what I am saying. What you are not doing is what I am doing. Are you saying then that there is no way to have the strategy start and show itself as being in a position? I have said repeatedly, in order to have the strategy show its Historical position which will match the account position if there is one, you must process Historical bars. If you are not willing to turn your filter on and off manually as necessary, then I guess you are right.

However, I will point out that at the very start of this thread, you said this: (I have added comments inside the description).

1) I have my strategy filter out historical bars so it uses real-time data only.

(In my scheme of things, this is the exception, not the default. So normally, the Strategy processes and shows its historical position.)

It actually does not matter either way, if the filter will be manually specified as needed, so let us assume that it is still being done with your defaults.

2) I enter the market as expected. My strategy is long and the account is long and I hold the security overnight.

So the strategy goes long, and the account reflect this.

3) I have to restart NT overnight for maintenance. When I restart the strategy, the account is long and the strategy should hypothetically be long, but it's flat because I've filtered out historical bars. (so you have the answer right here in this sentence. You have correctly identifed why the strategy is out of sync. That also tells you how to bring it into sync.)

Allow Historical bars to be processed, again by using the PropertyGrid parameter. When you restart, the Strategy will be long and so will the account, because you are not filtering out Historical signals.

The two are now out of sync and the next sell signal will be missed because the strategy thinks it's flat. - see above.

4) As a result, I edit my strategy to use historical bars upon startup only (using time stamps or some other method)

This is where you are trying to do something automatically that may be doable, but I do not explored. The filtering of Historical bars is done manually as needed, by using a parameter that is exposed in the PropertyGrid.

5) Now, I enable a strategy when the strategy is long but the account is flat. The two are again out of sync and the next exit long signal will force my account short

This is where you will now Stop the strategy, manually filter out Historical bars using the parameter in the PropertyGrid, and restart it. Now the Strategy will start flat, just like the account. i.e., return to your defaults.

The difference is, as I have said once, that you are trying to automatically control the filtering of Historical bars by using some condition in code: I have NOTHING in the code to make a decision about whether to process historical bars, other than to set a single parameter in the PropertyGrid.

Naturally you can reverse the logic of my manual procedures, if your preference is to by default not process Historical bars, but regardless, my method only works if the decision to process Historical bars is made and specified manually by the user, with no attempt by the code to set the filter.

 
if (RealtimeOnly) Historical = false;

is the only line that refers to Historical, and depends on nothing in the code, or time stamps, or anything.

RealtimeOnly is the name of the parameter in the PropertyGrid. Make it true, and Historical bars are not processed: make it false, and Historical bars are processed, but it MUST be done in the PropertyGrid, not by using some condition in code. In my scheme of things, the default for RealtimeOnly is false. I guess given your description, that will be the opposite of what it will be for you, if you are not to rewrite your entire logic.

[sjwelch]

The problem I posted in the original thread was solved thanks to your method. The NEW problem that I have been trying to explain is the opposite of the original problem. Instead of the account starting out flat and the strategy starting out long (the old problem), now the account starts out long and the strategy starts out flat (the NEW problem). Again, no matter what I filter/don't filter for the strategy, the strategy position will remain flat because historical data says that it's flat.

[koganam]

The problem I posted in the original thread was solved thanks to your method. The NEW problem that I have been trying to explain is the opposite of the original problem. Instead of the account starting out flat and the strategy starting out long (the old problem), now the account starts out long and the strategy starts out flat (the NEW problem). Again, no matter what I filter/don't filter for the strategy, the strategy position will remain flat because historical data says that it's flat.

If that is the case, the only way that the historical strategy will be flat while the account is long, is for the strategy to have (historically) closed the position in the overnight hours while the computer was off. So that when you turned the computer on, historical processing would show the position closed.

That points to a logic issue with time. As you are holding overnight, you evidently do not want the position to be closed. Therefore, use a time filter, so that there is no processing in the overnight hours, historical or otherwise.

[sjwelch]

koganam,

I must say, I admire your determination to help! Thanks for sticking with it.

As per your comment, I haven't fully debugged why my account position stayed long while my strategy position flattened, but here's what happened:

I closed NinjaTrader over the weekend, but before I closed it my strategy and account both indicated long. When I reopened it, the strategy indicated flat (my account obviously stayed long). I looked at the strategy performance chart, and saw there was a historical sell signal during the previous market day. This signal did not occur in real-time, so I'm assuming real-time data missed this signal somehow.

I don't know how to prove this theory since real-time data was lost when I restarted the strategy, so I think this will remain a mystery. For now I'm accepting the fact that my account and strategy can become out of sync in this particular way, and I'm wondering what I can do about it. I don't want to change the account position based on the strategy position (the NinjaTrader way of doing things). I'd rather change the strategy position based on the account position. This means I need to set the strategy position long without changing the account position.

BTW, I'm already using a custom template that's only active during 8:30 AM - 3:30 PM EST when the market is open. I'm assuming this filters out both real-time and historical bars outside of this time range?

[koganam]

... I closed NinjaTrader over the weekend, but before I closed it my strategy and account both indicated long. When I reopened it, the strategy indicated flat (my account obviously stayed long). I looked at the strategy performance chart, and saw there was a historical sell signal during the previous market day. This signal did not occur in real-time, so I'm assuming real-time data missed this signal somehow.

I don't know how to prove this theory since real-time data was lost when I restarted the strategy, so I think this will remain a mystery. For now I'm accepting the fact that my account and strategy can become out of sync in this particular way, and I'm wondering what I can do about it. I don't want to change the account position based on the strategy position (the NinjaTrader way of doing things). I'd rather change the strategy position based on the account position. This means I need to set the strategy position long without changing the account position.

Hm. I seem to have seen the same thing happen sometime in the past. As you say, it is impossible to reproduce. At least, in my case, when I tried Market Replay over the period, it correctly processed the signal. More usual though for me, is to have something happen in real time, and Historical not show the signal, especially so, I think, because I enter with Limit Orders.

There is one little technique you can use for filtering out that Historical signal, so that it is not processed.

Create a time filter, and have its defaults be in a time that would normally not trade (per your session template). Better yet, make the start and end times of the filter the same. Now when you want to filter out that pesky Historical signal that messed up the sync, just filter that bar out by setting the time filter to start just before the errant bar, and end just after it.

Remember to reset everything once the trade finally closes.

BTW, I'm already using a custom template that's only active during 8:30 AM - 3:30 PM EST when the market is open. I'm assuming this filters out both real-time and historical bars outside of this time range?

That should. In fact, it does not even draw the bars on the chart.

[sjwelch]

Create a time filter, and have its defaults be in a time that would normally not trade (per your session template). Better yet, make the start and end times of the filter the same. Now when you want to filter out that pesky Historical signal that messed up the sync, just filter that bar out by setting the time filter to start just before the errant bar, and end just after it.

I really like this idea. It could maybe even be automated with a little work. Lately I've been trying to use the unmanaged approach so I don't have to deal with strategy positions at all. If that doesn't work out for me, I'm definitely going to try this way.

Thanks again for the suggestion!