Trades on chart are correct but performance report is totally WRONG

[Zigman]

I've been running a simple strategy on a real time data feed (on a simulated account) using a "15 minute linebreak" chart for a couple days.

The fills that appear on the screen capture of the chart are 100% CORRECT.
(i.e. trading on an abstraction such as a line break chart can result in fills that are not anywhere near the body of the candle.)

The issue became evident when I ran a performance report and reviewed the individual trades...there is clearly something very wrong going on here.

The underlying program is a simple "always in the market" flip flop LONG-to-Short type program. So every Long should be followed by a Short and every Short should be followed by a Long - that was the case until trade #9 at which point all the trades are listed as LONG. Furthermore, the dollar value of the respective gains and losses are incorrect.

Please help me understand what is going on/why/how to fix it.

There is no reason for a line break chart to not be able to generate long/short signals that execute in the "real world" correctly.

Note that all trades are market orders (not limit orders) and the strategy in question was created with the wizard i.e. managed approach = nothing complicated at all.

I'm running Ninja version 7.0.1000.10 and the data is CQG

[Zigman]

trace and log and execution files for the above

trace and log and execution files as well as the cs file pertaining to my first post.

[NinjaTrader_Matthew]

Hello,

Thanks for posting your logs.

I'm unable to locate the Close and Buy for 1.2730 from log files. This would tell me that the performance report is correct, but the strategy calculated 'virtual' historic trades are incorrect.

To be clear, what you're seeing on the chart is the same as running a backtest and there at times can be discrepancies in real-time vs backtesting:

http://www.ninjatrader.com/support/helpGuides/nt7/discrepancies_real_time_vs_bac.htm

There's no information in your diagnostics to know why the trade was determined when the script was reloaded, but did not happen in real-time.

It's not often that you'd see extreme changes in price and indicator values on an 15 minute chart and the data is reloaded but it is possible. What I'd suggest is using Print() statements to print the values of the bars and indicator values in real-time and then compare them to when they are reloaded. This should give us an idea of exactly what is different and we can look into ways to ensure your strategy runs in real-time as it does in backtesting to the best in our abilities.

[Zigman]

more detail

Hi Matthew,

I didn't do the best job explaining the issue - let me try again:

First of all I am very familiar with discrepancies in real-time vs backtesting:
http://www.ninjatrader.com/support/h...ime_vs_bac.htm

As you noted, I agree that it is true (and to be expected) that the 'virtual'/historic trades are not reflective of reality. (i.e. results of a back test or the "historical" trades that are automatically drawn on a chart when the user enables a strategy on a chart)

And, even more so, in the case of a line break chart where 'virtual'/historic trades always "get filled" on the "open" of the bar where in real life the "open" may in fact have been way beyond the other end of the bar in question (i.e. real time fills are often nowhere near the body of the bar they supposedly "traded" in).

- None of the above is what I'm talking about in post #1 and #2 and none of it applies.

The fill (to close the profitable short trade) that occurred from 1.2747 to 1.2730 was not a 'virtual' or an historic trade but a live trade WITNESSED BY ME IN REAL TIME.

The CHART IS A 100% ACCURATE representation of what happened while I was watching the screen. So while you believe that the trades listed on the "Trades" tab of the performance report are accurate - I can certify that they are not.

This IS the problem.

Note that this is on top of the fact that all the real time trades after trade #9 on the "Trades" tab of the performance report are "LONG" !!! - This is impossible given that the strategy flips back and forth long-short-long-short 100% in the market at all times (real time entries are correctly shown on the chart long-short-long-short etc.)

It is impossible to have sequential "long" trades as is the case in the "Trades" tab of the performance report. Once again the "real time" trades chart is CORRECT and the real time trades listed on the "Trades" tab of the performance report are not.

Hopefully this more specific explanation will allow for more meaningfull examination on your part.

[NinjaTrader_Matthew]

So you're saying that you witnessed a trade in real-time, but it is not in your database, nor is it in the log files you provided?

[Zigman]

Correct.

Correct.

That's why I was totally shocked/dismayed when I saw the (real time) trades in the "Trades" tab of the performance report...

[shanemcdonald]

I went through all of this during the summer.
What I found out is this:

When I right click on a strategy from the strategy window and select real time performance report for a strategy, the report is not accurate and cannot be relied on whatsoever. Ninja Trader should communicate this major bug to everyone.

Only under the account performance tab can you find accurate performance results.
I have to have a separate sim account for each strategy I run on one instrument.
I have 100 sim accounts to test strategies on CL. You can run strategies on other instruments in the same account, but each CL strategy must be in a separate account.

Now you can do a performance report on each strategy from this tab with the advanced tab and selecting the instrument and the single account.

This is the only way to get accurate real time performance reports for my strategies.

The chart tab under those results will only allow you to view 1 minute charts on the results.

I hope this helps as I went through many corrupt and misleading performance reports before stumbling upon this.

No one was able to help me with this, and I did get a message that it may be a confirmed bug, but they could not duplicate it in real time.

The only way I discovered it was by watching it unfold in real time.

Again , I find that right clicking on a strategy from the strategy tab, and selecting real time performance report will yield unreliable, misleading and unacceptable results which cannot be used for analysis.

Try it and see for yourself.

sincerely
shane

[shanemcdonald]

also you will find the trades tab in each of these techniques completely different.

Place strategies in a separate account. Only one instrument per account.
Now you have accurate reports by running performance reports on that account from the account performance tab.

Reports from that strategy tab, when you right click and select real time performance report, are not accurate.

Compare the results from one instrument and strategy with these 2 different techniques.
You will see the difference.

[shanemcdonald]

I am referring to strategies being tested in real time. This is all real time testing.
I do not do backtesting.

[NinjaTrader_Matthew]

Correct.

That's why I was totally shocked/dismayed when I saw the (real time) trades in the "Trades" tab of the performance report...

Zigman,


To verify the information Shane has reported, does your 'real time' strategy performance match your Account performance tab for this time period?

[NinjaTrader_Matthew]

As an update, I believe you guys are running into separate issues.

Zigman,

The log files would show exactly what occurred at that time.

If you look at your logs for that day, you'll see that rather than the strategy going from long to short 1 contract, it starts going from 1 long to 2 long and flat and back to long for several trades. This explains why the performance report just shows Long trades for this period.

This starts to occur before the day you provided the log for. This indicates that your strategy position and account position got off on this day. To prevent this from occurring, I'd suggest adding MarketPosition checks in with your strategy logic to ensure that the strategy is taking trades in the direction you expect.


Shane,

If you were able to resolve this behavior by running each strategy an separate account, what you're running into is the way that trades are going to be paired on the account. When you're running multiple strategies in once account, you'll have to keep in mind that the trades are going to be paired with FIFO in the performance when comparing the single strategy in real-time.