Strategy backtest - still runs multiple times sometimes

[misova]

There was a bug in NT8, where
backtest in Strategy Analyzer ran multiple times, instead of only once, as would be expected.
Problem was related to the count of dataseries added in the strategy - if there were 5 data series added, then backtest ran 6 times sequentially....

This still happens - not always, but sometimes.
For now, I don't see the pattern, what causes this behaviour,
so I can reproduce it reliably. But it occurs each day, when doing development.
It is not so rare at all.

I want to say: The bug is not completely solved.
There are still some scenarios, when multiple backtests are running in sequence.

When this happens, backtest takes much more time = as many iterations are running sequentially. I handle this problem by forced shutdown of NT8 for now. After restart, it works fine for some time...

[NinjaTrader_JessicaP]

Thank you for your report misova . We will be investigating this on our end using the Sample MA Crossover strategy. If there is any additional information you come across that would be useful please let us know.

[misova]

When reproducing this bug, I would suggest
to not use simple strategy. It would be just waste of time without result, because, it never happens there.

Some meaningful way to get to this bug, is to have strategy,
that have several additional data-series - at least 4 or more,
so it is easily distinguishable, when the bug happens.

The bug is directly related to the new data-series added = if strategy has 4 dataseries added and the bug happens, then you will see 5 iterations running in sequence.

If only 1 primary dataseries is added - then there the bug just never happens.

My subjective feeling is, that it happens only during heavy backtesting, i.e.:

• run parallel multiple backtests at once - in multiple tabs
• run small optimization + together with backtests
• and so on... some more complex scenarios, when some thread conditions fail and result in this bug.

[misova]

Today, it is especially horrible to work with this problem.
I had to kill NT8 several times, because when running backtest,
it makes 5 iterations, instead of one. (I have one primary + 4 added data-series = 5 together).

It happened multiple times, so now I see the pattern - when it happens better.

It happens when multiple backtest are running concurently.
Common scenario is:

1. Run 1 backtest of strategy A
2. Run 2nd backtest of the same strategy A in another tab

I know, threading issues are hard to identify in code, but this
should be really fixed, because it is so inconvenient,
when one has to kill NT8 several times a day, because it freezes
by running multiple iterations of same backtest and consuming full memory...

[NinjaTrader_JessicaP]

When I attempted this using multiple instances in different tabs of each of the Sample Multi-Instrument and Sample Multi-Timeframe strategies, basket testing over the Dow 30 and similar lists and testing on single instruments, I was unable to generate extra backtest runs. Are there any other steps I can take on my end?

[misova]

Really nothing comes to my mind.

It just happens - when it happens - seemingly randomly,
but frequently enough, so it is clearly bug.

I also cannot reproduce, when I want,
or when I run just some backtest.

It happens during my backtesting work,
often after multiple backtests were ran, several compilations, switching strategies in Analyzer, and so on

I agree, this is extra tough to identify - why / when it happens.
Let's see, if I spot some more clues during my work...

[NinjaTrader_JessicaP]

We appreciate your continued interest. If it is at all possible to attempt to cause this with the built-in sample multi-series strategies, simply knowing that this definitely occurs or does not occur with these may be enough of a clue on our end to discover the cause of this behavior.

[Yemmy]

BackTest Not Showing Profit and Loss

Hello,

I'm new this forum. I opened a 20000 demo account with FXCM, and I also downloaded historical data for five months so I can use it for backtest. I noticed whenever I do the backtest the profit and loss amount does usually show. Please see the attachment. Thanks in anticipation to your reply

[NinjaTrader_JessicaP]

Thank you for your question Yemmy . It will make it easier for other users experiencing similar things if you start separate topics in their own forums threads.

It will be easier to answer questions about strategies not producing expected results if we could answer as many of the following questions as possible. Could you take a few minutes to review these questions? I am happy to explain further any of the below questions and the rationale behind asking them.

• Do you see expected results when running the same test environment on the SampleMaCrossOver strategy in NinjaTrader with a 3 and 14 period?
• • By expected results, I mean that the SampleMACrossover places trades whenever a 3 and 14 period SMA cross
• Who are you connected to? This is displayed in green on lower left corner of the Control Center window.
• Are you connected to your data feed provider when running this test?
• What instrument(s) (and expiry if applicable) have you selected?
• What Data Series Type have you selected? Example: Tick, Minute, Day
• What From and To date is selected?
• Is your strategy a multi instrument or multi time frame strategy?
• Do you receive an error on screen? Are there errors on the Log tab of the Control Center? If so, what do these errors report?
  

[Yemmy]

Hello,

Like I told you I'm new in this forum I couldn't just figure out how to start a new post. Please give me directions. Thanks

[NinjaTrader_JessicaP]

This video demonstrates how to create a new forums thread.

https://www.screencast.com/t/eqECFIpINA

I look forward to learning more about your strategy and to assisting further.

[Yemmy]

Hi

In Response to your questions

Do you see expected results when running the same test environment on the SampleMaCrossOver strategy in NinjaTrader with a 3 and 14 period? Yes I do see a cross over on the 60min chart it appears the strategy is executing well

• By expected results, I mean that the SampleMACrossover places trades whenever a 3 and 14 period SMA cross Yes I do see a cross over on the 60min chart it appears the strategy is executing well

Who are you connected to? This is displayed in green on lower left corner of the Control Center window. I'm connected to FXCM and in the bottom left corner it has fill color of green " Connected- FXCM"
Are you connected to your data feed provider when running this test? Yes, as at when I did the test I'm still connect to FXCM because the fill color is showing green
What instrument(s) (and expiry if applicable) have you selected? I don't quite understand what you mean be instrument but I did the backtest on EURUSD currency pair
What Data Series Type have you selected? Example: Tick, Minute, Day. I selected every tick in the backtest parameters but the timeframe was on the 60min
What From and To date is selected? I think it was from 1st of February 2017 to April ending 2017
Is your strategy a multi instrument or multi time frame strategy? I will assume multi instrument is the same as multi timeframe. I was just on one timeframe which is the 60min
Do you receive an error on screen? Are there errors on the Log tab of the Control Center? If so, what do these errors report? No, I don't receive any error

[NinjaTrader_JessicaP]

Thank you for the answers to these questions Yemmy. This indicates

• NinjaTrader is working correctly underneath your strategy
• There are no runtime errors which are preventing your strategy from placing trades

This leaves one possibility : your trade conditions are not being met during backtesting. The most likely cause of this is that your strategy is designed not to work during backtesting. If you see a line of code such as

[FONT=Courier New]if (Historical) return;[/FONT]

This means that you will have to re-design your strategy so that it can be used during backtesting.


A good way to start debugging your strategy would be to create a strategy that is guaranteed to place trades during backtesting. For example,

[FONT=Courier New]protected override void Initialize()
{
    SetProfitTarget(CalculationMode.Ticks, 5);
    SetStopLoss(CalculationMode.Ticks, 5);
}
[/FONT]
[FONT=Courier New]protected override void OnBarUpdate()
{
    if (Position.MarketPosition == MarketPosition.Flat)
    {
        EnterLong();
    }
}[/FONT]

I have attached such a strategy. Once you have proved to yourself this strategy places trades, you can add logic from your intended strategy to this test strategy piece-by-piece until you have determined the part that prevents you from placing trades.


We are happy to help in any way we can.

[Yemmy]

Hi,
The strategy that I did a backtest on I got it from one of NinjaTraders YouTubes video dated 5/17/03. The only different thing I did from what the instructor did is that I ran my backtest on EURUSD. His result showed PnL my didn’t. I even did a backtest on some strategy installed into my NT 7 by default. I was getting the result but no PnL. If there is way I can get the strategy file and send it to you please let me know, But I will do what you asked me do. Although I not good programmer the code line "if (Historical) return;" do I see before or after running the program. where do I click to check for it. is it on edit, compile or view
Thanks for your response I just started using NinjaTrader about a week and half ago and my primary reason for switching over from MT4 to NinjaTrader is the capability of the software to run a multiple time frame strategy. I hope I would be able to do this on NT 7
Thanks

[NinjaTrader_JessicaP]

The strategy we put together in our training videos is the SampleMACrossover strategy you tested in post #12. We won't need to examine any other code, since you have already confirmed this works.

To enable a strategy and get PnL

• Connect to your data feed provider
• Visit the Strategies tab of the Control Center
• Right-click -> New Strategy
• Select your strategy from the pulldown -> Configure your strategy -> OK
• Check the "Enable" box next to your strategy on the far right

Once your strategy has placed trades, you can see your PnL in the strategies tab. You can visit the Account Performance tab and click Generate to see more information.