Performance considerations and issues when running against multiple instruments...

[fxRichard]

As I mentioned in another post, I run a strategy across hundreds of stocks. Before I would kick off one instance of the strategy per symbol but have moved to adding all symbols into one instance of the strategy.

From your guys perspective what is the better way based on the way Ninja has been built?

My assumption is as follows:

1 Strategy - 500 Instruments
Lower memory footprint, less startup time and less resources required.
Synchronized Single threaded OnBarUpdate calls, possible lag in processing data when running against OnTick updates?

500 Instruments - 500 Strategies
Higher memory footprint, longer startup time, more resources required.
Each strategy instance runs in its own thread I assume, less lag when processing OnTick updates as they are asynchronous.

Are these assumptions correct? If so I may want to move back to the second option.

Additionally when I run against this many symbols and actually place trades through Ninja simulator I eventually get database contention issues and the logs fill up with this message:

2018-05-16 10:34:31:980|3|524288|Error on executing DB command: System.Data.SqlServerCe.SqlCeException (0x80004005): A duplicate value cannot be inserted into a unique index. [ Table name = Orders,Constraint name = PK__Orders__000000000000033E ]
   at System.Data.SqlServerCe.SqlCeCommand.ProcessResults(Int32 hr)
   at System.Data.SqlServerCe.SqlCeCommand.ExecuteCommandText(IntPtr& pCursor, Boolean& isBaseTableCursor)
   at System.Data.SqlServerCe.SqlCeCommand.ExecuteCommand(CommandBehavior behavior, String method, ResultSetOptions options)
   at System.Data.SqlServerCe.SqlCeCommand.ExecuteNonQuery()
   at NinjaTrader.Cbi.Order.DbAdd()
   at NinjaTrader.Cbi.DB.DBThread()

When this happens Ninja basically hangs at that point. This only happens when my strategy is actively placing a lot of orders through Ninja in both managed and unmanaged approach. When I move my order entries into my another external integration this becomes a non issue.

[NinjaTrader_JoshG]

Hello fxRichard,
Thanks for your post.

I do not think that it would be much of a difference either way. On one hand, putting multiple instruments in a single strategy would make it easier to modify at scale and adapt, but having the same strategy running on different instruments would likely take more time.

The thing is, you are pushing your PC(and the platform) really hard when you are running 500 strategies on one instrument or 500 instruments on one strategy. I would expect these types of errors to occur when I push my PC this hard as well.

That being said, we obviously do not want these errors to occur. I suspect that this error is a result of the strategy itself but I'd like to be sure. Try running the 'SampleMACrossover' strategy with multiple instruments( as many as you need to reproduce the error ) and also with multiple instances of the strategy.

[fxRichard]

Thanks for your response Josh.

The machine I'm running on is more than capable. I don't believe this to be due to the strategy as this only happens when placing orders through Ninja at a rapid pace against the live market. This issue doesn't happen when running backtests only against live market when we are placing orders through Ninja. As I mentioned if we move order management out of Ninja it's no problem and it hums along fine.

I'll see what I can throw together to run this test and share.