Urgent - Serious Help Needed

[Rustic]

If anyone has any information (support or individual user), please reply.

I have been in hardcore development mode for the better part of 6 weeks now. I have built, tweaked, optimized, and tuned the final set of algos I was planning on using (probably 300 hours worth of effort). I uninstalled and reinstalled NT8 today to get the new version, imported the export of my addon, verified everything is the same.

Now, I am getting COMPLETELY different results, and I'm losing my mind. All of these algos were running and live.

After 2-3 hours of troubleshooting I restored from a NT8 backup on a different computer, and I am now getting the results that I expect, however when I run the strategy analyzer, I am seeing different numbers of trades in total, and trades executing on wholly different days.

Here is a snapshot of one strategy for certain periods of 2011:

New install of NT8

Period	#	Cum. net profit	Net profit	Commission	Cum. max. drawdown	Max. Drawdown	Avg. MAE	Avg. MFE	Avg. ETD
2/3/2011 12:30 PM	1	$250.00	$250.00	$0.00	$0.00	$0.00	$62.50	$750.00	$500.00
3/4/2011 2:30 PM	2	($225.00)	($475.00)	$0.00	($475.00)	($475.00)	$550.00	$475.00	$950.00
3/23/2011 12:30 PM	3	($325.00)	($100.00)	$0.00	($575.00)	($100.00)	$712.50	$562.50	$662.50
3/30/2011 3:30 PM	4	$325.00	$650.00	$0.00	$0.00	$0.00	$37.50	$750.00	$100.00
Old install of NT8
Period	#	Cum. net profit	Net profit	Commission	Cum. max. drawdown	Max. Drawdown	Avg. MAE	Avg. MFE	Avg. ETD
2/3/2011 12:30 PM	1	$246.36	$246.36	$3.64	$0.00	$0.00	$62.50	$750.00	$503.64
3/4/2011 2:30 PM	2	($232.28)	($478.64)	$3.64	($478.64)	($478.64)	$550.00	$475.00	$953.64
3/21/2011 12:30 PM	3	$389.08	$621.36	$3.64	$0.00	$0.00	$62.50	$712.50	$91.14
3/30/2011 3:30 PM	4	$1,035.44	$646.36	$3.64	$0.00	$0.00	$37.50	$750.00	$103.64

I literally dont understand WTF happened. i hate to think that hundreds of hours of my work are now literally utter garbage. I dont understand why NT8 would suddenly decide on a clean install to not execute a trade on the 21st, and to execute it on the 23rd instead for some reason

Please help!!! what the heck do I do? and how do I know which is correct?

[cutzpr]

Are you sure you had all the historical data loaded into your new installation, that you updated the commission profile?

[Rustic]

So, here is the main issue:

If I do a clean install and remove the NT8 directory. Run the Strategy Analyzer once, then import historical chart data, and re-run the Strategy Analyzer I get the odd/wonky results both times

if i do a clean install and remove the NT8 directory, import the historical chart data FIRST and then run the strategy analyzer i get the results I was getting before.

I did a little digging into the actual historical data (exported it from old vs new and compared it), and the data is actually shifted so that the trade data is 7 hours LATER, so in the file where I get the results that I expect instead of the close being at 4pm EST it is at 11pm EST. However, if I dont have any trades exiting on session close, why would that make a difference anyway?

All I want to be able to do is utilize the work which I have had, and have it achieve the same results as before both historically and on a go-forward basis.

Im sick to my stomach because it looks like this issue is going to cause me to lose 300+ hours worth of work.

[sledge]

If anyone has any information (support or individual user), please reply.
...
I dont understand why NT8 would suddenly decide on a clean install to not execute a trade on the 21st, and to execute it on the 23rd instead for some reason

Please help!!! what the heck do I do? and how do I know which is correct?

Time for some more debugging, to see what the difference is in the data on those days OR what the problem in the code is.

one of those 2 things will become evident.

[momchi]

Rustic, in order for us to help you -- please, give us more info on your trading strategy. It would also help to use a terminology used in NT8 (ex. algos - is it a strategy? if so, what kind - is it built with the wizard, or it is custom programmed in C# etc.). I have had similar difficulties to work through, so here are a few things to observe/be careful about:

1. As CutzPr noted - you may have different historical data - are you comfortable determining the types of data used - historical, downloaded, for market replay, or have you changed your "historic data provider" in the settings? (Have you restored these to match your original after the fresh(?) re-install?) If not completely comfortable with all the types of data you can be running your analysis on, please try to cool your emotions (I have had same!!!!) and read through and/or experiment on different sets of data.

2. Notice, the numbers are a little different as you do not have commissions included in the new run

3. To answer the difference on 3/21 vs 3/23 - please plot and look at the chart (in the analysis run) and see if your strategy is not "on the border" of sending a trade order, or not -- examples could be: if you are comparing a floating-point number data (as from a MACD indicator) with another one (price), often there will be a small "floating-point accuracy/resolution" difference even between separate runs on same data - if your "trigger" is sensitive to these differences, it may not trigger to send the order

4. Inspect log for "orders" vs. "executions" and see if there is no issue there - ex. strategy did send an order, but it was not executed (learn about "slippage")

The results you show - are not "drastically different" -- it is only one trade (3/21 vs 3/23) that you need to debug! So, you are close - just take a deeper look.

You seem to be doing very well, and you seem very close to achieving real goals here -- don't let small issues like this discourage you - use them to learn -- BEFORE you put real money on the line! Good luck man !!!

[Rustic]

Okay, so I have been debugging for 5 hours now. A prior life had me doing a lot of tech work and programming.

The issue is not 1 trade, the issue is the trades that are executing are extraordinarily different. A random strategy i run now (with a clean install) has approximately 50% of the trades being executed, the first few trades of 2011 (the baseline year I am using) are the same, but every other set of trades become greatly disconnected. Over a 10 year Period, approximately 200 trades are missing from one algo alone)

Each strategy/algo went from being profitable, to a huge loser. If i start looking at the chart data, and I start to break down the indicators I use, price, and time, the charts, and indicators ALL are identical. When i start to go into the trades/executions they start to become wildly detached. There are a huge number of trades which simply do not get recognized as trades in the 'old' I dont know what is going on.

Honestly, I am just at a loss. I see trades which should execute in the new only executing in the old, and im looking at trades in the old that are not executing in the new. In my opinion the trades that are executing should be a superset of both, but instead, its simply not working that way.

[Rustic]

Also, as a side note, Everything of relevance is occurring in OnBarUpdate. I am bringing in data into State.Configure (like the Adv/.DECL for the McClellan.)

[Rustic]

Ive basically printed out each and every indicator value, and price, and variable in the script, by bar... and the historical data is not the same between the 2 versions, for whatever reason, the first couple months of 2011 are fine, but every month after that has almost no correlation what so ever.

Im sure it's possible, but what could have possibly caused my historical prices to be bad? It literally caused 300+ hours of work to go up in smoke.

[NT-Roland]

Hi Rustic,

Do you use the same datafeed before and after the upgrade? What about Adjusts for Splits and Gobal Merge Policy? What about time zone? Are all settings still the same?

[Rustic]

I duplicated all the settings and verified. I verified that they were identical. If I change settings do I need to drop all historical data and reload?

I verified all of the time zones as well. I don’t even know how to get back to where I was.

Same datafeed.

[Rustic]

I’ve tried every permutation you can think of of uninstalling, deleting the NT8 folder, reinstalling connecting to a data provider , and choosing a connection (including signing up for a new demo account). I can’t replicate what I had before.

ive exported the old data and compared it to the new data... and the interesting thing is... roughly the first 10-15k Rows from each front month contract are not in the new. But the 2nd half absolutely does.

for example in the 3-19 contract every record until 12/23 does not match. But every record after that does. And all the settings are the same.

I did vlookups between both sets of data and the data simply does not match prior to 12/23

[NinjaTrader_Jim]

Hello Rustic,

Thanks for your post. And thank you to all of the community members who have given valuable advise.

I did vlookups between both sets of data and the data simply does not match prior to 12/23

Ok, it sounds like you have determined the issue has to do with a difference in historical data being available on the new platform. Data providers generally hold data going a certain range back, so requesting for some set of data may be available one week, but the next week that data may be replaced with newer data that the provider is tracking. If you see a specific data discrepancy, you could open a new chart of that data and use Reload All Historical to request a new data set to fill any gaps or discrepancies. You could also move the data off your platform (stored in the Documents/NinjaTrader 8/db/ folder) so then you can compare a fresh data request to what you already have.

In line with the advise others have given in this thread, the same data + the same code + the same settings will equal the same results. If you are getting different results, something must be different in one of these areas. Checking where your trades start to go wrong and comparing the indicator values and data points at that time can better determine why those trades are wrong and can help make data issues like this easier to spot.

if i do a clean install and remove the NT8 directory, import the historical chart data FIRST and then run the strategy analyzer i get the results I was getting before.

This would be a consistent way to assure that you are bringing in the same data to perform the same test and expect the same results. The fact that you are stating this tells us that there may have been some different data fetched on the old platform, or the full amount of data was not requested on the new platform when this data was not imported manually. The bottom line is that strategy backtests can start performing completely differently when the underlying data is different.

If there is anything else we can do to help, please let us know.

[koganam]

Ive basically printed out each and every indicator value, and price, and variable in the script, by bar... and the historical data is not the same between the 2 versions, for whatever reason, the first couple months of 2011 are fine, but every month after that has almost no correlation what so ever.

Im sure it's possible, but what could have possibly caused my historical prices to be bad? It literally caused 300+ hours of work to go up in smoke.

You may have different merge settings on the installations. That will always cause there to be a difference in the historical data that the program sees.

Does this happen with a stock or ETF? If not, that will point towards confirmation of a merge settings difference.