Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

Delay problem with realtime Data, crash, ignored issues!

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Delay problem with realtime Data, crash, ignored issues!

    I am using NT 7.0.1000.1, In fact I even bought it, since I believe in your efforts to improve the program and that you are going to remove remaining obvious bugs.

    I observed the following problems, they are more or less connected, easy to reproduce, annoying, but unresolved for a long time. I want to clarify that none of the following issues happend due to inferior hardware, in fact the system used for testing is one of the fastest single CPU systems available with plenty of RAM and a fast SSDD, win7 64 bit.


    for a long time a known issue:
    while backtesting, NT becomes mostly unresponsive - that means if you try to shift e.g. a NT chart window, it is laggy and takes time, the degree of unresponsiveness can vary.

    This leads to a problem:
    Problem 1: I observed that during a backtest, incoming realtime Data is obviously not loaded correctly into the NT Database (for open charts) - realtime chart data can become delayed - that means during and after the backtest, my PC clock shows e.g. 9.40am, but the chart and chartdata latest time is 9.34am. Even if I open a new chart (after the backtest) - the time and data is still delayed and will not adjust. Workaround: restart, reload of data.
    1. After a restart the realtime data is displayed correctly again.
    2. Reloading historical while running live strategies also works.

    But:
    Problem 2: both "workarounds" will make you loose the display of "Realized profit"in the strategy tab, since strategies got stoppend and restarted. (long time known issue)!
    Problem 3: For those strategies, the lines connecting your prior realtime trades in the performance graph window will be displayed incorrectly - weird, unconnected to chart, and your realized profit for "historical and realtime" will be for sure wrong! (known issue) - this basically alway happens when you shut down NT unproperly while simulating or trading realtime - it does not matter how- pc crash, power loss, kill NT process in Task manager, NT crashes, etc.

    Solution: not sure, but it might be that you'll have to add one or 2 more extra columns in your DB to fix it, but I am sure it is possible, since as of now, NT looks very incomplete regarding this issue.

    Problem 4: If you try scroll back and forth that performance graph with the weird lines, NT will eventually crash with a big red X across the screen!
    (known issue)

    To work around problem 1 to 4 it is mandatory after an NT restart to remove and add all strategies manually(to charts and/or strategy tab) and to set again all parameters for each strategy. By doing so I will loose performance of realtime (simulated) trades.
    But better than having another crash by looking at the wrong performance results if I do not set up all strategies anew.
    Claimed strategy persistence for realtime or realtime simulated trades is not working properly yet. Please fix! (the FREE NT part might be working, but what for you expect users to pay is not yet stable)

    Problem 5:
    The most dangerous problem:
    Sometimes, if many strategies are run at the same time, NT can become unresponsive for a second or 2 as well, or for any other reasons.
    *** That means there as well realtimedata can become delayed !!! ***
    So there is a true risk, that while running simulation or LIVE strategies, that MAYBE at some point, your realtime data is not in sync with what NT believes to be realtime.
    NT will then react to an ordersignal, that was 5 Minutes ago... - can be very expenisve. Please FIX all those issues asap, since they are very very very important!

    I can reproduce them anytime! I discussed the weird lines in the graph during realtimesimulation + NT restart or crash with you a long time ago, but somehow you did not consider it as a important issue it seems. The bug is still there, and the red X is still there.


    Another unconnected issue I would have liked to mention once again, lets call it Problem 6:

    The Print(" ") method should be ignored during/for backtests!!! Since it is useless - if not explain why please.
    If you have a complex strategy with several parameters, you might need "Print()" to help you debug, understand etc.

    This happens if I forget to put // or /* */ in front of every Print() method before I start a more complex backtest:
    - After a very short time into the backtest, my 12 gig RAM are filled up with worthless Print("") data. Result: The backtest will slow down to almost a halt and /or NT will freeze.This depends directly on numbers of iterations.

    It is rather timeconsuming /repetitive to add // or /* */ to ignore the Print() methods and to remove them after the test again... and even worse: sometimes I forget to make NT ignore the "Print()" or I oversee one single Print() - result:
    NT will freeze, I need to wait for a long time or shut down NT with Task Manager (the more common approach) - this whole print() issue interrupts the workflow a lot.

    Please make the backtest routine ignore any "print()" methods.
    Last edited by azuul; 12-16-2010, 07:49 AM. Reason: or

    #2
    azuul, thanks for taking the time to post your concerns and experienced issues - would you mind sending in your diagnostic trace / log files via the Help > Mail to Support option so we can review them as well and respond?

    The Print method in the backtest is considered useful, oftentimes you need to debug a value in historical sequence as well...you would need to add code in your scripts to be able to toggle the print / debug mode on / off as needed.
    BertrandNinjaTrader Customer Service

    Comment


      #3
      Bertrand, thanks for your reply.

      I do not believe that while having several (ten-) thousand iterations during a backtest - that u can see anything beside hundret thousands lines of useless text. It is simply information overkill and leads to a crash of NT. Therefore I still cannot see any advantages.

      For seeing print in historical context - which is for sure very useful - I just open a chart with desired timeframe, add the strategy with the desired parameters and open the output window with the print() enabled - then I enable the strategy - I will have all the historical prints I can desire to further analyse it in context with chart/price/indicator-movement.

      The other described issues are all reproduceable - you do not really need a trace or log, please try yourself - I believe that would help. Some issues I encountered already in rather early versions, some I discussed with NT staff already, and I also sent some tracelog back that time. I suggest you use NT as I described and experience it it firsthand, maybe it is not (only) a bug, but a "strategic" GUI or DB flaw that needs work urgently.

      I recall someone from NT-staff back that time concluded that nothing can be done about the weird lines after restarting NT + previously having executed simulated orders. I believe you should reconsider, it delivers wrong realtime results as described in my first post and the eventually crash with big X shows that even your NT software gets crazy about those wrong results and weird lines.

      Since NT is running a long time and is getting restarted from time to time, it is sometimes hard and timeconsuming to point to the exact time in the log if there is nothing obvious. I "believe" the most relevant message before having that red X was:

      2010-12-16 09:58:31:250 in OnUnhandledThreadException
      2010-12-16 09:58:31:250 ********* exception trapped *********
      2010-12-16 09:58:31:250 Der Objektverweis wurde nicht auf eine Objektinstanz festgelegt.
      2010-12-16 09:58:31:250 bei NinjaTrader.Data.MemBars.GetHigh(Int32 index)
      bei NinjaTrader.Data.Bars.GetHigh(Int32 index)
      bei NinjaTrader.Gui.Chart.ChartExecution.GetBounds(Cha rtControl chartControl, ChartPanel chartPanel, ScaleJustification scaleJustification)
      bei NinjaTrader.Gui.Chart.ChartPanel.CalculateMinMax(S caleJustification scaleJustification)
      bei NinjaTrader.Gui.Chart.ChartControl.CalculateMinMax AndMargins(Graphics graphics)
      bei NinjaTrader.Gui.Chart.ChartControl.OnPaint(Object sender, PaintEventArgs e)
      bei System.Windows.Forms.Control.OnPaint(PaintEventArg s e)
      bei System.Windows.Forms.Control.PaintWithErrorHandlin g(PaintEventArgs e, Int16 layer, Boolean disposeEventArgs)
      bei System.Windows.Forms.Control.WmPaint(Message& m)
      bei System.Windows.Forms.Control.WndProc(Message& m)
      bei System.Windows.Forms.ScrollableControl.WndProc(Mes sage& m)
      bei System.Windows.Forms.Control.ControlNativeWindow.O nMessage(Message& m)
      bei System.Windows.Forms.Control.ControlNativeWindow.W ndProc(Message& m)
      bei System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

      ............
      2010-12-16 10:31:39:793 Gui.WorkspaceOptions.ApplicationExit
      2010-12-16 10:31:40:816 Gui.Globals.AppExit: deactivating hot keys
      2010-12-16 10:31:40:816 Gui.Globals.AppExit: shutting down news timer
      2010-12-16 10:31:40:816 Gui.Globals.AppExit: shutting down message timer
      2010-12-16 10:31:40:816 Gui.Globals.AppExit: shutting down file type watcher
      2010-12-16 10:31:40:816 Gui.Globals.AppExit: shutting down ATI server
      2010-12-16 10:31:40:816 Gui.Globals.AppExit: disposing autoTrade component
      2010-12-16 10:31:40:816 Gui.Globals.AppExit: shutting down data server
      2010-12-16 10:31:40:816 Gui.Globals.AppExit: shutting down SMTP server
      2010-12-16 10:31:40:816 Gui.Globals.AppExit: closing connections
      2010-12-16 10:31:40:832 (z_esignal) Cbi.Connection.Close1
      2010-12-16 10:31:40:832 (z_esignal) Cbi.Connection.Close2
      2010-12-16 10:31:40:832 (z_esignal) Cbi.Connection.Close3
      2010-12-16 10:31:40:832 (z_esignal) Cbi.Connection.Close4
      2010-12-16 10:31:40:863 (z_esignal) Cbi.Connection.Close5
      2010-12-16 10:31:40:863 (z_esignal) Cbi.Connection.Close6
      2010-12-16 10:31:40:863 (z_esignal) Cbi.Connection.Close8
      2010-12-16 10:31:40:863 (z_esignal) ESignal.StandardAdapter.Disconnect
      2010-12-16 10:31:40:863 (z_esignal) Cbi.Connection.ProcessEventArgs.ConnectionStatusEv entArgs Disconnected Disconnected NoError
      2010-12-16 10:31:40:863 (z_esignal) Cbi.Connection.Close9
      2010-12-16 10:31:40:863 Gui.Globals.AppExit: closing replay connection
      2010-12-16 10:31:40:863 Server.Client.Close
      2010-12-16 10:31:40:879 Cbi.Globals.ApplicationExit
      2010-12-16 10:31:40:879 Db.Globals.ApplicationExit
      2010-12-16 10:31:40:894 Cbi.License.ApplicationExit
      2010-12-16 10:31:40:910 Data.Collective2Adapter.ApplicationExit
      2010-12-16 10:31:40:910 Checking backup reminder...
      2010-12-16 10:31:40:910 ************************ Session End ************************


      I do not really want to "convince" you of the issues I mentioned, just inform that they are still there - I already discussed them a longer time ago. I would not spend so much time in that forum post if they were not important and recurring. I just do not have the time and ressources to report every single incident, some issues disappeared with later releases, but since this is your RC - and no one else seems to feel responsible to report those things, I do believe the things mentioned have to be addressed quickly to deliver a reliable and complete product.

      I sent tracefile
      Last edited by azuul; 12-16-2010, 09:58 AM.

      Comment


        #4
        azuul, to clarfiy a few things here - no issues has been ignored in the beta process, however for all to be addressed ones we would need a reproducible scenario that we can follow step by step on our machines to pinpoint the issue for our development team. I will look through the summarized issues you raised, but the trace exception you posted resembles a bug we had a good time ago, so I just replied to your email if you're on a upgrade or clean install of NT7 R1?

        For the Prints, I can see what you say and will forward it for consideration - thanks for the feedback, for now I would consider simply adding a debug switch in your code to be able to turn those off when not needed for working on a code issue.
        BertrandNinjaTrader Customer Service

        Comment


          #5
          Betrand, thanks for your reply -
          I understand that you put a lot of effort in your support, which is for sure unique of its kind.
          The idea with the switch sounds like a feasible workaround.
          I upgraded NT from the last 1 (or 2) version for sure, as far as I remember I also reinstalled NT new at one point - but copied the Ninjatrader.sdf file from a different machine, because I quickly needed the collected tickdata.... there was no complaint from NT at that point and everything seemed to work just fine...
          So the copied sdf file could be a reason for the problems!!? I would be more than happy if there were no more weird lines or wrong historical and realtime perfomance after reenabling a live simulated strategy!! - In this case I could have saved a lot of keystrokes and owe u an apology.
          Last edited by azuul; 12-16-2010, 11:29 AM.

          Comment


            #6
            No worries, I just replied by email how achieve a clean R1 install - I believe it's worth a try for you at this point. You can save tick data directly from the MyDocuments > NinjaTrader 7 > DB subfolders and restore later, the SDF now does not contain this anymore as it was the case with 6.5
            BertrandNinjaTrader Customer Service

            Comment


              #7
              Bertrand,

              I deinstalled NT, removed all files and folders. After the clean install of the the latest version I only copied the DB folder (without the NinjaTrader.sdf file) and bin/custom to the correct locations - everything seemed ok at first.

              After a few backtests there was suddenly specific tickdata missing in the charts (September was there, half October, November completely missing, and Dec was there again). The data seems to be in the DB, but somehow NT is not adding correctly to the charts. NT restart, reload of data, repair of DB is not helping to solve the issue, copying the DB froma backup also not helping - going to reinstall NT completely once again...

              What I observed so far - the timelag is still there, and it is connected to the laggy behavior of NT if it is "calculating".

              I had a few tick and a 5/10/30 second chart of the same instrument opend - FDAX
              running a few strategies at the same time (RAM and CPU were almost inactive)

              Then I started a shorter backtest (5 Minutes) - during the "loading instrument" phase, NT is completely not responsive to whatsover, charts do not actualize data.

              Also during the first 1 or 2 Minutes into the Backtest, NT is hardly responsive, Abort button is NOT working. After pressing it 100 times in a row, eventually NT is aborting (after 50 times hitting abort without success you start to contemplate that the task manager seems your best friend) - Abort button only works if NT "awakes" for a second from being not repsonsive - if you miss that second, you have to wait again half a minute for your next chance.... CPU is about 50-70%, Ram 50% used, during that backtest I could open and play around with any program in windows without having a laggy behavior of my OS or the other program.

              What happend is: after the backtest was finished and the lag stopped, the chartdata was delayed about 2 Minutes.
              I tried the same again, chart Data delayd approx 4 Minutes.
              Opening new chart, Minute, Tick or Second: same result, the latest time on the chart is 4 Minutes behind.
              Funny observation: the 5 second chart had its "own" time after that: - sometimes the 5 second bars appears after 5 second, sometimes a bar appears after 15 second and sometimes 4 bars appear in 3 seconds.
              Result: the charts came closer again to the real-time - from 4 Minutes to about 90 seconds - only to fall back again to 120 seconds after that and then came close again to 100 seconds. Only restart of NT fixed this problem or a reconnect to Dataprovider

              The timelag of charts can also happen without a backtest I believe... it has maybe to do with a minor laggy behavior of NT if it is under "load" of whatever kind.

              Is there a way to make sure NT is "realtime"? Do you have any solution to fix this?


              thanks
              Last edited by azuul; 12-17-2010, 06:02 AM.

              Comment


                #8
                Even after fresh reinstall Data is missing - I think it is maybe not in the DB subfolder "tick/FDAX 12-10 - not even of my backup folder of the old DB (which was working fine)

                but obviously some of the missing tick data might be in the subfolder "cache/Eurex Equity Index Futures.W_Europe Standard Time/tick/FDAX 12-10

                The files there are named differently - is there a way to bring them back into the DB folder or to make NT "read" them into the cache??

                I had still hopes - because I also "exported" the Future tick data before deinstalling NT - but here also a problem: the bid /ask tick data for the december FDAX goes inexplicably only until 01. November.. ( did the export yesterday).

                the export of the "last" data shows exactly the gap with missing data 16.10 till 1.12.
                Bid/Ask/Last are 45178, 45192, 45181kb in size.

                Looks like a file size restriction that caused data loss????

                This is getting better and better...
                This data was/is very important for my work....!!!!
                Last edited by azuul; 12-17-2010, 06:42 AM.

                Comment


                  #9
                  azuul, you likely see this due to a corrupted cache repository - to rebuild it please leave the data as is in the DB and exit NT.

                  Then go to: MyDocuments\NinjaTrader 7\db\cache and delete the subfolders here (leave the parent folder intact!).

                  Then restart NT, reconnect and recheck - it would now rebuild the cache per session template used.
                  BertrandNinjaTrader Customer Service

                  Comment


                    #10
                    I will try reproducing your backtest lag observations here, to iron out any custom strategy code getting in the way - would you mind rechecking with the per default provided SampleMACrossOver strategy which I will use in testing as well?

                    Thanks for the support,
                    BertrandNinjaTrader Customer Service

                    Comment


                      #11
                      It did not work - I did a small edit to my prior post - check again maybe t his explains why it did not work...

                      The tick data is provided only 2 weeks back from e-signal. i am still having the gap. The data seems missing in the DB /tick folder!! The data SEEMS to be in the cache folder! It was working fine this morning, I did not touch the backup folder, the backup folder contains the same DB files and the actual folder, the october seems complete, and still after deleting the cache folder, the chart displays only till 15.th october and from 2nd December on.
                      November is in both DB missing !!! The days missing seem to be in the "cache" folder of my backupfolder.

                      /MyDocuments /Ninjatrader 7 is actal installation directory
                      /MyDocuments / NT7 backup / Ninjatrader 7 is the location of the backup - does NT somehow acces or change the files there??

                      If I install NT new and delete all DB and copy the backup DB files and the strategy again same as yesterday it SHOULD be working, since it worked this morning to a certain point?!

                      It is confusing I have no idea what I could have done wrong....

                      Maybe it has to do with the export of DB files???? bid ask was only exported to 1.11 ? Maybe this corrupted the db somehow shortly before I made a clean install?
                      Last edited by azuul; 12-17-2010, 07:15 AM.

                      Comment


                        #12
                        I installed NT new completely, and copied the DB files again (takes rather long time) :

                        Now I do have data in November, more than before it seems, but still there are days missing - for Tick as well as for Minute Data! How does this happen and what to do? Before a new install of NT I had no gaps at all

                        Comment


                          #13
                          azuul, there would be no file size limitation for the export - have you checked into your Historical Data Manager directly to see what data is actually stored in it for the tick and time granularities?

                          The backup folder would not be touched by NT - only when you choose the File > Utilities > BackUp and Restore command.
                          BertrandNinjaTrader Customer Service

                          Comment


                            #14
                            Bertrand, I am confused. About fdax dec 2010_
                            Historical Data Manager/ Edit/ last / tick/ half of October is missing and November completely.

                            When I open a chart tick data for the last 3 month, the data is there! (where does it come from?) - am having still a 3 days gap in November tick chart which was not there the last weeks and months. And the last clean install before i had a 6 week gap. (half october and november)

                            I do not understand where the data comes from if not from the DB and why the chart has gaps and sometimes no gaps and sometimes more gaps. It seems random... Minute chart also has gaps now, but different ones. Please clarify /help to have consistent data with which I can work. Before the NT reinstall I had no gaps, after the first clean install a huge gap, after the 3rd clean install now smaller ones - always followed the same steps and only copied DB files from backup into new folder.(and strategies)
                            Right now I am deleting the cache folders again as instsructed by you to see what will happen.

                            About the lagging chart: Traceorders= true will slow NT down (not the cpu and not the memory) cpu around 40% - no difference if traceorders is true or not - "loading of data" can take very long - during which time NT is almost completely frozen - during that time chart can start to lag behind it seems.
                            Falls probably into the same category as the print() issue.
                            Key quesition is nevertheless how NT does make sure to be "realtime" - if there are only smaller lags during backtest, it does seem to have no effect, even if chart hangs for 20 seconds, after that time quickly 4*5 second bars are added and all is fine. But what if NT lags for a longer time for whatever reasons?

                            EDIT : after deleting the cache folder (as instructed) the gap becomes huge again - and half october and full november missing: As I suspected it seems NT is pulling Data from the cache folder to fill missig DB data, or am I wrong? Why is the data not in the DB in the first place??? I had NT runnung most of the time 24/7 during October and November and December.... why is it in cache instead? And is it possible to bring it back into the DB in case it is missing there?

                            2 full days in row nothing but trying to fix NT problems and in the end creating bigger problems for me than I had before...this is a bit frustrating.

                            Any idea/suggestion where to get complete tick data for the DEC FDAX for the last 4 month?
                            Last edited by azuul; 12-17-2010, 09:12 AM.

                            Comment


                              #15
                              azuul, sorry for the frustrations experienced getting this to work for you - would you mind contacting me through Help > Mail to Support and including your chart data and DB so I could look into for you on my end?

                              The cache is build from the DB data, this in turn would hold historical data downloaded from the provider and realtime data received and stored as you were connected (depending on options set under Tools > Options > Data).

                              Are you sure you copied over all the data contained in the tick and minute folders as you did you fresh install?

                              Are you always creating the same internal / type chart with the same session templates applied?

                              Loading tick data for extended periods can unfortunately take some times, we would not suggest trying to load it during peak times if this would result in maxing out your local resources.

                              You can normally get long tick histories directly at the exchanges for example or dedicated tick data services where txt / ascii files are offered for download / import to fill gaps in the local repository.
                              BertrandNinjaTrader Customer Service

                              Comment

                              Latest Posts

                              Collapse

                              Topics Statistics Last Post
                              Started by DJ888, 04-16-2024, 06:09 PM
                              4 responses
                              12 views
                              0 likes
                              Last Post DJ888
                              by DJ888
                               
                              Started by terofs, Today, 04:18 PM
                              0 responses
                              11 views
                              0 likes
                              Last Post terofs
                              by terofs
                               
                              Started by nandhumca, Today, 03:41 PM
                              0 responses
                              7 views
                              0 likes
                              Last Post nandhumca  
                              Started by The_Sec, Today, 03:37 PM
                              0 responses
                              3 views
                              0 likes
                              Last Post The_Sec
                              by The_Sec
                               
                              Started by GwFutures1988, Today, 02:48 PM
                              1 response
                              9 views
                              0 likes
                              Last Post NinjaTrader_Clayton  
                              Working...
                              X