• If this is your first visit, you will have to register before you can post. To view messages, please scroll below and select the forum that you would like to visits. Questions? Be sure to check out the Forum FAQ.

Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

All GomRecorder Indicators

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    #16
    Data not saved

    Gomi,
    Disabling time filter had no effect. Further, I removed the recorder and kept the CD with recording status true, file format binary and ran replay for couple of days. The binary file was the same size at the end of the replay even though "Recording Binary OK" was showing all along. Pressing F5 erased all the CD data and reset to zero. This shows no data was saved.
    FYI I am testing it on NT B5

    Comment


      #17
      Maybe file is corrupt ? Could you try removing it ?

      EDIT : Also keep in mind you can't "backfill" data preceding last written date. You can only write data past the last written date.
      Last edited by gomifromparis; 12-06-2009, 10:18 AM.

      Comment


        #18
        Yes, file was corrupt. Good call. Now it seems to be running as expected. Glad with it.

        Ninja is a good product but it's you Gomi that give it that edge for traders like myself. Thank you, for w/o your contributions, other charting platforms would have drawn my interest.

        SA

        Comment


          #19
          Thanks

          I added the GomFileConverter in post #1 with documentation in post #5

          Comment


            #20
            Gomi you really went over the top with the volume ladder. I am sure many a MD customer is singing the praises of Gomi. Thanks so much.

            Comment


              #21
              BRILLIANT work again Gomi!!!

              Comment


                #22
                lastprice

                Calculation Mode: UpDownTick
                volume is up if price>lastprice

                How do you calculate lastprice say on a 10 minute bar, is lastprice=opening price or can you look at tick data inside every bar ?

                .

                Comment


                  #23
                  Gom:

                  I really love what you have done. The next step should be increasing the stability and reliability of the on disk data-storage infrastructure.

                  I also have had a variety of problems recording data. The time-stamp would update when I do an reload of NS or restart NT but the size would not change and so on.

                  As you suggested it might be a case of binary file corruption.
                  -----------------------------------------------------------------
                  The next step in the development of the GOM tick data-storage system is to add some fault-tolerance and error-detection & reporting mechanisms.

                  -> Print out a warning to the output window if the binary file reader/writer discovers an error. Produce an audio clue (a beep) to warn users to check.

                  -> The next step, of course would be to recover as much data as possible from the corrupted file.

                  -> When it comes to reducing the chance of corruption, I am not a C#/MS guru but perhaps doing periodic flushing of the write buffer might help reduce the chances of corrupted files.
                  ---------------------
                  An alternative approach is to have a flow which tries to ensure that the bin file remains uncorrupted as much as possible.


                  (a) We could first save the tick information as a text file and then run a utility to convert into the binary format and then append it to the existing binary file for that instrument. The text format will allow manual repairs of the tick data and the conversion to binary will be useful for archival purposes. Further we can maintain different text files for different sessions (timestamp the name) so that the binary can be incrementally updated

                  (b) The dispose() method of the RecorderIndicator can call the function to perform the text to bin conversion for that session. If the function fails to convert the text file it prints a message and exits without corrupting the binary file, and printing a text message in the log/output window and also the $GOMDIRECTORY about what was wrong. Instead of working directly with the bin file, create a fresh copy while updating, and then rename it to the original bin file after its post-update intrgrity has been verified.
                  ---------
                  Added later:

                  I noticed that you have already added the converter in the fifth post of this thread (which was previously reserved for future use). Much of what I have suggested above can now be automated using some Window Scripts (which I am not good at writing (hint)...)

                  Also noticed that the ConversionUtility is not using the $GOMFOLDER env variable to produce the output file. Also the Window title "Form1" could be changed (yes, yes I am being picky, but this is a thing of beauty and any blemish stand out).
                  (c) We would also need a command line version of the GOMTXT2BIN converter which can be called to convert the text files to bin after the text file has been fixed.
                  Last edited by aviat72; 12-09-2009, 10:01 PM.

                  Comment


                    #24
                    Originally posted by deanz View Post
                    Calculation Mode: UpDownTick
                    volume is up if price>lastprice

                    How do you calculate lastprice say on a 10 minute bar, is lastprice=opening price or can you look at tick data inside every bar ?

                    .
                    lastprice is the last tick price, so it looks at every tick inside the bar.

                    Comment


                      #25
                      It seems to me that the SMoving Average is not customizable. The parameter is (5) on default and I am not able to change the settings

                      Comment


                        #26
                        Originally posted by Sergio66 View Post
                        It seems to me that the SMoving Average is not customizable. The parameter is (5) on default and I am not able to change the settings
                        Yep, sorry, bug in 6.5...

                        Comment


                          #27
                          Nice post

                          Some thoughts :

                          1. This "corruption" thing seems rare, I haven't encountered myself, but there are some messages on the board mentioning it... It might be a bug but it's hard to analyze since I've never encountered it.

                          2. The write buffer is flushed on every tick.

                          3. Automatic process in Dispose() may be more a problem than the solution. File reading/writing may be quite time consuming with large files or small CPU computers, so shutown or indicator removal will take a long time, eventually users will kill the process and perfect files will become corrupt

                          4. If you really need file completeness I don't think you can avoid a backfill external process. I use QCollector to downlaod daily/weekly the data, and I construct my historical file myself. Who knows when your computer will reboot/NT will crash/power will shut down etc..

                          5. I think a more robust solution for file backup/restoration would be to install a software like Crashplan. This sofware will allow you to backup the file as frequently as you like, even every 5 minutes if you wish. You will be able to restore any version from the past. It incorporates file deduplication so only modified parts will be saved. It is able to backup open files using Volume Shadow Copy. It is able to backup on another computer in case your disk crahes. Great software, but with free version you'll only be able to backup daily.

                          6. Command line for the converter is a good idea. Had the idea but have been more or less lazy :-)





                          Originally posted by aviat72 View Post
                          Gom:

                          I really love what you have done. The next step should be increasing the stability and reliability of the on disk data-storage infrastructure.

                          I also have had a variety of problems recording data. The time-stamp would update when I do an reload of NS or restart NT but the size would not change and so on.

                          As you suggested it might be a case of binary file corruption.
                          -----------------------------------------------------------------
                          The next step in the development of the GOM tick data-storage system is to add some fault-tolerance and error-detection & reporting mechanisms.

                          -> Print out a warning to the output window if the binary file reader/writer discovers an error. Produce an audio clue (a beep) to warn users to check.

                          -> The next step, of course would be to recover as much data as possible from the corrupted file.

                          -> When it comes to reducing the chance of corruption, I am not a C#/MS guru but perhaps doing periodic flushing of the write buffer might help reduce the chances of corrupted files.
                          ---------------------
                          An alternative approach is to have a flow which tries to ensure that the bin file remains uncorrupted as much as possible.


                          (a) We could first save the tick information as a text file and then run a utility to convert into the binary format and then append it to the existing binary file for that instrument. The text format will allow manual repairs of the tick data and the conversion to binary will be useful for archival purposes. Further we can maintain different text files for different sessions (timestamp the name) so that the binary can be incrementally updated

                          (b) The dispose() method of the RecorderIndicator can call the function to perform the text to bin conversion for that session. If the function fails to convert the text file it prints a message and exits without corrupting the binary file, and printing a text message in the log/output window and also the $GOMDIRECTORY about what was wrong. Instead of working directly with the bin file, create a fresh copy while updating, and then rename it to the original bin file after its post-update intrgrity has been verified.
                          ---------
                          Added later:

                          I noticed that you have already added the converter in the fifth post of this thread (which was previously reserved for future use). Much of what I have suggested above can now be automated using some Window Scripts (which I am not good at writing (hint)...)

                          Also noticed that the ConversionUtility is not using the $GOMFOLDER env variable to produce the output file. Also the Window title "Form1" could be changed (yes, yes I am being picky, but this is a thing of beauty and any blemish stand out).

                          Comment


                            #28
                            Originally posted by Sergio66 View Post
                            It seems to me that the SMoving Average is not customizable. The parameter is (5) on default and I am not able to change the settings
                            Gomi,
                            I really appreciate your work. I suck at programming and was wondering if it would be easy to fix the GomCDSMA? Also, could it be changed to an EMA?

                            Thanks,
                            Mpe

                            Comment


                              #29
                              I dug out the constant delta bar type and posted it on first post. Documentation and chart is a few posts later.

                              Comment


                                #30
                                Sorry I have no time but you can modify this code in Initialize ()
                                Code:
                                    
                                #if !NT7
                                  indic=SMA(GomCD(GomCDCalculationModeType.BidAsk,GomCDChartType.CumulativeChart,GomFileFormat.Binary,GomFilterModeType.None,1,false).DeltaClose,5);
                                #endif
                                remove the 5 and type period
                                replace SMA with whatever you want

                                Sorry


                                Originally posted by mpe66 View Post
                                Gomi,
                                I really appreciate your work. I suck at programming and was wondering if it would be easy to fix the GomCDSMA? Also, could it be changed to an EMA?

                                Thanks,
                                Mpe

                                Comment

                                Latest Posts

                                Collapse

                                Topics Statistics Last Post
                                Started by Mykro, Today, 02:16 PM
                                1 response
                                10 views
                                0 likes
                                Last Post NinjaTrader_Jesse  
                                Started by tonynt, Today, 02:06 PM
                                1 response
                                4 views
                                0 likes
                                Last Post NinjaTrader_Jesse  
                                Started by itrader46, Today, 01:53 PM
                                1 response
                                2 views
                                0 likes
                                Last Post NinjaTrader_Jim  
                                Started by maindoor, Today, 01:06 PM
                                1 response
                                10 views
                                0 likes
                                Last Post NinjaTrader_PatrickG  
                                Started by faustf, 11-07-2019, 01:49 PM
                                6 responses
                                20 views
                                0 likes
                                Last Post NinjaTrader_PaulH  
                                Working...
                                X