• If this is your first visit, you will have to register before you can post. To view messages, please scroll below and select the forum that you would like to visits. Questions? Be sure to check out the Forum FAQ.

Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

^TICK index tick data processed asynchronous

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    ^TICK index tick data processed asynchronous

    Hopefully you can explain why the following occurs.
    I have a simple indicator that plots the ^Tick high, low, and close values.
    BarsInProgress 0 is a 30 second chart.
    BarsInProgress 1 is a 1 tick ^Tick data series.
    Calculate = On bar close.
    Every once in a while I notice the last tick from BarsInProgress 1 is processed after BiP 0 has processed.
    Here is what I'm seeing in the NS Output windows...
    BiP-1 26660 : 08:24:22 | Closes[0]: 136 tickHigh: 148 tickLow: -41 (OBC) .Realtime
    BiP-1 26661 : 08:24:23 | Closes[0]: 108 tickHigh: 148 tickLow: -41 (OBC) .Realtime
    BiP-1 26662 : 08:24:24 | Closes[0]: 110 tickHigh: 148 tickLow: -41 (OBC) .Realtime
    BiP-1 26663 : 08:24:25 | Closes[0]: 80 tickHigh: 148 tickLow: -41 (OBC) .Realtime
    BiP-1 26664 : 08:24:26 | Closes[0]: 80 tickHigh: 148 tickLow: -41 (OBC) .Realtime
    BiP-1 26665 : 08:24:27 | Closes[0]: 72 tickHigh: 148 tickLow: -41 (OBC) .Realtime
    BiP-1 26666 : 08:24:28 | Closes[0]: 32 tickHigh: 148 tickLow: -41 (OBC) .Realtime
    # BiP--0 888 : 08:24:30 | Closes[1]: 32 (OBC) .Realtime
    BiP-1 26667 : 08:24:29 | Closes[0]: 30 tickHigh: 30 tickLow: 30 (OBC) .Realtime
    !! This tick was processed late! Primary bar time: 08:24:30 Secondary bar time:08:24:29
    BiP-1 26668 : 08:24:30 | Closes[0]: 38 tickHigh: 38 tickLow: 30 (OBC) .Realtime
    BiP-1 26669 : 08:24:31 | Closes[0]: 30 tickHigh: 38 tickLow: 30 (OBC) .Realtime
    BiP-1 26670 : 08:24:32.003 | Closes[0]: 22 tickHigh: 38 tickLow: 22 (OBC) .Realtime
    BiP-1 26671 : 08:24:33 | Closes[0]: 22 tickHigh: 38 tickLow: 22 (OBC) .Realtime


    and

    BiP-1 26903 : 08:28:25 | Closes[0]: -195 tickHigh: -153 tickLow: -243 (OBC) .Realtime
    BiP-1 26904 : 08:28:26 | Closes[0]: -187 tickHigh: -153 tickLow: -243 (OBC) .Realtime
    BiP-1 26905 : 08:28:27 | Closes[0]: -219 tickHigh: -153 tickLow: -243 (OBC) .Realtime
    BiP-1 26906 : 08:28:28 | Closes[0]: -189 tickHigh: -153 tickLow: -243 (OBC) .Realtime
    # BiP--0 896 : 08:28:30 | Closes[1]: -189 (OBC) .Realtime
    BiP-1 26907 : 08:28:29 | Closes[0]: -201 tickHigh: -201 tickLow: -201 (OBC) .Realtime
    !! This tick was processed late! Primary bar time: 08:28:30 Secondary bar time:08:28:29
    BiP-1 26908 : 08:28:30 | Closes[0]: -207 tickHigh: -201 tickLow: -207 (OBC) .Realtime
    BiP-1 26909 : 08:28:31 | Closes[0]: -153 tickHigh: -153 tickLow: -207 (OBC) .Realtime
    BiP-1 26910 : 08:28:32 | Closes[0]: -171 tickHigh: -153 tickLow: -207 (OBC) .Realtime
    BiP-1 26911 : 08:28:33 | Closes[0]: -157 tickHigh: -153 tickLow: -207 (OBC) .Realtime
    BiP-1 26912 : 08:28:34 | Closes[0]: -143 tickHigh: -143 tickLow: -207 (OBC) .Realtime
    BiP-1 26913 : 08:28:35 | Closes[0]: -161 tickHigh: -143 tickLow: -207 (OBC) .Realtime


    or sometimes BiP 0 processes to late, like this...
    BiP-1 16825 : 12:10:26 | Closes[0]: 111 tickHigh: 209 tickLow: 103 (OBC) .Realtime
    BiP-1 16826 : 12:10:27 | Closes[0]: 81 tickHigh: 209 tickLow: 81 (OBC) .Realtime
    BiP-1 16827 : 12:10:28 | Closes[0]: 55 tickHigh: 209 tickLow: 55 (OBC) .Realtime
    BiP-1 16828 : 12:10:29 | Closes[0]: 47 tickHigh: 209 tickLow: 47 (OBC) .Realtime
    BiP-1 16829 : 12:10:30 | Closes[0]: 37 tickHigh: 209 tickLow: 37 (OBC) .Realtime
    BiP-1 16830 : 12:10:31 | Closes[0]: 43 tickHigh: 209 tickLow: 37 (OBC) .Realtime
    BiP-1 16831 : 12:10:32 | Closes[0]: 33 tickHigh: 209 tickLow: 33 (OBC) .Realtime
    BiP-1 16832 : 12:10:33 | Closes[0]: 7 tickHigh: 209 tickLow: 7 (OBC) .Realtime
    BiP-1 16833 : 12:10:34 | Closes[0]: 15 tickHigh: 209 tickLow: 7 (OBC) .Realtime
    BiP-1 16834 : 12:10:35 | Closes[0]: 9 tickHigh: 209 tickLow: 7 (OBC) .Realtime
    # BiP--0 1121 : 12:10:30 | Closes[1]: 9 (OBC) .Realtime
    BiP-1 16835 : 12:10:36 | Closes[0]: -7 tickHigh: -7 tickLow: -7 (OBC) .Realtime
    BiP-1 16836 : 12:10:37 | Closes[0]: -53 tickHigh: -7 tickLow: -53 (OBC) .Realtime
    BiP-1 16837 : 12:10:38 | Closes[0]: -55 tickHigh: -7 tickLow: -55 (OBC) .Realtime


    When that occurs the wrong Close value is plotted.


    90% of the time the BiP is processed synchronously like this...
    BiP-1 26933 : 08:28:55 | Closes[0]: -149 tickHigh: -141 tickLow: -251 (OBC) .Realtime
    BiP-1 26934 : 08:28:56 | Closes[0]: -167 tickHigh: -141 tickLow: -251 (OBC) .Realtime
    BiP-1 26935 : 08:28:57 | Closes[0]: -141 tickHigh: -141 tickLow: -251 (OBC) .Realtime
    BiP-1 26936 : 08:28:58 | Closes[0]: -159 tickHigh: -141 tickLow: -251 (OBC) .Realtime
    BiP-1 26937 : 08:28:59 | Closes[0]: -147 tickHigh: -141 tickLow: -251 (OBC) .Realtime
    # BiP--0 897 : 08:29:00 | Closes[1]: -147 (OBC) .Realtime
    BiP-1 26938 : 08:29:00 | Closes[0]: -131 tickHigh: -131 tickLow: -131 (OBC) .Realtime
    BiP-1 26939 : 08:29:01 | Closes[0]: -79 tickHigh: -79 tickLow: -131 (OBC) .Realtime
    BiP-1 26940 : 08:29:02 | Closes[0]: -47 tickHigh: -47 tickLow: -131 (OBC) .Realtime
    BiP-1 26941 : 08:29:03 | Closes[0]: 1 tickHigh: 1 tickLow: -131 (OBC) .Realtime
    BiP-1 26942 : 08:29:04 | Closes[0]: 11 tickHigh: 11 tickLow: -131 (OBC) .Realtime

    Please find attached the indicator and a chart image, since chart templates can't be attached.

    Click image for larger version  Name:	Chart and Data box.png Views:	0 Size:	68.2 KB ID:	1076920Click image for larger version  Name:	Chart NQ and TICK.png Views:	0 Size:	50.0 KB ID:	1076917

    Is the indicator coded wrong? Does the ^Tick index 1-tick data series not synchronize correctly?
    Bottom line is how do we get the indicator to match the ^TICK bars OHLC values on the chart?
    Attached Files
    Last edited by zacwhitesi; 11-06-2019, 12:37 PM.
    zacwhitesi
    NinjaTrader Ecosystem Vendor - Shark Indicators

    #2
    Hello zacwhitesi,

    Thank you for your post.

    So that I may try to reproduce this on my end as accurately as possible, what data provider are you connected to? I'd first like to rule out that it's something with the data itself rather than a code issue first.

    Do you see this occur if you download Market Replay data for today and run it over the same time?

    Thanks in advance; I look forward to assisting you further
    Kate W.NinjaTrader Customer Service

    Comment


      #3
      Hi there, it's CQG.

      DEBUG [2019-11-06 07:02:01,357] - Certificate with subject [CN=www.cqgtrader.com, OU=Gateway, O=CQG Inc., L=Denver, S=Colorado, C=US] has passed verification at CertificatePolicy
      DEBUG [2019-11-06 07:02:01,364] - Certificate with subject [CN=www.cqgtrader.com, OU=Gateway, O=CQG Inc., L=Denver, S=Colorado, C=US] has passed verification at CertificatePolicy
      DEBUG [2019-11-06 07:02:01,367] - Certificate with subject [CN=www.cqgtrader.com, OU=Gateway, O=CQG Inc., L=Denver, S=Colorado, C=US] has passed verification at ServerCertificateValidationCallback
      INFO [2019-11-06 07:02:01,535] - ASLU: ReportClientActivity(******,*****) method calling at https://login.cqgnet.com/AUTHENTICAT...THSERVICE.ASMX auth service
      INFO [2019-11-06 07:02:01,699] - ASLU: ReportClientActivity method at https://login.cqgnet.com/AUTHENTICAT...THSERVICE.ASMX returned request status: Success
      Last edited by zacwhitesi; 11-06-2019, 02:14 PM.
      zacwhitesi
      NinjaTrader Ecosystem Vendor - Shark Indicators

      Comment


        #4
        Hello zacwhitesi,

        Thank you for your reply.

        I've run this for awhile today, and didn't see this occurring; however much of the day I wasn't running it on the same data feed.

        I'm going to run this for awhile longer tomorrow and see if I can get this to trigger.

        Thanks in advance; I look forward to resolving this for you.
        Kate W.NinjaTrader Customer Service

        Comment


          #5
          Is CQG sending the ^TICK index with a timestamp, or is Ninja assigning a timestamp when the tick is received?

          FYI, the ^TICK bars, as far as I can tell in my spot checking, always close on the correct tick. i.e. No extra tick or missing tick, like the indicator incurs.
          zacwhitesi
          NinjaTrader Ecosystem Vendor - Shark Indicators

          Comment


            #6
            Hello zacwhitesi,

            Thank you for your reply.

            CQG timestamps are native to their data. This makes me wonder if perhaps your computer clock is off just enough to cause an occasional discrepancy between the time a tick is stamped for and NinjaTrader's time functions which run off your computer clock in real time. If you look at historical data or run back over that time period in Market Replay, does the same issue occur?

            If it doesn't, it's likely the PC clock being off just a tiny little bit.

            Please use the steps below to sync the PC clock:
            • Shutdown NinjaTrader
            • Right-click the clock in the lower right corner of your desktop
            • Select Adjust date/time
            • For Windows 10 click 'Additional date, time, & regional settings' -> then click 'Set the time and date'
            • Select the 'Internet Time' tab at the top
            • Set the server to time.nist.gov and then click Update.
            • If the message that appears says successful your PC clock should now be updated.
            • If not, select a different server from the Server: drop-down and try again (repeat until one of the servers is successful)
            After updating the PC clock, please restart NinjaTrader, then right-click the chart and select Reload All Historical Data. Then test the indicator again on real time data.

            Please let us know if we may be of further assistance.
            Kate W.NinjaTrader Customer Service

            Comment


              #7
              I use a program called Dimension 4, which synchronizes the clock every time Windows starts up. The clock is usually adjusted by a few tenths of a second.
              I'll get back to you about the market playback test.
              zacwhitesi
              NinjaTrader Ecosystem Vendor - Shark Indicators

              Comment


                #8
                Yes, ^TICK ticks are also out of sync with Playback Connection.

                BiP-1 25194 : 07:59:56 | Closes[0]: 109 tickHigh: 115 tickLow: 86 (OBC) .Realtime
                BiP-1 25195 : 07:59:57 | Closes[0]: 99 tickHigh: 115 tickLow: 86 (OBC) .Realtime
                BiP-1 25196 : 07:59:58 | Closes[0]: 87 tickHigh: 115 tickLow: 86 (OBC) .Realtime
                # BiP--0 1679 : 08:00:00 | Closes[1]: 87 (OBC) .Realtime
                BiP-1 25197 : 07:59:59 | Closes[0]: 95 tickHigh: 95 tickLow: 95 (OBC) .Realtime
                !! This tick was processed late! Primary bar time: 08:00:00 Secondary bar time:07:59:59
                BiP-1 25198 : 08:00:00.023 | Closes[0]: 116 tickHigh: 116 tickLow: 95 (OBC) .Realtime
                BiP-1 25199 : 08:00:01 | Closes[0]: 76 tickHigh: 116 tickLow: 76 (OBC) .Realtime
                BiP-1 25200 : 08:00:02 | Closes[0]: 46 tickHigh: 116 tickLow: 46 (OBC) .Realtime

                BiP-1 25554 : 08:05:56 | Closes[0]: 231 tickHigh: 285 tickLow: 216 (OBC) .Realtime
                BiP-1 25555 : 08:05:57 | Closes[0]: 261 tickHigh: 285 tickLow: 216 (OBC) .Realtime
                BiP-1 25556 : 08:05:58 | Closes[0]: 243 tickHigh: 285 tickLow: 216 (OBC) .Realtime
                # BiP--0 1703 : 08:06:00 | Closes[1]: 243 (OBC) .Realtime
                BiP-1 25557 : 08:05:59 | Closes[0]: 213 tickHigh: 213 tickLow: 213 (OBC) .Realtime
                !! This tick was processed late! Primary bar time: 08:06:00 Secondary bar time:08:05:59
                BiP-1 25558 : 08:06:00.003 | Closes[0]: 189 tickHigh: 213 tickLow: 189 (OBC) .Realtime
                BiP-1 25559 : 08:06:01 | Closes[0]: 213 tickHigh: 213 tickLow: 189 (OBC) .Realtime
                BiP-1 25560 : 08:06:02 | Closes[0]: 205 tickHigh: 213 tickLow: 189 (OBC) .Realtime


                ect.
                zacwhitesi
                NinjaTrader Ecosystem Vendor - Shark Indicators

                Comment


                  #9
                  Hello zacwhitesi,

                  Thank you for your reply.

                  Just so I can make sure whether we're testing on identical data or not, did you download the market replay data from the Historical Data window, or do you have your NinjaTrader installation set up to record data for Market Replay?

                  Thanks in advance; I look forward to resolving this for you.
                  Kate W.NinjaTrader Customer Service

                  Comment


                    #10
                    It was downloaded. I do not record it.
                    zacwhitesi
                    NinjaTrader Ecosystem Vendor - Shark Indicators

                    Comment


                      #11
                      Hello zacwhitesi,

                      Thank you for your patience.

                      I've noticed this occurring as well and after taking another look at the code and data, the issue is that the secondary series isn't of the same instrument so ticks are coming in at different times.

                      Basically, you have your primary 15 second NQ series, and you've got a 1 tick ^TICK series. Let's look closely at a spot where we had an out of order print:

                      Click image for larger version

Name:	2019-11-11_1216.png
Views:	23
Size:	29.4 KB
ID:	1077364
                      Here we've got an "out of order" looking print. We can see that the 14:10:30 bar for BIP == 0 isn't printing until after what you'd think should be the first two seconds of that bar.

                      The thing is that until the first tick of that bar comes in, OnBarUpdate isn't going to be triggered because it doesn't have anything coming in to tell it it should be making a new bar.

                      If we pull up an NQ 15 second chart and then add in a single tick chart, we can see that there was a tick recorded at 14:10:29.219, but the next tick didn't come in until 14:10:32.059, triggering OnBarUpdate and printing what appears to be out of order.

                      Click image for larger version

Name:	2019-11-11_1246.png
Views:	20
Size:	68.5 KB
ID:	1077365Click image for larger version

Name:	2019-11-11_1248.png
Views:	21
Size:	72.4 KB
ID:	1077366Click image for larger version

Name:	2019-11-11_1248_001.png
Views:	21
Size:	76.4 KB
ID:	1077367

                      If you were using the same instrument and expiry for both the primary and the secondary series, you wouldn't have to worry about this, but because you're using two different instruments and mixing time based and non-time based intervals, you can't guarantee that ticks will come in exactly at the same times.

                      Please let us know if we may be of further assistance to you.
                      Kate W.NinjaTrader Customer Service

                      Comment


                        #12
                        Hi Kate,
                        Yeah, that makes sense. I realized that after posting that scenario, and should have removed it.
                        My main concern is when the ^TICK data is processed late, after the primary bar closes. See the first two Output window text from post #1, and post #8.
                        Thanks.
                        zacwhitesi
                        NinjaTrader Ecosystem Vendor - Shark Indicators

                        Comment


                          #13
                          Hello zacwhitesi,

                          Thank you for your patience.

                          I've been testing this further, and while I've encountered the situation I went over in my last post several times, I've yet to encounter what you're seeing.

                          I'm wondering if now if you may be experiencing some mild connection quality issues from time to time as that can cause quite a few different issues - if ticks are being slightly delayed in getting to you that could certainly cause the behavior you're seeing.

                          Would you be able to send your log and trace files in to us directly so I can see if there's any quality issues being reported? You can do so in the platform by going to Help > Mail to Support. Log and Trace should be checked automatically. Please put 2327531 ATTN Kate W. in the subject line and a link to this forum thread in the message body.

                          Thanks in advance; I look forward to assisting you further.
                          Kate W.NinjaTrader Customer Service

                          Comment

                          Latest Posts

                          Collapse

                          Topics Statistics Last Post
                          Started by rontronimous, Today, 05:55 PM
                          0 responses
                          4 views
                          0 likes
                          Last Post rontronimous  
                          Started by bcgeek, Today, 03:43 PM
                          2 responses
                          6 views
                          0 likes
                          Last Post bcgeek
                          by bcgeek
                           
                          Started by Mberg, Today, 04:05 PM
                          0 responses
                          7 views
                          0 likes
                          Last Post Mberg
                          by Mberg
                           
                          Started by dkbyond, Today, 03:52 PM
                          0 responses
                          4 views
                          0 likes
                          Last Post dkbyond
                          by dkbyond
                           
                          Started by Newtrader101, Today, 03:09 PM
                          2 responses
                          5 views
                          0 likes
                          Last Post Newtrader101  
                          Working...
                          X