• If this is your first visit, you will have to register before you can post. To view messages, please scroll below and select the forum that you would like to visits. Questions? Be sure to check out the Forum FAQ.

Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

Differences between historical and realtime tick processing

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    Differences between historical and realtime tick processing

    I wanted to visually see how different a small base bar series is from a 1 tick.last series comparing realtime to historical. I wish to rely on the 1 tick last series for use in strategy optimization and backtesting, without having to use tickreplay. Given that the vast majority of trades on the ES are single or just a few contracts I was expecting to see just a few differences.


    Code:
    protected override void OnStateChange()
    {
        if (State == State.SetDefaults)
        {
              Name = "VolBaseVersusAddSeries";
              IsSuspendedWhileInactive = true;
              IsOverlay = false;
              AddPlot(Brushes.DarkBlue, "BaseVol");
              AddPlot(Brushes.Red, "LastVol");
              AddLine(Brushes.Black, 1, "One");
         }
         else if (State == State.DataLoaded)
         {
              Calculate = Calculate.OnEachTick;
         }
         else if (State == State.Configure)
         {
              AddDataSeries(Instrument.FullName, BarsPeriodType.Tick, 1, MarketDataType.Last);
         }
    }
    
    protected override void OnBarUpdate()
    {
    
        //1 tick last series
         if ( BarsInProgress == 1 )
        {
             lastBarVolume += Volumes[1][0];
             return;
        }
        if ((CurrentBars[0] > 1) && ( BarsInProgress == 0))
        {
             //basebar and first tick
             if (IsFirstTickOfBar)
        {
             if (State != State.Historical)
             {
             PlotBrushes[0][0] = Brushes.Green;
             PlotBrushes[1][0] = Brushes.Yellow;
        }
    
              BaseVolume[offset] = Volume[offset] * -1;
              LastVolume[offset] = lastBarVolume;
              lastBarVolume = 0.0;
       }
    }
    }
    As you can see from the image below in realtime (green and yellow bars) there is a minor difference between close bar volumes, given the last trade might have pushed the volume over bar's size (in this case 60). However in historical (blue and red) there is enormous differences in volume size. What is causing the historical 1 tick series to behave so differently between realtime (or replay) and historical? Surely when flushing data to disc NT is not altering feed timestamps and adding trades together is it? Is there any way to configure historical ticks collection within the framework such that small series data can be used consistently whether in history or realtime.


    Click image for larger version  Name:	ES.PNG Views:	1 Size:	883.2 KB ID:	1039208
    Last edited by lavalampmj; 11-08-2018, 07:32 PM.

    #2
    Hello lavalampmj,

    In general, doing this type of comparison will generate discrepancies due to both the data's being used and the type of processing that occurs in the platform.

    The approach you are taking without using tick replay and adding a 1 tick series is one way to accomplish this but you will see discrepancies as you have pointed out. In your logic, you are using IsFirstTickOfBar will report true for historical bars and true/false for realtime when using OnEachTick which is one part of the problem you have displayed. The historical logic will perform differently than the realtime logic in your sample.

    You can also run into discrepancies based on the timestamp of the 1 tick data being shared, you can read more about this concept in the help guide here: https://ninjatrader.com/support/help...meFrameObjects

    I have attached an example of one concept of accumulating based on a 1 tick series. This calculates primary series bar volume from 1 tick series and an EMA(14) from that calculated bar volume. Please note this is not a total solution as there can still be discrepancies, this is just a bit more normalized compared to your image.


    I look forward to being of further assistance.
    Attached Files
    JesseNinjaTrader Customer Service

    Comment


      #3
      Thanks for the code. It was was quite enlightening, however it seems to generate very similar results vis a vis historical (shown below )and in realtime too. As I'm new to the platform I was just trying to grasp its limitations and the core processing trade-offs made by its designers. My conclusion, basically if I use this sort of fine grained series as input for strategy logic, then I'll need to avoid using historical backtesting and the optimizer and only use tickreplay. Speaking of performance tradeoffs and design decisions, with more memory, more and faster cpu cores, some decisions that made sense a decade ago make little sense today. For me it would be a huge enhancement if "tick replay" and "replay 10" could be used as a basis for the strategy optimizer/backtest.

      Thanks again for your help.


      Click image for larger version

Name:	capture2.PNG
Views:	1
Size:	697.0 KB
ID:	1039353

      Comment

      Latest Posts

      Collapse

      Topics Statistics Last Post
      Started by Erwin Beckers, Today, 01:23 AM
      0 responses
      2 views
      0 likes
      Last Post Erwin Beckers  
      Started by Mark Taylor, Yesterday, 11:36 PM
      0 responses
      3 views
      0 likes
      Last Post Mark Taylor  
      Started by Bluebeep, Yesterday, 08:38 PM
      0 responses
      7 views
      0 likes
      Last Post Bluebeep  
      Started by stokhastic, Yesterday, 07:08 PM
      0 responses
      3 views
      0 likes
      Last Post stokhastic  
      Started by jlkramer16, Yesterday, 04:29 PM
      1 response
      4 views
      0 likes
      Last Post jlkramer16  
      Working...
      X