• If this is your first visit, you will have to register before you can post. To view messages, please scroll below and select the forum that you would like to visits. Questions? Be sure to check out the Forum FAQ.

Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

Differences between historical and realtime tick processing

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Differences between historical and realtime tick processing

    I wanted to visually see how different a small base bar series is from a 1 tick.last series comparing realtime to historical. I wish to rely on the 1 tick last series for use in strategy optimization and backtesting, without having to use tickreplay. Given that the vast majority of trades on the ES are single or just a few contracts I was expecting to see just a few differences.


    Code:
    protected override void OnStateChange()
    {
        if (State == State.SetDefaults)
        {
              Name = "VolBaseVersusAddSeries";
              IsSuspendedWhileInactive = true;
              IsOverlay = false;
              AddPlot(Brushes.DarkBlue, "BaseVol");
              AddPlot(Brushes.Red, "LastVol");
              AddLine(Brushes.Black, 1, "One");
         }
         else if (State == State.DataLoaded)
         {
              Calculate = Calculate.OnEachTick;
         }
         else if (State == State.Configure)
         {
              AddDataSeries(Instrument.FullName, BarsPeriodType.Tick, 1, MarketDataType.Last);
         }
    }
    
    protected override void OnBarUpdate()
    {
    
        //1 tick last series
         if ( BarsInProgress == 1 )
        {
             lastBarVolume += Volumes[1][0];
             return;
        }
        if ((CurrentBars[0] > 1) && ( BarsInProgress == 0))
        {
             //basebar and first tick
             if (IsFirstTickOfBar)
        {
             if (State != State.Historical)
             {
             PlotBrushes[0][0] = Brushes.Green;
             PlotBrushes[1][0] = Brushes.Yellow;
        }
    
              BaseVolume[offset] = Volume[offset] * -1;
              LastVolume[offset] = lastBarVolume;
              lastBarVolume = 0.0;
       }
    }
    }
    As you can see from the image below in realtime (green and yellow bars) there is a minor difference between close bar volumes, given the last trade might have pushed the volume over bar's size (in this case 60). However in historical (blue and red) there is enormous differences in volume size. What is causing the historical 1 tick series to behave so differently between realtime (or replay) and historical? Surely when flushing data to disc NT is not altering feed timestamps and adding trades together is it? Is there any way to configure historical ticks collection within the framework such that small series data can be used consistently whether in history or realtime.


    Click image for larger version  Name:	ES.PNG Views:	1 Size:	883.2 KB ID:	1039208
    Last edited by lavalampmj; 11-08-2018, 08:32 PM.

  • lavalampmj
    replied
    Thanks for the code. It was was quite enlightening, however it seems to generate very similar results vis a vis historical (shown below )and in realtime too. As I'm new to the platform I was just trying to grasp its limitations and the core processing trade-offs made by its designers. My conclusion, basically if I use this sort of fine grained series as input for strategy logic, then I'll need to avoid using historical backtesting and the optimizer and only use tickreplay. Speaking of performance tradeoffs and design decisions, with more memory, more and faster cpu cores, some decisions that made sense a decade ago make little sense today. For me it would be a huge enhancement if "tick replay" and "replay 10" could be used as a basis for the strategy optimizer/backtest.

    Thanks again for your help.


    Click image for larger version

Name:	capture2.PNG
Views:	1
Size:	697.0 KB
ID:	1039353

    Leave a comment:


  • NinjaTrader_Jesse
    replied
    Hello lavalampmj,

    In general, doing this type of comparison will generate discrepancies due to both the data's being used and the type of processing that occurs in the platform.

    The approach you are taking without using tick replay and adding a 1 tick series is one way to accomplish this but you will see discrepancies as you have pointed out. In your logic, you are using IsFirstTickOfBar will report true for historical bars and true/false for realtime when using OnEachTick which is one part of the problem you have displayed. The historical logic will perform differently than the realtime logic in your sample.

    You can also run into discrepancies based on the timestamp of the 1 tick data being shared, you can read more about this concept in the help guide here: https://ninjatrader.com/support/help...meFrameObjects

    I have attached an example of one concept of accumulating based on a 1 tick series. This calculates primary series bar volume from 1 tick series and an EMA(14) from that calculated bar volume. Please note this is not a total solution as there can still be discrepancies, this is just a bit more normalized compared to your image.


    I look forward to being of further assistance.
    Attached Files

    Leave a comment:

Latest Posts

Collapse

Topics Statistics Last Post
Started by NinjaCustomer, Today, 01:48 AM
0 responses
1 view
0 likes
Last Post NinjaCustomer  
Started by aligator, Yesterday, 10:11 PM
0 responses
2 views
0 likes
Last Post aligator  
Started by psquare, Yesterday, 03:11 PM
4 responses
20 views
0 likes
Last Post psquare
by psquare
 
Started by ballboy11, Yesterday, 03:20 PM
1 response
12 views
0 likes
Last Post NinjaTrader_ChelseaB  
Started by tonynt, Yesterday, 02:18 PM
1 response
11 views
0 likes
Last Post NinjaTrader_JoshG  
Working...
X