Go Back   NinjaTrader Support Forum > NinjaTrader 8 > Platform Technical Support

Platform Technical Support General platform technical support for NinjaTrader 8.

NinjaTrader
Reply
 
Thread Tools Display Modes
Old 12-02-2016, 05:21 PM   #1
Ricam
Senior Member
 
Join Date: Jul 2009
Posts: 365
Thanks: 117
Thanked 52 times in 38 posts
Default Are Tick Replay and Bid and Ask historical files really necessary?

According to the Ninjatrader website, Historical Bid/Ask prices are now stored with each last trade tick data.
  1. If this is the case, why can't the bid and ask prices associated with the trade price of a secondary one tick price series be accessed directly in OnBarUpdate, without having to use Tick Replay?
  2. The prices from the bid, ask and last historical data files cannot be relied upon to be synchronized because of overlapping time stamps. Which raises the question of what purpose the bid and ask files serve, since I can't think of any way to use them for anything. Why not store the bid, ask and last in ONE file, with all records in the correct chronological order as received from the data feed?
Last edited by Ricam; 12-02-2016 at 05:25 PM.
Ricam is offline  
Reply With Quote
Old 12-04-2016, 03:05 PM   #2
NinjaTrader_ChelseaB
NinjaTrader Customer Service
 
NinjaTrader_ChelseaB's Avatar
 
Join Date: Dec 2012
Location: Denver, CO
Posts: 8,279
Thanks: 125
Thanked 1,836 times in 1,719 posts
Default

Hello Ricam,

You can access the tick ask or tick bid data without using Tick Replay by adding these as secondary series to your script using AddDataSeries.

Enabling tick replay does this for you for convenience, but uses the same data that would be used if you add the series to your script yourself and uses this for providing snapshot tick data to OnMarketUpdate which typically only runs in real-time.

This new addition is meant as an easier alternative to coding your script to use intra-bar granularity.

Can you provide historical data that demonstrates the downloaded historical data is out of sync?
NinjaTrader_ChelseaB is offline  
Reply With Quote
Old 12-04-2016, 03:50 PM   #3
Ricam
Senior Member
 
Join Date: Jul 2009
Posts: 365
Thanks: 117
Thanked 52 times in 38 posts
Default

Quote:
Originally Posted by NinjaTrader_ChelseaB View Post
Hello Ricam,

You can access the tick ask or tick bid data without using Tick Replay by adding these as secondary series to your script using AddDataSeries.

Enabling tick replay does this for you for convenience, but uses the same data that would be used if you add the series to your script yourself and uses this for providing snapshot tick data to OnMarketUpdate which typically only runs in real-time.

This new addition is meant as an easier alternative to coding your script to use intra-bar granularity.

Can you provide historical data that demonstrates the downloaded historical data is out of sync?
There seems to be a lack of internal communication over there.

The method you suggest is in conflict with Ninjatrader's own documentation and sample code.Tick Replay is not some kind of alternative shortcut "for convenience" It is the ONLY way to get the best bid and ask associated with a trade. These prices are hidden away someplace in the Last file where they can't be accessed other than in OnMarketData in Tick Replay.

You can't get the correct bid and ask associated with a trade by sequentially reading the bid, ask and last of a one tick data sereis.. The Ninjatrader Help article, buy sell pressure and volume indicators, and sample code for getting bid ask do not say to use the method you mention. I can't find it now, but at one time there was a specific warning to NOT use this method..I did try using this method and can assure you that it gives worthless, mostly random results.

The reason this can't work is that there are often multiple records in each of the files bid, ask and last that have the exact same time stamp. (Presumably this is because of the way the exchange matching engine works, generating the same time stamp for multiple events that can happen each time there is a match..)

Therefore in the historical state there is no possible way for the various BIP events to fire in the correct historical sequence since the sequence information cannot be reconstructed from three separate files that have overlapping multiple records with the same time stamps. This is the whole point of why according to the former Ninjatrader documentation that i can't find anymore, Tick Replay is currently MANDATORY for historical bid ask data. Maybe the information is in the Release Notes somewhere, I don't want to look right now.

This makes the bid and ask files completely useless. Meanwhile my queries about putting the bid, ask and last events in one file in correct chronological order, and about modifying GetCurrentBid and GetCurrentAsk to work historically on a one tick data series by getting the information from the Last file, continue to be ignored. Please let me know if I may be of further assistance.
Last edited by Ricam; 12-04-2016 at 04:35 PM.
Ricam is offline  
Reply With Quote
Old 12-04-2016, 04:34 PM   #4
NinjaTrader_ChelseaB
NinjaTrader Customer Service
 
NinjaTrader_ChelseaB's Avatar
 
Join Date: Dec 2012
Location: Denver, CO
Posts: 8,279
Thanks: 125
Thanked 1,836 times in 1,719 posts
Default

Hello Ricam,

From the help guide:
"Tick Replay was NOT designed to provide accuracy in backtesting concerning order fills and execution and should NOT be used to expect the exact sequence of executions as running a strategy on live data. For greater order-fill resolution and accuracy in strategy backtesting, you can use the High Fill Resolution in the Strategy Analyzer".

http://ninjatrader.com/support/helpG...ick_replay.htm

If you are wanting the tick replay for accurate fills, this will not do that.

I highly recommend you review this document in the help guide.

That said, adding intra-bar granularity of ask and bid as I have suggested in my previous email, will do just that.

This is how intra-bar granularity was added for accurate fills and for intra-bar actions (typically in OnMarketData but instead in OnBarUpdate) with NinjaTrader 7.

We have a reference sample that demonstrates how to do this.
http://www.ninjatrader.com/support/f...ead.php?t=6652

For further information see:
http://ninjatrader.com/support/forum...117#post463117

You can get the correct bid and ask in OnBarUpdate by adding a secondary series of 1 tick ask and a secondary series of 1 tick bid.
http://ninjatrader.com/support/helpG...dataseries.htm


Do you have any historical data / output to demonstrate the ask and bid prices are out of sync?
NinjaTrader_ChelseaB is offline  
Reply With Quote
Old 12-04-2016, 04:39 PM   #5
Ricam
Senior Member
 
Join Date: Jul 2009
Posts: 365
Thanks: 117
Thanked 52 times in 38 posts
Default

What are you talking about?
I never said anything about back testing.

Did you even read what I said? The points I raised have not been answered, and new straw men have been brought in, adding nothing other than confusion.

I think you need to refer my question to Brett,
Last edited by Ricam; 12-04-2016 at 04:48 PM.
Ricam is offline  
Reply With Quote
Old 12-04-2016, 04:46 PM   #6
NinjaTrader_ChelseaB
NinjaTrader Customer Service
 
NinjaTrader_ChelseaB's Avatar
 
Join Date: Dec 2012
Location: Denver, CO
Posts: 8,279
Thanks: 125
Thanked 1,836 times in 1,719 posts
Default

Hello Ricam,

Tick Replay is only used when backtesting (historical).
This does not apply to real-time data.

Are you referring to real-time or backtesting (historical)?
NinjaTrader_ChelseaB is offline  
Reply With Quote
Old 12-04-2016, 05:15 PM   #7
Ricam
Senior Member
 
Join Date: Jul 2009
Posts: 365
Thanks: 117
Thanked 52 times in 38 posts
Default

I am using Tick Replay for accessing bid ask historical prices to calculate the deltas.

Your statement about Tick Replay only being for back testing is just plain wrong.

Since you don't know what I am talking about, please refer this to someone who does.
Last edited by Ricam; 12-04-2016 at 05:18 PM.
Ricam is offline  
Reply With Quote
Old 12-04-2016, 05:20 PM   #8
NinjaTrader_ChelseaB
NinjaTrader Customer Service
 
NinjaTrader_ChelseaB's Avatar
 
Join Date: Dec 2012
Location: Denver, CO
Posts: 8,279
Thanks: 125
Thanked 1,836 times in 1,719 posts
Default

Hello Ricam,

This is stated in the help guide.

From the description at the top of the help guide:
"In general, this means adding additional logic to the OnMarketData() event handler, however, Tick Replay can also be used to call OnBarUpdate() "OnEachTick" or "OnPriceChange" during historical calculations."
http://ninjatrader.com/support/helpG...ick_replay.htm

Also, from the same help guide document:
"Tick replay was only ONLY designed to work with MarketDataType.Last. A TickReplay indicator or strategy should NOT be mixed with a MarketDataType.Ask or MarketDataType.Bid series."

I highly recommend you review this document in the help guide.

I will escalate this thread to the NinjaScript lead.
Last edited by NinjaTrader_ChelseaB; 12-04-2016 at 05:22 PM.
NinjaTrader_ChelseaB is offline  
Reply With Quote
Old 12-04-2016, 05:34 PM   #9
Ricam
Senior Member
 
Join Date: Jul 2009
Posts: 365
Thanks: 117
Thanked 52 times in 38 posts
Default

I already know that. You do not seem to be able to understand what I have been saying.You are in over your head, please refer this to somebody else.
Ricam is offline  
Reply With Quote
Old 12-05-2016, 04:45 PM   #10
NinjaTrader_Brett
NinjaTrader Product Management
 
NinjaTrader_Brett's Avatar
 
Join Date: Dec 2009
Location: Denver, CO
Posts: 7,258
Thanks: 149
Thanked 568 times in 482 posts
Default

To clear up some confusion:

* Tick replay replays 'last' events (trades) and along with the 'last' event throws the current state of bid/ask at the time of 'last' which is recorded and saved with the last event.
* Yes you are correct in that bid/ask data files now no longer need to be used for such traditional type of analysis.
* They still are available since for several reasons:

1. Forex only has bid/ask and does not have last therefore you still need traditional bid/ask there.
2. There are some analysis that can be done based on rate of changes of ask/bid. Tick replay you would not be able to do that since we only store the current state of bid/ask with last at the time of a last event. You would mis all bid/ask changes made from the provider in between. Bid/Ask would retain that directly.

This is the technical difference between them.

-Brett
NinjaTrader_Brett is offline  
Reply With Quote
Old 12-07-2016, 04:06 PM   #11
Ricam
Senior Member
 
Join Date: Jul 2009
Posts: 365
Thanks: 117
Thanked 52 times in 38 posts
Default Restating previous questions

There were a lot of posts in this thread since the first one, but my original questions remained unanswered.


Quote:
Originally Posted by Ricam View Post
According to the Ninjatrader website, Historical Bid/Ask prices are now stored with each last trade tick data.
  1. If this is the case, why can't the bid and ask prices associated with the trade price of a secondary one tick price series be accessed directly in OnBarUpdate, without having to use Tick Replay?
  2. The prices from the bid, ask and last historical data files cannot be relied upon to be synchronized because of overlapping time stamps. Which raises the question of what purpose the bid and ask files serve, since I can't think of any way to use them for anything. Why not store the bid, ask and last in ONE file, with all records in the correct chronological order as received from the data feed?
Ricam is offline  
Reply With Quote
Old 12-08-2016, 02:54 PM   #12
NinjaTrader_Brett
NinjaTrader Product Management
 
NinjaTrader_Brett's Avatar
 
Join Date: Dec 2009
Location: Denver, CO
Posts: 7,258
Thanks: 149
Thanked 568 times in 482 posts
Default

Sorry about that, see below my answers inline to your question below.

Quote:
If this is the case, why can't the bid and ask prices associated with the trade price of a secondary one tick price series be accessed directly in OnBarUpdate, without having to use Tick Replay?
[Brett]: Since that assumes you will be required to run the indicator Calculate=On Each Tick to access the Tick Replay data. Having it playback in OnMarketData which is how you would access the data in realtime without using TickReplay is consistent and doesn't come with the penalty of forcing indicator calculations on each tick.


Quote:
The prices from the bid, ask and last historical data files cannot be relied upon to be synchronized because of overlapping time stamps. Which raises the question of what purpose the bid and ask files serve, since I can't think of any way to use them for anything. Why not store the bid, ask and last in ONE file, with all records in the correct chronological order as received from the data feed?
[Brett] : If you want to do analysis on all Bid updates, use MarketDataType.Bid. If you want to do analysis on trade(tick) data but want to get the current state of bid/ask at the time of the trade (tick) then use tick replay.
NinjaTrader_Brett is offline  
Reply With Quote
Old 01-12-2018, 07:13 PM   #13
Ricam
Senior Member
 
Join Date: Jul 2009
Posts: 365
Thanks: 117
Thanked 52 times in 38 posts
Default still no answer to item 1

Tick Replay is a performance disaster during chart loading so I will ask once again:


Originally Posted by Ricam View Post
According to the Ninjatrader website, Historical Bid/Ask prices are now stored with each last trade tick data.
If this is the case, why can't the bid and ask prices associated with the trade price of a secondary one tick price series be accessed directly in OnBarUpdate, without having to use Tick Replay?
The prices from the bid, ask and last historical data files cannot be relied upon to be synchronized because of overlapping time stamps. Which raises the question of what purpose the bid and ask files serve, since I can't think of any way to use them for anything. Why not store the bid, ask and last in ONE file, with all records in the correct chronological order as received from the data feed?
Ricam is offline  
Reply With Quote
Old 01-13-2018, 12:20 AM   #14
timmyb
Senior Member
 
Join Date: Mar 2008
Posts: 221
Thanks: 2
Thanked 0 times in 0 posts
Default

I also am curious why I pay 80+ per month for a quality data feed that Ninja is unable to use the data for correctly. Especially with the new rollout of your ladder version etc. I would personally liketo see and ability to line up the trades in correct chronological order received from the data feed.
timmyb is offline  
Reply With Quote
Old 01-16-2018, 12:24 PM   #15
NinjaTrader_Brett
NinjaTrader Product Management
 
NinjaTrader_Brett's Avatar
 
Join Date: Dec 2009
Location: Denver, CO
Posts: 7,258
Thanks: 149
Thanked 568 times in 482 posts
Default

NinjaTrader 8 does have support for bid/ask stamped last events, so that you could use 1 tick data to do a bid/ask comparison to see if that tick occurred at/above the ask or at/below the bid.

This is part of the data Tick Replay uses to throw OnMarketData events when the tick replay simulation is running to rebuild the bars (and subsequently any applied NinjaScript indicators/strategies). You can use Tick Replay to get access to the bid/ask stamped data inside of OnMarketData, however it is not Tick Replays primary use case. Where Tick Replays primary use case is to simulate building bars as they would have been built in real-time, and hence have a performance penalty.

If all you need is the bid/asked stamped data on a 1 tick to do some analysis if that tick was a buy/sell, then using Tick Replay could be considered overkill. Executing the below code on a 1 tick series will allow you to get the bid/ask stamped data that is attached to a 1 tick trade without requiring the use of Tick Replay.

double askPrice = BarsArray[1].GetAsk(CurrentBars[1]);
double bidPrice = BarsArray[1].GetBid(CurrentBars[1]);

Note: There are some implication of this approach, since now you need to deal with building a multi-time frame indicator which is not as simple as a regular single series indicator. However, it performs much better then requiring Tick Replay if bid/ask stamped last data is all you require. Its for sure a trade off between selecting which method to use, there is no 'best' approach.

Note: The above code sample will only work on an Indicator/Strategies with an added 1 tick series.
Last edited by NinjaTrader_Brett; 01-16-2018 at 12:44 PM.
NinjaTrader_Brett is offline  
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Historical tick side (ask/bid) alex_bbfg Indicator Development 5 12-02-2015 03:06 PM
Historical 1-tick Last, Bid, Ask cls71 Version 8 Beta 9 10-01-2015 04:48 PM
Anyone have market replay files and/or tick data for CL since 1/1/2007? richwaters General Development 0 02-28-2011 10:28 AM
Creating and accessing historical tick data files kaydgee Strategy Development 1 09-05-2010 10:30 AM
Where Can I find historical replay files? maxreturn Platform Technical Support 0 02-10-2009 01:50 PM


All times are GMT -6. The time now is 06:19 PM.

Futures, foreign currency and options trading contains substantial risk and is not for every investor. An investor could potentially lose all or more than the initial investment. Risk capital is money that can be lost without jeopardizing ones financial security or lifestyle. Only risk capital should be used for trading and only those with sufficient risk capital should consider trading. Past performance is not necessarily indicative of future results. View Full Risk Disclosure.

CFTC Rules 4.41 - Hypothetical or Simulated performance results have certain limitations, unlike an actual performance record, simulated results do not represent actual trading. Also, since the trades have not been executed, the results may have under-or-over compensated for the impact, if any, of certain market factors, such as lack of liquidity. Simulated trading programs in general are also subject to the fact that they are designed with the benefit of hindsight. No representation is being made that any account will or is likely to achieve profit or losses similar to those shown.

This website is hosted and operated by NinjaTrader, LLC (“NT”), a software development company which owns and supports all proprietary technology relating to and including the NinjaTrader trading platform. NT is an affiliated company to NinjaTrader Brokerage (“NTB”), which is a NFA registered introducing broker (NFA #0339976) providing brokerage services to traders of futures and foreign exchange products. This website is intended for educational and informational purposes only and should not be viewed as a solicitation or recommendation of any product, service or trading strategy. No offer or solicitation to buy or sell securities, securities derivative or futures products of any kind, or any type of trading or investment advice, recommendation or strategy, is made, given, or in any manner endorsed by any NT affiliate and the information made available on this Web site is not an offer or solicitation of any kind. Specific questions related to a brokerage account should be sent to your broker directly. The content and opinions expressed on this website are those of the authors and do not necessarily reflect the official policy or position of NT or any of its affiliates. 

Vendors along with their websites, products and services, collectively referred to as (“Vendor Content”), are independent persons or companies that are in no manner affiliated with NT or any if its affiliates. NT or any of its affiliates are not responsible for, do not approve, recommend or endorse any Vendor Content referenced on this website and it’s your sole responsibility to evaluate Vendor Content. Please be aware that any performance information provided by a vendor should be considered hypothetical and must contain the disclosures required by NFA Rule 2-29(c). If you are interested in learning more about, or investigating the quality of, any such Vendor Content you must contact the vendor, provider or seller of such Vendor Content. No person employed by, or associated with, NT or any of its affiliates is authorized to provide any information about any such Vendor Content.