Time gaps on simple charts

[Ricam]

HDS 6 possible problem

As of 7 PM PST January 28, HDS6 does not seem to be sending historical data after connecting to AMP CQG data feed. Trace files attached for your viewing pleasure, I haven't looked at them yet.

Thanks, as usual !

[reach4thelasers]

IS Ninjatrader still having issues with missing data then?

[NinjaTrader_Matthew]

IS Ninjatrader still having issues with missing data then?

As it is still beta, the NinjaTrader historical data servers are prone to outages and gaps per the expectation notes in the welcome to beta page of the help guide; however a dedicated provider like Kinetick, or IQfeed, etc should be stable in my experience.

[Ricam]

As of this morning 735 AM PST HDS5 is gapping again.

About an hour or so later was OK, I also sent an email to platform support. Thx.

[reach4thelasers]

An awesome feature would be the ability to switch off local chart-caching - because lets face it.... After 6 months this feature is still buggy as hell! I've been home from work for a hour fiddling with ninja-trader and have seen gaps in my chart like the attached screenshot about 5 times. It solves itself when i "reload historical data"

I've got 1Gbps fibre piped into my apartment and a very expensive IQFeed institutional data subscription. I don't care about local cached data - what I care about is accurate data.

[reach4thelasers]

Pretty much every chart I had open a day ago, Is now wrong when I open Ninjatrader and requires a "reload".

[reach4thelasers]

The gap in each and every circumstance starts at midnight on one day and ends at 10pm the next day. So the root cause of this issue is obvious....

As discussed in this post:

http://ninjatrader.com/support/forum/showthread.php?t=79443

Ninjatrader's belief that the trading day starts at midnight EST and ends at Midnight EST is wrong. Not to mention a bit of a narrow-minded 'center of the world' view to be honest!). The trading day actually starts when the Kiwis and Aussies get to work 9 or 10am on a monday morning (the beginning of a new day for planet earth). It is still 5pm the day before in EST - but people on the other side of the world have woken up and are trading already... and each new day finishes and ends there.

Therefore each trading day for 24h Markets starts at 5pm EST and ends at 5pm EST. However Ninjatrader is caching midnight-midnight locally. So when I fire up Ninjatrader and load some charts, switch timeframes etc. Its got midnight to midnight cached - basically its got a day, plus 7 hours extra of the following day. When Ninjatrader requests the rest of the data from IQFeed, or FXCM (or anyone else outside of Ninjatrader HQ) it correctly provides the rest of the days as 5pm EST - 5pm EST for the day's data set. I'm in GMT - 5 hours ahead. (10pm is the start of a new trading day here)

Hence I see a gap that starts at Midnight (end of the local Ninjatrader data) and ends at 10pm (the correct beginning of the next day supplied by IQFeed)

BOOM!

[NinjaTrader_DaveI]

Thank you, that could certainly be a clue. I'm curious -- Is anyone else seeing a consistent timeframe in their gaps as lasers is?

Regarding the timeframe of historical data downloads, you should actually see data downloaded from midnight to 11:59pm in your local time zone, not necessarily in EST (unless that is your PC time zone). I can see your point about market times, as I come from a forex background myself, but keep in mind that NinjaTrader is not exclusively a forex-trading platform. The history of the platform is much more in the futures markets, which revolve around national exchanges, such as the CME, which do make their nation the center of the universe, so to speak. As an ES trader, for example, the week begins on Monday around 8:30am in Chicago, where the CME resides. The same holds true for the NYSE, which considers New York the center of the universe, etc. Forex is the only market I'm aware of that is truly global in its timing.

Because NinjaTrader is used to trade a wide range of assets in a wide range of (mostly national) exchanges, we have to load data in a way that is not geared toward one asset class or another, which is why we define a day in terms of your local clock, midnight to 11:59 in your time zone.

[reach4thelasers]

Nope CME, CBOT, COMEX, NYMEX are all exactly the same... aside from a one-hour gap.

Sunday-Friday 18:00 - 17:00 New York time/ET (17:00 - 16:00 Chicago Time/CT) with a 1-hour break each day beginning at 17:00 (16:00 CT)

http://www.cmegroup.com/trading-hours.html#

https://www.google.co.uk/webhp#q=cme%20trading%20hours

The sun rises first in a new day in new zealand and that's where all 24 hour markets start their day.

Equities markets: London Stock Exchange, Nasdaq, Shenzen, NYSE tend to be local time... This is because to the Market Capitalisation of individual companies is far lower than the total value of all the worlds pounds, dollars, grain. Individual companies could have their market capital wiped away by a massive market order in the low liquidity after hours - its too risky.

[reach4thelasers]

Here's a powershell script I wrote to fix the gaps in the chart:

Open a powershell and paste the following code into the console. Leave the powershell console open and minimized in the background. No more gaps

UPDATE: spoke to soon. Still getting Gaps.

Ninjatrader: Please ask your test team to set their time zone in Ninjatrader and in windows to GMT and disable NT historical data servers and use Ninjatrader for a few days. Load data on several instruments. Close down then go home for the day. Open the same chart the following day - you should see gaps. The previous day will end at 12:00 midnight and the next day will start at 10:00pm.


NB - ignore this powershell script it doesn't help

$ntDataFolder = "$env:USERPROFILE\Documents\NinjaTrader 8\db\"
$folderMatches = "(cache|day|minute|tick)"

$directories = dir -Directory $ntDataFolder | Where-Object {$_.Name -match $folderMatches}

$result = @($directories | % {

$dir = $_.FullName;
Get-ChildItem -Path $dir -Include * -File -Recurse | foreach { $_.Delete()}
$fsw = New-Object IO.FileSystemWatcher $dir, '*' -Property @{
IncludeSubdirectories = $true
NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite'
};
$oc = Register-ObjectEvent $fsw Created -Action {
$path = $Event.SourceEventArgs.FullPath;
$name = $Event.SourceEventArgs.Name;
$changeType = $Event.SourceEventArgs.ChangeType;
$timeStamp = $Event.TimeGenerated;
Write-Host "The file '$name' was $changeType and flushed at $timeStamp";
Remove-Item $path;
};
new-object PSObject -Property @{ Watcher = $fsw; OnCreated = $oc };
});