Please implement autosave !!!

[xTrader1]

I'm collecting a real time tick data from MT4 using external connection feature, incl. recording market replay. Sometimes I have a power failure, mostly because of a weather. There is no option in NT to autosave a data every period of time e.g. every 10 minutes. As a result of a power failure today, several hours up to several days of my data is lost, and there is no possibility to recreate it, especially not for market replay.
Repare DB doesn't help.
I cannot afford a data center quality UPS, and a small office UPS supplying power for for 10-30 minutes doesn't solve a problem, when the data collection is performed unattended.
Please implement an autosave option ASAP.

[NinjaTrader_Jason]

Hello xTrader1,

Data you record for the market replay is saved automatically. It is stored at the location below.

Documents-->NinjaTrader 7-->db-->data folder

The data folder will contain folders named per date which in turn contain files per instrument. I do not see why recorded data is deleted after a power failure. How did you determine that market replay data was lost?

You can download market replay data at Control Center-->File-->Utilities-->Download Replay Data.

[sledge]

I'm collecting a real time tick data from MT4 using external connection feature, incl. recording market replay. Sometimes I have a power failure, mostly because of a weather. There is no option in NT to autosave a data every period of time e.g. every 10 minutes. As a result of a power failure today, several hours up to several days of my data is lost, and there is no possibility to recreate it, especially not for market replay.
Repare DB doesn't help.
I cannot afford a data center quality UPS, and a small office UPS supplying power for for 10-30 minutes doesn't solve a problem, when the data collection is performed unattended.
Please implement an autosave option ASAP.

Get a laptop? That's 4+ hours there.

you should be able to get 3 hours on cable modem UPS and close to 3 hours on 4 port wifi router on separate UPS assuming quality 900 Watts UPS assuming

[xTrader1]

Things are more complicated. Please see attached screen captures. There are (a lot more) messages informing about a corrupted historical data for instruments. Degree of a damage is different for instruments. While most instruments as to reload historical data from 12/12 till 13/12 , see attached image for $GBPAUD it's asking to reload historical data from 5/12 till 13/12. On another screen capture I show, that a tick data files for 13/12 exists and time stamp of files is 13/12 , 13:15 , very recent to a power failure.
Yet another screen capture shows historical data manager window – shows that a tick data exists for 13/12 13:00. but an attempt to access a data shows, that the program is looking for the next hour tick data file, which exists, but cannot be read.
All above happened with ver. 7.0.0.11
I installed 7.0.0.16 over the old version , but the same occurs.
I had the same problem with previous power failures, but never researched it, so I have no screen shots before an attempt to repair the database, but “historical data load failure” occured exactly at the same way, without repairing the database.
I didn't run a market replay, but .ntm files are available either. So probably there is a bug, not a lack of data.

[xTrader1]

Market replay update:
Market Replay $GBPAUD seems to work OK.

[NinjaTrader_Jason]

I see, thank you for clarifying. After the power failure, historical data became corrupted. Please note that these errors refer to historical data and not market replay data.

Did you reload the historical data for the instruments and dates listed in the errors? If so and the issue remains, please manually delete the corrupted historical data in the Historical Data Manager-->Edit-tab. Subsequently load the data by connecting to your data feed provider.

[xTrader1]

Please notice again, that I'm acquiring my data from Oanda MT4 using external data feed. Therefore I don't want to reload a data from your server. Corruption of historical data seems to be a bug, because, as you can see from attached images, historical files exist. So it seems to me that the bug should be researched and corrected.

[NinjaTrader_Jason]

Unfortunately you can experience corrupt historical data during power failures.

The files exist, but the data is corrupted. The corrupted data needs to be reloaded or deleted.

[xTrader1]

So, if I understand correctly, historical data files corruption during power failure is a "feature" of NT7 and nothing cannot be done?

[coolmoss]

Corruption of data in ANY charting platform is going to be a HUGE issue if the plug is constantly being pulled on your pc. In trading you are competing against the smartest minds in the world and if you can't even have your computer consistently getting power, that's not a good way to start. I know I'm being harsh, but it's not meant as an attack. It's meant as a wake up call. This is hardly a Ninja issue.

[NinjaTrader_Jason]

Unfortunately historical data can become corrupted during a power failure. I would not consider this a feature of the software, but a consequence of a power failure.

Are you able to export historical data from MT4? You can manually import historical data in NinjaTrader as per the instructions at the link below.

http://www.ninjatrader.com/support/helpGuides/nt7/importing.htm

If I understand correctly, you store data in NinjaTrader with realtime data received from MT4 through the External Data Feed connection. If you can export data from the source (MT4), you could import it in NinjaTrader. However the data needs to be in a certain format.

[xTrader1]

You're completely right regarding a live trading equipment, but not a development platform. For development I can live with the corruption, as I have enough uncorrupted data.
Just for now I wanted to know, whether a data corruption phenomenon is known to NT folks.

[xTrader1]

Just FYI, the problem can be solved by deleting “troublemaking” files. It means the last hour tick files for all instruments and the last day minute file for all instruments should be deleted.
And if I really want to retrieve the last day lost data from noncorrupted market replay data, I can export every tick to NT format data file and import it to history.

It is not possible to import tick data from MT4.

So we settled.

[koganam]

So, if I understand correctly, historical data files corruption during power failure is a "feature" of NT7 and nothing cannot be done?

If your computer is writing data to the hard drive and there is a power failure, data corruption is the norm, not the exception.

Heck, even if you manually remove a drive (external) while data is being written, the data is liable to be corrupted.

That is why all modern operating systems practically try to force you to manually dismount a drive before you can remove it or cut power to it.

This is a computer feature. Not a NinjaTrader bug. Not a bug, PERIOD.

[xTrader1]

Koganam, would you read my post carefully, you would see, than I didin't ask “why there is a data corruption when power failure occurs”, but looked for a better recovery procedure than reloading historical data from NinjaTrader server. And I really don't understand why should I reload historical data for 8 days, when Ninja collects only tick data and creates a minute data based on incoming ticks.
Believe me, I know a little about operating systems, teaching a course "Operating Systems Principles" in the past in a college and being a software engineer.
In previous post I wrote that deleting the last, corrupted files saves remaining data, but I was wrong.
Therefore I continue to complain about an inadequate recovery from such case.