Importing historical data

[nicbizz]

Hi,

If I import 2 files of the same name, will it remove and replace the data of the original file with the new file, or will it simply append the data of the new file into the database?

For example, I downloaded YM 06-12 from TickWrite in March last year, and the file was about 20MB (due to it being only March, and the June contract was very sparsely traded at that time). I imported this into NinjaTrader.

When I download YM 06-12 last week, the file was 144MB. If I were to import the new file, what happens to the data I previously imported? Does NT completely trash the earlier data, and does a fresh import with the new file? Or will old data remain, and I end up with duplicate data? Or is NT smart enough to "fill in the blanks" with the new data, while leaving old/duplicate lines untouched?

Thanks.

-Nick

[NinjaTrader_ChelseaB]

Hello Nick,

Thanks for your note.

Yes, if you import historical data, the imported data will overwrite any data in NinjaTrader from the same time period.


Please let me know if this does not resolve your inquiry.

[nicbizz]

Hi Chelsea,

When you say "overwrite .... from the same time period", do you mean *all* lines of data from the earlier YM 06-12 file is removed, and then only is the new YM 06-12 imported,

-OR-

only duplicate lines within the files are overwritten, where as unique entries from the older files are maintained?

Perhaps this is best illustrated with a simple example:

Old YM 06-12 contains the following lines of data.

20091222 093149;615.00;1
20091222 093152;615.00;1
20091222 093153;615.00;1 *
20091222 093155;615.00;1 *

New YM 06-12 contains following lines of data:

20091222 093149;615.00;1
20091222 093152;615.00;1
20091222 093156;615.00;1 **
20091222 093231;615.00;1 **
20091222 093232;615.00;1 **
20091222 093232;615.00;1 **

The lines with a single asterisk (" * ") exists in the old file but not the new.
The lines with the double asterisk (" ** ") exists in the new file but not the old.
The first 2 lines of both files are the same.

After I import the new YM 06-12, which lines of data exists in my database?

-Nick

[NinjaTrader_ChelseaB]

Hello Nick,

Thanks for your reply.

Using your data as an example, after the import only the new data will be in your historical database.

20091222 093149;615.00;1
20091222 093152;615.00;1
20091222 093156;615.00;1 **
20091222 093231;615.00;1 **
20091222 093232;615.00;1 **
20091222 093232;615.00;1 **


This is because the data is over the same time period. The import will not leave the old data in the database even if there is no replacement bar.


Please let me know if this does not resolve your inquiry.