• If this is your first visit, you will have to register before you can post. To view messages, please scroll below and select the forum that you would like to visits. Questions? Be sure to check out the Forum FAQ.

Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

NScript to Import Historical Data

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    NScript to Import Historical Data

    Howdy--

    Is it possible to import historical data via NScript as opposed to the NT GUI (i.e., Tools > Historical Data Manager... > Import)?

    I am going to write a StreamReader and then StreamWriter indicator to pull daily futures data all the way back to the 1980s, format that data to the NT import format type and then write the properly formatted data to an appropriately named .txt file. Given my current understanding, I would then need to import each contract (i.e., GC 06-86.Last.txt) into NT via the Historical Data Manager GUI, which as you might imagine, will be rather labor intensive for several instruments going back to the 1980s (i.e., energy, metals, ag, currencies, etc). Therefore, I am interested if I can program the data importation process via NScript, which would save me a TON of screen time.

    Thanks,

    Aventeren

    #2
    Hello Aventeren,

    Thank you for your note.

    Unfortunately, there are no supported means to achieve this in code. And if it is possible it would not be supported.
    Patrick H.NinjaTrader Customer Service

    Comment


      #3
      Follow Up Question:
      The source for my daily data reports the Volume as a string that looks like "555.0" instead of "555" (i.e., it looks like a double as opposed to an int).

      When I attempt to import, will the ".0" portion of the volume cause the import to fail?

      Do I first need to remove the ".0" from the volume before importing?

      Thanks,

      Aventeren

      Comment


        #4
        Date Order

        Also, does it matter if the lines are ordered by descending dates (i.e., 12/31/13, 12/30/13, 12/29/13, etc) or does the data need to be sorted in an ascending fashion (i.e., 12/1/13, 12/2/13, 12/3/13, etc)?

        Thanks,

        Aventeren

        Comment


          #5
          Year Format of .txt file

          Howdy--

          NT asks for a month and year in the .txt file per the Help Guide (i.e., "ES 09-12"). Obviously this is for the ES 09 contract in 2012.

          I am looking to import historical data as far back as 1959 (Wheat). So for instance, I am trying to import the ZW 12 contract in 1959. Would the correct file name be called "ZW 12-59.Last.txt"?

          Thanks,

          Aventeren

          Comment


            #6
            I parsed out the ".0" from the volume and I sorted the data to be descending prior to creating the .txt file. I then created a new ZW contract called 12-59 with a rollover date of 01/14/1960 (which I will change later).

            I was able to get the "ZW 12-59.Last.txt" data to import, which I confirmed via the Historical Data Manager -> Edit opened "ZW 12-59" and confirmed the data was there and it matched the source.

            I then opened a ZW 12-14 contract chart and loaded 25,000 days (2014 - 1959 == 55 * 365 = 20,075 days --> so 25,000 days covers us). However the data still only goes back to 12-09 and the 12-59 data does not show up.

            Given that there is a 1959 to 2009 data gap in my db, is there a setting that NT is looking to that would preclude the 12-59 contract data from showing on the chart?

            Do I first need to import all of the other contracts prior to the historical data showing up?

            Thanks,

            Aventeren

            Comment


              #7
              Aventeren,

              Have you defined a contract for that period, 12-59?

              You would need to set the contract and the dates for it in the contracts defined section -
              Tools -> Instrument Manager -> ZW -> Edit -> Misc Tab and scroll to the bottom.
              Cal H.NinjaTrader Customer Service

              Comment


                #8
                Originally posted by NinjaTrader_Cal View Post
                Aventeren,

                Have you defined a contract for that period, 12-59?

                You would need to set the contract and the dates for it in the contracts defined section -
                Tools -> Instrument Manager -> ZW -> Edit -> Misc Tab and scroll to the bottom.
                Yes, I did. Rollover date was 1/14/60...but maybe it should have been 9/14/59?

                Comment


                  #9
                  Aventeren,

                  Yes, the December contract typically rolls over in September of that year.
                  Cal H.NinjaTrader Customer Service

                  Comment


                    #10
                    So I tried to load just the ZW 12-59 contract and I got an error that the "NT data server does not support this instrument, no data available".

                    To confirm, I can see the data in the Historical Data Manager's Edit tab (under daily) and I created a new ZW 12-59 contract with a Rollover Date of 9.11.1959.

                    I am connected to Continuum.

                    I then disconnected from Continuum and connected to Kinetic's Free EOD connection, and I got an error message saying that the "Unable to load data, unknown instrument (panic)".

                    Any ideas on why I can't chart this data?

                    Thanks,

                    Aventeren
                    Last edited by aventeren; 09-12-2014, 03:47 PM.

                    Comment


                      #11
                      I've done some digging around, and it appears that NT is looking to populate the chart data from the data server and NT will not look at the imported data if the data server does not have the instrument/contract. Therefore, ZW only goes back until 2005--and from my understanding I would need to delete all of the ZW db data via the Historical Data Manager -> Edit tab--and then NT will look first to the imported data and then back fill the missing data from the server.

                      I'm confused. This doesn't make sense that I would have to delete data provided by the data server to get historical data to chart.

                      Help please.

                      Comment


                        #12
                        Here are some screenshots of ZW and ZW 12-59 data, settings and error messages.

                        Click image for larger version

Name:	ZW Instrument Manager, Symbol Map.png
Views:	1
Size:	194.5 KB
ID:	872310

                        Click image for larger version

Name:	ZW 12-59 Historical Data Manager.jpg
Views:	1
Size:	259.8 KB
ID:	872311

                        Click image for larger version

Name:	ZW 12-59 Instrument Manager.png
Views:	1
Size:	259.0 KB
ID:	872312

                        Click image for larger version

Name:	Error Messages, ZW 12-59.jpg
Views:	1
Size:	74.7 KB
ID:	872313

                        Comment


                          #13
                          So by disconnecting from all data feeds, I am able to get the data to plot.

                          Click image for larger version

Name:	ZW 12-59, Disconnected.jpg
Views:	1
Size:	73.4 KB
ID:	872314

                          Is there a way to have the data plot while being connected?

                          Comment


                            #14
                            Hello aventeren,

                            Thank you for your response.

                            It is possible by going to Tools > Options > Data > uncheck the 'Get data from server' option > OK. Then connect to your data feed provider.
                            Patrick H.NinjaTrader Customer Service

                            Comment


                              #15
                              Originally posted by NinjaTrader_PatrickH View Post
                              Hello aventeren,

                              Thank you for your response.

                              It is possible by going to Tools > Options > Data > uncheck the 'Get data from server' option > OK. Then connect to your data feed provider.
                              Patrick--

                              I tried unchecking this box, restarted NT (with connection to Continuum) and then reloaded the ZW 12-14 daily contract with 30,000 days of data starting today. The ZW 12-59 daily data still does not show up on the chart.

                              Any other ideas?

                              Thanks,

                              Aventeren

                              Comment

                              Latest Posts

                              Collapse

                              Topics Statistics Last Post
                              Started by chrismcdermut, Today, 01:46 PM
                              3 responses
                              17 views
                              0 likes
                              Last Post MojoJojo  
                              Started by Bricolico, Today, 09:28 PM
                              0 responses
                              5 views
                              0 likes
                              Last Post Bricolico  
                              Started by Tradwell385, Today, 08:51 PM
                              0 responses
                              7 views
                              0 likes
                              Last Post Tradwell385  
                              Started by dvcpro4, Today, 07:08 PM
                              0 responses
                              19 views
                              0 likes
                              Last Post dvcpro4
                              by dvcpro4
                               
                              Started by pennywise, Today, 06:40 PM
                              0 responses
                              8 views
                              0 likes
                              Last Post pennywise  
                              Working...
                              X