• If this is your first visit, you will have to register before you can post. To view messages, please scroll below and select the forum that you would like to visits. Questions? Be sure to check out the Forum FAQ.

Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

How do I scan as many stocks as possible?

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    How do I scan as many stocks as possible?

    Hello. I'm trying to get my script to work with as many stocks as possible. I'm using the same TD Ameritrade account as on Thinkorswim (fully funded for realtime data), where they limit scans to a max of 2000 stocks. I want to scan over 7000, and I can display them all in the Market Analyzer, but most of the rows never update. I realize many stocks get little to no volume on any given day, but I'm seeing no activity from the likes of TSLA, which is obviously not right.

    This has me wondering if Thinkorswim's 2000 limit is mandated by TD Ameritrade across all platforms. Can anyone confirm limits on the number of simultaneous requests through TD Ameritrade? Are there any such limits on Kinetick, if I subscribe? Is there anything else I can do to get reliable data from more stocks?

    I'll include my script for reference. I'm trying to catch the huge, spontaneous dips on no news - basically what Timothy Sykes prefers doing now, but without all the research. The script itself seems to work well, though I need to refine my constraints as I collect more data. I only need the one column, so I remove all others to reduce the load. I record a small buffer myself, so there are no historic data requests in this script.

    Code:
    namespace NinjaTrader.NinjaScript.MarketAnalyzerColumns
    {
        public class DipScanner : MarketAnalyzerColumn
        {
            private const int LOOKBACK = 12; // how many minutes of data to record
            private const float THRESHOLD = -10; // in percentage points
            private const bool PRINT = false;
    
            private int lastAlert; // last time the alert was triggered
            private int thresholdMultiple; // this script alerts again before the cooldown expires if percent change reaches a new multiple of THRESHOLD (-10%, -20%, -30%, etc.)
            // @TODO: pick a threshold for volume
            // @TODO: volume appears to be misleading during pre-market, showing yesterday's total
            private long dailyVolume;
            private bool cleanup;
    
            public List<int> times; // in minutes
            public List<double> prices;
    
            protected override void OnStateChange()
            {
                if (State == State.SetDefaults) {
                    IsDataSeriesRequired = false;
                }
                else if (State == State.DataLoaded) {                
                    lastAlert = -10000;
                    cleanup = true;
                    times = new List<int>();
                    prices = new List<double>();                                
                    CurrentValue = 0;
                }
                else if (State == State.Terminated) {
                    if (cleanup) {
                        times = null;
                        prices = null;
                    }
                }
            }
    
            protected override void OnMarketData(MarketDataEventArgs data)
            {
                if (data.MarketDataType == MarketDataType.DailyVolume) {
                    dailyVolume = data.Volume; // this should be zero on the first Evaluate since Last seems to come before DailyVolume
                }
                else if (data.MarketDataType == MarketDataType.Last) {
                    int minute = Environment.TickCount / 60000; // TickCount is in milliseconds. up to 49.8 days after system startup
                    Evaluate(minute, data.Price);
                }
            }
    
            private void Evaluate(int minute, double price)
            {
                // take only one snapshot per minute
                if (times.Count == 0 || times[times.Count-1] != minute) {
                    times.Add(minute);
                    prices.Add(price);
    
                    // start at the end of the array and move forward. on the first minute at or above LOOKBACK, remove everything before it. protects against skips in market data.
                    // if successful, calculate the percent change and alert if it falls below the threshold
                    for (int i = times.Count-2; i >= 0; i--) {
                        if (minute - times[i] >= LOOKBACK) {
                            times.RemoveRange(0, i);
                            prices.RemoveRange(0, i);
    
                            float percent = (float)CalculatePercent();                
                            // reset threshold if the alert cooldown has expired
                            if (minute - lastAlert >= LOOKBACK)
                                thresholdMultiple = 1;
    
                            if (percent <= THRESHOLD * thresholdMultiple) {
                                string emailMessage = "Volume: " + dailyVolume + " ... " + times[0] + "/" + prices[0] + " ... " + minute + "/" + price + " ... Threshold Multiple: " + thresholdMultiple;
                                SendMail("[email protected]", "[" + Instrument.FullName + "] ... " + percent + "% at " + DateTime.Now.ToString("hh:mm tt"), emailMessage);
                                string logMessage = "[" + Instrument.FullName + "] " + percent + "% at " + DateTime.Now.ToString("hh:mm tt") + " ... " + times[0] + "/" + prices[0] + " ... " + minute + "/" + price + " ... Volume: " + dailyVolume + " ... Threshold Multiple: " + thresholdMultiple;
                                Log(logMessage, LogLevel.Information);
                                Print(">>>> " + logMessage);
    
                                lastAlert = minute;
                                thresholdMultiple = (int)(percent / THRESHOLD) + 1;
                            }
    
                            CurrentValue = percent;                
                            break;
                        }
                    }
                }
            }
    
            private double CalculatePercent()
            {
                return (prices[prices.Count-1] - prices[0]) / prices[0] * 100;
            }
        }
    }

    #2
    Hello Phoenix2518,

    Thank you for the post.

    I took a look to see if we have any notes on a specific symbol limit with TDA but was unable to locate anything noted about this for this provider. You would likely need to contact TDA directly to confirm if they have a specific symbol limit with the account/data you are using.

    Regarding the symbols which are not receiving data in the market analyzer, yes this could be due to volume but this could also be if the limit was reached and the instrument has not been subscribed to. You may be able to confirm this by using the control centers Log tab to see if any errors are showing up when you load the MA with the 7000 instruments.

    Kinetick and most other providers will also have limits, this is a question which you generally will need to ask each of the data providers you are interested in. This is also a case where you may need to create a filtered list of instruments to trade to reduce the total number of requests.


    I look forward to being of further assistance.
    JesseNinjaTrader Customer Service

    Comment


      #3
      I forgot to mention, I was getting numerous errors about invalid security symbols. I had dismissed them since there weren't enough to account for the inactive stocks, but they numbered exactly 100, which was suspicious. I removed these entries, and it turns out all the errors were suppressing a few more, but only 105 symbols in total were invalid. I haven't seen any other errors.

      Unfortunately, Kinetick's highest plan maxes out at 500 concurrent symbols. I'll see what TD Ameritrade says about their limits, but I expect I'll have to filter my list through Finviz. Okay, thanks. That's probably as far as I can take this, at least until I can afford $1K a month to subscribe to IEX + Firehose.

      Comment

      Latest Posts

      Collapse

      Topics Statistics Last Post
      Started by maxgn, Today, 03:34 PM
      0 responses
      4 views
      0 likes
      Last Post maxgn
      by maxgn
       
      Started by Boonfly8, Today, 02:47 PM
      1 response
      5 views
      0 likes
      Last Post NinjaTrader_Jesse  
      Started by marioresa, Today, 02:20 PM
      1 response
      2 views
      0 likes
      Last Post NinjaTrader_Kate  
      Started by Parmenides48, Today, 01:26 PM
      1 response
      20 views
      0 likes
      Last Post NinjaTrader_JoshG  
      Started by pjsmith, Today, 12:39 PM
      1 response
      5 views
      0 likes
      Last Post NinjaTrader_PatrickH  
      Working...
      X