Coding/Designing a generic thread-safe limiter (i.e. limit the execution of X() to Y many times per second)

Something like this?

using Timer = System.Threading.Timer;

class Limiter{
    public static readonly Limiter Instance = new Limiter();

    Limiter(){}

    int         max;
    Semaphore   counter;
    List<Timer> timers  = new List<Timer>();

    // warning: not thread safe!
    public int MaxExecutionPerSecond{
        get{return max;}
        set{counter = new Semaphore(max = value, value);}
    }

    public void WaitIfRequired(){
        // Our semaphore starts with a count of MaxExecutionPerSecond.
        // When we call WaitOne(), it decrements the count.  If the count
        // is already zero, the call to WaitOne() will block until another
        // thread calls Release() to increment the count.
        counter.WaitOne();

        // Set a timer to increment the semaphore in one second.
        Timer t = null;
        t = new Timer(o=>{
            // Increment the semaphore.
            counter.Release();

            // We no longer need to protect this timer from the GC.
            timers.Remove(t);
            t.Dispose();
        });

        // Hold a reference to this timer to keep it from being disposed of.
        timers.Add(t);

        // Set the timer to release the semaphore in one second.
        t.Change(1000, Timeout.Infinite);
    }
}

EDIT

One thing to keep in mind is that the above code will only prevent many threads starting at once. If the threads are long running, then it's still possible to have many threads running at once. For instance, if you start 5 threads per second, but each thread runs for 1 second, then at any given time after 2 seconds, you'll theoretically have 10 threads running.

If you want to make sure you never have more than 5 threads running at once, the simplest thing to do is to dispense with the custom Limiter class, and just use a Semaphore directly.

const int maxThreadCount = 5;
static Semaphore counter = new Semaphore(maxThreadCount, maxThreadCount);

static void NewThread(object state){
    counter.WaitOne();

    // do something

    counter.Release();
}

Now that's just about as simple as it can be. One caveat though: creating new threads and immediately sleeping them is generally considered a bad idea. This uses system resources to create a thread that's not doing anything. It's better to queue up requests and start up new threads (or better yet, use thread pool threads) to handle them only when they are eligible to run. This is more complicated and harder to get right. Depending on design, it might require an additional thread for a scheduler/manage. Something like this:

class Limiter{
    class WorkData{
        readonly ParameterizedThreadStart action;
        readonly object                   data;

        public ParameterizedThreadStart Action{get{return action;}}
        public object                   Data  {get{return data;}}

        public WorkData(ParameterizedThreadStart action, object data){
            this.action = action;
            this.data   = data;
        }
    }

    readonly Semaphore       threadCount;
    readonly Queue<WorkData> workQueue   = new Queue<WorkData>();
    readonly Semaphore       queueCount  = new Semaphore(0, int.MaxValue);

    public Limiter(int maxThreadCount){
        threadCount = new Semaphore(maxThreadCount, maxThreadCount);
        Thread t = new Thread(StartWorkItems);
        t.IsBackground = true;
        t.Start();
    }

    void StartWorkItems(object ignored){
        while(queueCount.WaitOne() && threadCount.WaitOne()){
            WorkData wd;
            lock(workQueue)
                wd = workQueue.Dequeue();

            ThreadPool.QueueUserWorkItem(DoWork, wd);
        }
    }
    void DoWork(object state){
        WorkData wd = (WorkData)state;
        wd.Action(wd.Data);
        counter.Release();
    }

    public void QueueWork(ParameterizedThreadStart action, object data){
        lock(workQueue)
            workQueue.Enqueue(new WorkData(action, data));
        queueCount.Release();
    }
}

In this class, I've removed the singleton property and given the constructor a maxThreadCount parameter. This avoids the lack of the thread safety in the property of the first class. There are other ways thread safety can be added, but this was the simplest.


For the kind of resolution you are looking, DateTime.Now is an excellent clock and very cheap. It updates with a accuracy of a bit over 15 msec. Here's an example, call the Operation() method just before you perform the operation:

using System;

class Throttle {
  private int mTrigger;
  private int mOperations;
  private DateTime mStart;

  public Throttle(int maxOperationsPerSecond) {
    if (maxOperationsPerSecond < 1) throw new ArgumentException();
    mTrigger = maxOperationsPerSecond;
  }
  public void Operation() {
    mOperations += 1;
    if (mOperations > mTrigger) {
      TimeSpan span = DateTime.UtcNow - mStart;
      if (span.TotalMilliseconds < 1000)
        System.Threading.Thread.Sleep(1000 - (int)span.TotalMilliseconds);
      mOperations = 1;
    }
    if (mOperations == 1) mStart = DateTime.UtcNow;
  }
}

Create the instance of the class in your thread, don't share it.


I assume that getting the current time is inexpensive compared to whatever it is that the function would be doing.

In that case, I've done something like this in Scala, and the design pattern would go something like:

  • WaitIfRequired should block if already executing. (It would be a "synchronized method" in Java; I'm not sure what the .NET equivalent is.)
  • Maintain a queue of times that WaitIfRequired was called.
  • If the length of the queue goes over MaxExecutionPerSecond, cut it back until it is no more than MaxExecutionPerSecond long.
  • Pop the top off the queue, if it's MaxExecutionPerSecond long. If the time now is more than a second since then, push the current time on the tail of the queue and return immediately; enough time has elapsed. If it is less for a second, sleep for the amount of time needed for it to be a second (i.e. 1 second - time elapsed) before pushing and returning.

That's it.

You can play with this by asking for no more than N calls in time T by replacing "one second" by T.

Now, if you don't have synchronized methods, things get a little more interesting (but only a little). You then need an external locking mechanism to make sure only one thread at a time is reading times and waiting.


Comments

  1. Gentile

    • 2018/1/21

    Coding/Designing a generic thread-safe limiter (i.e. limit the execution of X() to Y many times per second) .net performance multithreading thread-safety limit.

  2. Brycen

    • 2020/4/25

    I'm planning to design a class to limit the execution of a function to given amount within the specified time, for example: Process max. 5 files in 1 second It should be thread-safe and performan

  3. Karson

    • 2018/2/3

    A non-reentrant function can often, but not always, be identified by its threadsafe function */ int diff(int x, int y) { int delta; delta = y - x; 

  4. Lopez

    • 2015/4/24

    The following sections provide general guidance about when to use a thread-safe collection versus its non-thread-safe equivalent that has a user-provided lock around its read and write operations. Because performance may vary depending on many factors, the guidance is not specific and is not necessarily valid in all circumstances.

  5. Judah

    • 2016/5/28

    First, we design a new efficient and thread-safe object model for dynamic languages, only requiring synchronization when writing to objects shared between 

  6. Fiore

    • 2021/1/20

    It's also possible to achieve thread-safety using the set of atomic classes that Java provides, including AtomicInteger, AtomicLong, AtomicBoolean, and AtomicReference. Atomic classes allow us to perform atomic operations, which are thread-safe, without using synchronization. An atomic operation is executed in one single machine level operation.

  7. Adonis

    • 2021/3/4

    Q: Why are transport-level communication services often inappropriate for Q: Would it make sense to limit the number of threads in a server process?

  8. Franklin

    • 2017/8/4

    Encoding &sol; Creating a Generic Thread-Safe Limiter &lpar;ie Limit X &lpar;&rpar; to Y several times per second&rpar; I'm planning to design a class to limit the execution of a function to given amount within the specified time, for example: Process max. 5 files in 1 second It should be thread-safe and performance hit should be minimal.

  9. Jad

    • 2021/6/8

    The text is intended for use in a second year Operational I tried to avoid creating formulas out of thin air, as is often done for the 

  10. Corbin

    • 2018/10/3

    This allows each thread to encounter any throttle notification and tell the limiter, and it only keeps record of the most recently encountered one, since it will also be the last to expire. Initially, I thought I would need to synchronize everywhere, but in the end, I didn't use any.

  11. Levi

    • 2017/11/18

    re-coding the algorithm executed by the DSP instead of by hardware DSPs and FPGAs are often connected to other parts of the system via low-latency.

  12. Kelvin

    • 2017/4/21

    Delegating Thread Safety to Multiple Underlying State Variables. for All Things Concurrency for that, see Concurrent Programming in Java (Lea, 2000).

  13. Castelli

    • 2019/5/16

    Technically, LDAP is a directory access protocol to an X.500 directory service, This question is raised many times, in different forms.

  14. Noah

    • 2018/12/8

    the and a in i it with that at this on from he my or we but as be they not will one time just like have people so can first which good know year all day 

  15. Brandon

    • 2017/10/7

    Computer Architecture Formulas. 1. CPU time = Instruction count Clock cycles per instruction Clock cycle time. 2. X is n times faster than Y: n=.

Comments are closed.

Recent Posts