The token-bucket package provides an implementation of the
token bucket algorithm
suitable for use in web applications for shaping or policing request
rates. This implementation does not require the use of an independent
timer thread to manage the bucket state.
Compared to other rate-limiting algorithms that use a simple counter, the token bucket algorithm provides the following advantages:
- The thundering herd problem is avoided since bucket capacity is replenished gradually, rather than being immediately refilled at the beginning of each epoch as is common with simple fixed window counters.
- Burst duration can be explicitly controlled
Moving window algorithms are resitant to bursting, but at the cost of additional processing and memory overhead vs. the token bucket algorithm which uses a simple, fast counter per key. The latter approach does allow for bursting, but only for a controlled duration.