Rate limiting protects your services from abuse and keeps latency predictable under load. Of the many approaches, the token bucket is the most widely used because it allows short bursts while enforcing a steady average rate.
How it works
Imagine a bucket that holds tokens:
- Tokens are added at a fixed rate (say, 10 per second).
- The bucket has a maximum capacity — it can't overflow.
- Every request removes one token. No token, no service.
This lets a client burst up to the bucket's capacity, then throttles them to the refill rate.
A Go implementation
package ratelimit
import (
"sync"
"time"
)
type TokenBucket struct {
mu sync.Mutex
tokens float64
capacity float64
refillRate float64 // tokens per second
lastRefill time.Time
}
func New(capacity, refillRate float64) *TokenBucket {
return &TokenBucket{
tokens: capacity,
capacity: capacity,
refillRate: refillRate,
lastRefill: time.Now(),
}
}
func (b *TokenBucket) Allow() bool {
b.mu.Lock()
defer b.mu.Unlock()
now := time.Now()
elapsed := now.Sub(b.lastRefill).Seconds()
b.tokens = min(b.capacity, b.tokens+elapsed*b.refillRate)
b.lastRefill = now
if b.tokens >= 1 {
b.tokens--
return true
}
return false
}The same idea in TypeScript
export class TokenBucket {
private tokens: number
private lastRefill = Date.now()
constructor(
private capacity: number,
private refillPerSecond: number,
) {
this.tokens = capacity
}
allow(): boolean {
const now = Date.now()
const elapsed = (now - this.lastRefill) / 1000
this.tokens = Math.min(this.capacity, this.tokens + elapsed * this.refillPerSecond)
this.lastRefill = now
if (this.tokens >= 1) {
this.tokens -= 1
return true
}
return false
}
}Choosing parameters
The two knobs — capacity and refillRate — map directly to product behavior:
- Capacity controls how large a burst you tolerate.
- Refill rate controls the sustained throughput per client.
For a public API, a good starting point is a capacity equal to a few seconds of your refill rate. Store buckets in Redis when you need to share state across multiple instances.