About a year ago, we migrated an old rate limiter in order to serve more traffic and accommodate a more resilient platform architecture. We adopted a replicated Redis backend with client-side sharding. In the end, it worked out great, but we learned some lessons along the way.

The Problem

We had an old rate limiter that was simple enough:

• For every request, determine a “key” for the current rate limit
• In Memcached, increment the value of that key, setting it to 1 if there wasn’t any current value
• Also, if there wasn’t already one, set a “reset at” value in Memcached, using a related key (eg, “#{key}:reset_at“)
• When incrementing, if the “reset at” value is in the past, ignore the existing value and set a new “reset at”
• At the beginning of each request, if the value for the key is above the limit, and “reset at” is in the future, then reject the request

(There might have been more nuance to it, but that’s the main idea.)

However, this limiter had two problems…

This thread was posted by one of our members via one of our news source trackers.