Rate Limiting the right way

While dealing with external services you would want to throttle requests. In Go, its pretty simple to implement it with something like this.

While this works for tens of operations a second, this does not scale for 1000s of operations a second. We have been working with an AMQP queue that needs to be drained with back pressure. Our downstream system is sending SMSs using a gateway and can only send a prescribed amount of SMSs but it is greater than tens of operations a second. There is a supplementary time package which provides rate limiting that works with token buckets.

The token bucket is an algorithm used in packet switched computer networks and telecommunications networks. It can be used to check that data transmissions, in the form of packets, conform to defined limits on bandwidth and burstiness (a measure of the unevenness or variations in the traffic flow). It can also be used as a scheduling algorithm to determine the timing of transmissions that will comply with the limits set for the bandwidth and burstiness. – Wikipedia

So you start off by creating a limit which defines the interval. For example, if you would like for 2000 events a minute.

You would then create a limiter that would setup the burst rate per second. So in our case you would want something like this.

Then you can reserve a token and sleep for the duration indicated by the reservation.

You can verify if things are going fine by printing the value every second.

You can also reserve multiple tokens at a time and do as many operations. This will ensure that the scheduling overhead on the goroutine is avoided.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s