Top Description Inners Fields Constructors Methods
com.google.common.util.concurrent

public abstract Class RateLimiter

extends Object
Class Inheritance
Known Direct Subclasses
com.google.common.util.concurrent.SmoothRateLimiter
Annotations
@Beta
@J2ktIncompatible
@GwtIncompatible
@ElementTypesAreNonnullByDefault
Static Imports
com.google.common.base.Preconditions.checkArgument, .Preconditions.checkNotNull, com.google.common.util.concurrent.Internal.toNanosSaturated, java.lang.Math.max, java.util.concurrent.TimeUnit.MICROSECONDS, .TimeUnit.SECONDS
A rate limiter. Conceptually, a rate limiter distributes permits at a configurable rate. Each acquire() blocks if necessary until a permit is available, and then takes it. Once acquired, permits need not be released.

RateLimiter is safe for concurrent use: It will restrict the total rate of calls from all threads. Note, however, that it does not guarantee fairness.

Rate limiters are often used to restrict the rate at which some physical or logical resource is accessed. This is in contrast to java.util.concurrent.Semaphore which restricts the number of concurrent accesses instead of the rate (note though that concurrency and rate are closely related, e.g. see Little's Law).

A RateLimiter is defined primarily by the rate at which permits are issued. Absent additional configuration, permits will be distributed at a fixed rate, defined in terms of permits per second. Permits will be distributed smoothly, with the delay between individual permits being adjusted to ensure that the configured rate is maintained.

It is possible to configure a RateLimiter to have a warmup period during which time the permits issued each second steadily increases until it hits the stable rate.

As an example, imagine that we have a list of tasks to execute, but we don't want to submit more than 2 per second: 

final RateLimiter rateLimiter = RateLimiter.create(2.0); // rate is "2 permits per second"
void submitTasks(List<Runnable> tasks, Executor executor) {
  for (Runnable task : tasks) {
    rateLimiter.acquire(); // may wait
    executor.execute(task);
  }
}

As another example, imagine that we produce a stream of data, and we want to cap it at 5kb per second. This could be accomplished by requiring a permit per byte, and specifying a rate of 5000 permits per second: 

final RateLimiter rateLimiter = RateLimiter.create(5000.0); // rate = 5000 permits per second
void submitPacket(byte[] packet) {
  rateLimiter.acquire(packet.length);
  networkService.send(packet);
}

It is important to note that the number of permits requested never affects the throttling of the request itself (an invocation to acquire(1) and an invocation to acquire(1000) will result in exactly the same throttling, if any), but it affects the throttling of the next request. I.e., if an expensive task arrives at an idle RateLimiter, it will be granted immediately, but it is the next request that will experience extra throttling, thus paying for the cost of the expensive task.

Author
Dimitris Andreou
Since
13.0

Nested and Inner Type Summary

Modifier and TypeClass and Description
pack-priv abstract static class
RateLimiter.SleepingStopwatch

Field Summary

Modifier and TypeField and Description
private volatile Object
mutexDoNotUseDirectly

private final RateLimiter.SleepingStopwatch
stopwatch

The underlying timer; used both to measure elapsed time and sleep as necessary.

Constructor Summary

AccessConstructor and Description
pack-priv
RateLimiter(RateLimiter.SleepingStopwatch stopwatch)

Method Summary

Modifier and TypeMethod and Description
public double

Returns:

time spent sleeping to enforce rate, in seconds; 0.0 if not rate-limited
acquire()

Acquires a single permit from this RateLimiter, blocking until the request can be granted.

public double

Returns:

time spent sleeping to enforce rate, in seconds; 0.0 if not rate-limited
acquire(int
the number of permits to acquire
permits)

Acquires the given number of permits from this RateLimiter, blocking until the request can be granted.

private boolean
canAcquire(long nowMicros, long timeoutMicros)

private static void
checkPermits(int permits)

public static RateLimiter
create(double
the rate of the returned RateLimiter, measured in how many permits become available per second
permitsPerSecond)

Creates a RateLimiter with the specified stable throughput, given as "permits per second" (commonly referred to as QPS, queries per second).

pack-priv static RateLimiter
create(double permitsPerSecond, RateLimiter.SleepingStopwatch stopwatch)

public static RateLimiter
create(double
the rate of the returned RateLimiter, measured in how many permits become available per second
permitsPerSecond, Duration
the duration of the period where the RateLimiter ramps up its rate, before reaching its stable (maximum) rate
warmupPeriod)

Creates a RateLimiter with the specified stable throughput, given as "permits per second" (commonly referred to as QPS, queries per second), and a warmup period, during which the RateLimiter smoothly ramps up its rate, until it reaches its maximum rate at the end of the period (as long as there are enough requests to saturate it).

public static RateLimiter
create(double
the rate of the returned RateLimiter, measured in how many permits become available per second
permitsPerSecond, long
the duration of the period where the RateLimiter ramps up its rate, before reaching its stable (maximum) rate
warmupPeriod, TimeUnit
the time unit of the warmupPeriod argument
unit)

Creates a RateLimiter with the specified stable throughput, given as "permits per second" (commonly referred to as QPS, queries per second), and a warmup period, during which the RateLimiter smoothly ramps up its rate, until it reaches its maximum rate at the end of the period (as long as there are enough requests to saturate it).

pack-priv static RateLimiter
create(double permitsPerSecond, long warmupPeriod, TimeUnit unit, double coldFactor, RateLimiter.SleepingStopwatch stopwatch)

pack-priv abstract double
doGetRate()

pack-priv abstract void
doSetRate(double permitsPerSecond, long nowMicros)

public final double
getRate()

Returns the stable rate (as permits per seconds) with which this RateLimiter is configured with.

private Object
mutex()

pack-priv abstract long

Returns:

the time that permits are available, or, if permits are available immediately, an arbitrary past or present time
queryEarliestAvailable(long nowMicros)

Returns the earliest time that permits are available (with one caveat).

pack-priv final long

Returns:

time in microseconds to wait until the resource can be acquired, never negative
reserve(int permits)

Reserves the given number of permits from this RateLimiter for future use, returning the number of microseconds until the reservation can be consumed.

pack-priv final long

Returns:

the required wait time, never negative
reserveAndGetWaitLength(int permits, long nowMicros)

Reserves next ticket and returns the wait time that the caller must wait for.

pack-priv abstract long

Returns:

the time that the permits may be used, or, if the permits may be used immediately, an arbitrary past or present time
reserveEarliestAvailable(int permits, long nowMicros)

Reserves the requested number of permits and returns the time that those permits can be used (with one caveat).

public final void
setRate(double
the new stable rate of this RateLimiter
permitsPerSecond)

Updates the stable rate of this RateLimiter, that is, the permitsPerSecond argument provided in the factory method that constructed the RateLimiter.

public String
toString()

Overrides java.lang.Object.toString.

Returns a string representation of the object.
public boolean

Returns:

true if the permit was acquired, false otherwise
tryAcquire(Duration
the maximum time to wait for the permit. Negative values are treated as zero.
timeout)

Acquires a permit from this RateLimiter if it can be obtained without exceeding the specified timeout, or returns false immediately (without waiting) if the permit would not have been granted before the timeout expired.

public boolean

Returns:

true if the permit was acquired, false otherwise
tryAcquire(long
the maximum time to wait for the permit. Negative values are treated as zero.
timeout, TimeUnit
the time unit of the timeout argument
unit)

Acquires a permit from this RateLimiter if it can be obtained without exceeding the specified timeout, or returns false immediately (without waiting) if the permit would not have been granted before the timeout expired.

public boolean

Returns:

true if the permits were acquired, false otherwise
tryAcquire(int
the number of permits to acquire
permits)

Acquires permits from this RateLimiter if it can be acquired immediately without delay.

public boolean

Returns:

true if the permit was acquired, false otherwise
tryAcquire()

Acquires a permit from this RateLimiter if it can be acquired immediately without delay.

public boolean

Returns:

true if the permits were acquired, false otherwise
tryAcquire(int
the number of permits to acquire
permits, Duration
the maximum time to wait for the permits. Negative values are treated as zero.
timeout)

Acquires the given number of permits from this RateLimiter if it can be obtained without exceeding the specified timeout, or returns false immediately (without waiting) if the permits would not have been granted before the timeout expired.

public boolean

Returns:

true if the permits were acquired, false otherwise
tryAcquire(int
the number of permits to acquire
permits, long
the maximum time to wait for the permits. Negative values are treated as zero.
timeout, TimeUnit
the time unit of the timeout argument
unit)

Acquires the given number of permits from this RateLimiter if it can be obtained without exceeding the specified timeout, or returns false immediately (without waiting) if the permits would not have been granted before the timeout expired.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAllwaitwaitwait

Field Detail

mutexDoNotUseDirectlyback to summary
private volatile Object mutexDoNotUseDirectly
Annotations
@CheckForNull
stopwatchback to summary
private final RateLimiter.SleepingStopwatch stopwatch

The underlying timer; used both to measure elapsed time and sleep as necessary. A separate object to facilitate testing.

Constructor Detail

RateLimiterback to summary
pack-priv RateLimiter(RateLimiter.SleepingStopwatch stopwatch)

Method Detail

acquireback to summary
public double acquire()

Acquires a single permit from this RateLimiter, blocking until the request can be granted. Tells the amount of time slept, if any.

This method is equivalent to acquire(1).

Returns:double

time spent sleeping to enforce rate, in seconds; 0.0 if not rate-limited

Annotations
@CanIgnoreReturnValue
Since
16.0 (present in 13.0 with void return type})
acquireback to summary
public double acquire(int permits)

Acquires the given number of permits from this RateLimiter, blocking until the request can be granted. Tells the amount of time slept, if any.

Parameters
permits:int

the number of permits to acquire

Returns:double

time spent sleeping to enforce rate, in seconds; 0.0 if not rate-limited

Annotations
@CanIgnoreReturnValue
Exceptions
IllegalArgumentException:
if the requested number of permits is negative or zero
Since
16.0 (present in 13.0 with void return type})
canAcquireback to summary
private boolean canAcquire(long nowMicros, long timeoutMicros)
checkPermitsback to summary
private static void checkPermits(int permits)
createback to summary
public static RateLimiter create(double permitsPerSecond)

Creates a RateLimiter with the specified stable throughput, given as "permits per second" (commonly referred to as QPS, queries per second).

The returned RateLimiter ensures that on average no more than permitsPerSecond are issued during any given second, with sustained requests being smoothly spread over each second. When the incoming request rate exceeds permitsPerSecond the rate limiter will release one permit every (1.0 / permitsPerSecond) seconds. When the rate limiter is unused, bursts of up to permitsPerSecond permits will be allowed, with subsequent requests being smoothly limited at the stable rate of permitsPerSecond.

Parameters
permitsPerSecond:double

the rate of the returned RateLimiter, measured in how many permits become available per second

Exceptions
IllegalArgumentException:
if permitsPerSecond is negative or zero
createback to summary
pack-priv static RateLimiter create(double permitsPerSecond, RateLimiter.SleepingStopwatch stopwatch)
Annotations
@VisibleForTesting
createback to summary
public static RateLimiter create(double permitsPerSecond, Duration warmupPeriod)

Creates a RateLimiter with the specified stable throughput, given as "permits per second" (commonly referred to as QPS, queries per second), and a warmup period, during which the RateLimiter smoothly ramps up its rate, until it reaches its maximum rate at the end of the period (as long as there are enough requests to saturate it). Similarly, if the RateLimiter is left unused for a duration of warmupPeriod, it will gradually return to its "cold" state, i.e. it will go through the same warming up process as when it was first created.

The returned RateLimiter is intended for cases where the resource that actually fulfills the requests (e.g., a remote server) needs "warmup" time, rather than being immediately accessed at the stable (maximum) rate.

The returned RateLimiter starts in a "cold" state (i.e. the warmup period will follow), and if it is left unused for long enough, it will return to that state.

Parameters
permitsPerSecond:double

the rate of the returned RateLimiter, measured in how many permits become available per second

warmupPeriod:Duration

the duration of the period where the RateLimiter ramps up its rate, before reaching its stable (maximum) rate

Exceptions
IllegalArgumentException:
if permitsPerSecond is negative or zero or warmupPeriod is negative
Since
28.0
createback to summary
public static RateLimiter create(double permitsPerSecond, long warmupPeriod, TimeUnit unit)

Creates a RateLimiter with the specified stable throughput, given as "permits per second" (commonly referred to as QPS, queries per second), and a warmup period, during which the RateLimiter smoothly ramps up its rate, until it reaches its maximum rate at the end of the period (as long as there are enough requests to saturate it). Similarly, if the RateLimiter is left unused for a duration of warmupPeriod, it will gradually return to its "cold" state, i.e. it will go through the same warming up process as when it was first created.

The returned RateLimiter is intended for cases where the resource that actually fulfills the requests (e.g., a remote server) needs "warmup" time, rather than being immediately accessed at the stable (maximum) rate.

The returned RateLimiter starts in a "cold" state (i.e. the warmup period will follow), and if it is left unused for long enough, it will return to that state.

Parameters
permitsPerSecond:double

the rate of the returned RateLimiter, measured in how many permits become available per second

warmupPeriod:long

the duration of the period where the RateLimiter ramps up its rate, before reaching its stable (maximum) rate

unit:TimeUnit

the time unit of the warmupPeriod argument

Annotations
@SuppressWarnings:GoodTime
Exceptions
IllegalArgumentException:
if permitsPerSecond is negative or zero or warmupPeriod is negative
createback to summary
pack-priv static RateLimiter create(double permitsPerSecond, long warmupPeriod, TimeUnit unit, double coldFactor, RateLimiter.SleepingStopwatch stopwatch)
Annotations
@VisibleForTesting
doGetRateback to summary
pack-priv abstract double doGetRate()
doSetRateback to summary
pack-priv abstract void doSetRate(double permitsPerSecond, long nowMicros)
getRateback to summary
public final double getRate()

Returns the stable rate (as permits per seconds) with which this RateLimiter is configured with. The initial value of this is the same as the permitsPerSecond argument passed in the factory method that produced this RateLimiter, and it is only updated after invocations to setRate.

mutexback to summary
private Object mutex()
queryEarliestAvailableback to summary
pack-priv abstract long queryEarliestAvailable(long nowMicros)

Returns the earliest time that permits are available (with one caveat).

Returns:long

the time that permits are available, or, if permits are available immediately, an arbitrary past or present time

reserveback to summary
pack-priv final long reserve(int permits)

Reserves the given number of permits from this RateLimiter for future use, returning the number of microseconds until the reservation can be consumed.

Returns:long

time in microseconds to wait until the resource can be acquired, never negative

reserveAndGetWaitLengthback to summary
pack-priv final long reserveAndGetWaitLength(int permits, long nowMicros)

Reserves next ticket and returns the wait time that the caller must wait for.

Returns:long

the required wait time, never negative

reserveEarliestAvailableback to summary
pack-priv abstract long reserveEarliestAvailable(int permits, long nowMicros)

Reserves the requested number of permits and returns the time that those permits can be used (with one caveat).

Returns:long

the time that the permits may be used, or, if the permits may be used immediately, an arbitrary past or present time

setRateback to summary
public final void setRate(double permitsPerSecond)

Updates the stable rate of this RateLimiter, that is, the permitsPerSecond argument provided in the factory method that constructed the RateLimiter. Currently throttled threads will not be awakened as a result of this invocation, thus they do not observe the new rate; only subsequent requests will.

Note though that, since each request repays (by waiting, if necessary) the cost of the previous request, this means that the very next request after an invocation to setRate will not be affected by the new rate; it will pay the cost of the previous request, which is in terms of the previous rate.

The behavior of the RateLimiter is not modified in any other way, e.g. if the RateLimiter was configured with a warmup period of 20 seconds, it still has a warmup period of 20 seconds after this method invocation.

Parameters
permitsPerSecond:double

the new stable rate of this RateLimiter

Exceptions
IllegalArgumentException:
if permitsPerSecond is negative or zero
toStringback to summary
public String toString()

Overrides java.lang.Object.toString.

Doc from java.lang.Object.toString.

Returns a string representation of the object.

Returns:String

a string representation of the object

Annotations
@Override
tryAcquireback to summary
public boolean tryAcquire(Duration timeout)

Acquires a permit from this RateLimiter if it can be obtained without exceeding the specified timeout, or returns false immediately (without waiting) if the permit would not have been granted before the timeout expired.

This method is equivalent to tryAcquire(1, timeout).

Parameters
timeout:Duration

the maximum time to wait for the permit. Negative values are treated as zero.

Returns:boolean

true if the permit was acquired, false otherwise

Exceptions
IllegalArgumentException:
if the requested number of permits is negative or zero
Since
28.0
tryAcquireback to summary
public boolean tryAcquire(long timeout, TimeUnit unit)

Acquires a permit from this RateLimiter if it can be obtained without exceeding the specified timeout, or returns false immediately (without waiting) if the permit would not have been granted before the timeout expired.

This method is equivalent to tryAcquire(1, timeout, unit).

Parameters
timeout:long

the maximum time to wait for the permit. Negative values are treated as zero.

unit:TimeUnit

the time unit of the timeout argument

Returns:boolean

true if the permit was acquired, false otherwise

Annotations
@SuppressWarnings:GoodTime
Exceptions
IllegalArgumentException:
if the requested number of permits is negative or zero
tryAcquireback to summary
public boolean tryAcquire(int permits)

Acquires permits from this RateLimiter if it can be acquired immediately without delay.

This method is equivalent to tryAcquire(permits, 0, anyUnit).

Parameters
permits:int

the number of permits to acquire

Returns:boolean

true if the permits were acquired, false otherwise

Exceptions
IllegalArgumentException:
if the requested number of permits is negative or zero
Since
14.0
tryAcquireback to summary
public boolean tryAcquire()

Acquires a permit from this RateLimiter if it can be acquired immediately without delay.

This method is equivalent to tryAcquire(1).

Returns:boolean

true if the permit was acquired, false otherwise

Since
14.0
tryAcquireback to summary
public boolean tryAcquire(int permits, Duration timeout)

Acquires the given number of permits from this RateLimiter if it can be obtained without exceeding the specified timeout, or returns false immediately (without waiting) if the permits would not have been granted before the timeout expired.

Parameters
permits:int

the number of permits to acquire

timeout:Duration

the maximum time to wait for the permits. Negative values are treated as zero.

Returns:boolean

true if the permits were acquired, false otherwise

Exceptions
IllegalArgumentException:
if the requested number of permits is negative or zero
Since
28.0
tryAcquireback to summary
public boolean tryAcquire(int permits, long timeout, TimeUnit unit)

Acquires the given number of permits from this RateLimiter if it can be obtained without exceeding the specified timeout, or returns false immediately (without waiting) if the permits would not have been granted before the timeout expired.

Parameters
permits:int

the number of permits to acquire

timeout:long

the maximum time to wait for the permits. Negative values are treated as zero.

unit:TimeUnit

the time unit of the timeout argument

Returns:boolean

true if the permits were acquired, false otherwise

Annotations
@SuppressWarnings:GoodTime
Exceptions
IllegalArgumentException:
if the requested number of permits is negative or zero
com.google.common.util.concurrent back to summary

pack-priv abstract Class RateLimiter.SleepingStopwatch

extends Object
Class Inheritance
  • java.lang.Object
  • com.google.common.util.concurrent.RateLimiter.SleepingStopwatch

Constructor Summary

AccessConstructor and Description
protected
SleepingStopwatch()

Constructor for use by subclasses.

Method Summary

Modifier and TypeMethod and Description
public static RateLimiter.SleepingStopwatch
createFromSystemTimer()

protected abstract long
readMicros()

protected abstract void
sleepMicrosUninterruptibly(long micros)

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Constructor Detail

SleepingStopwatchback to summary
protected SleepingStopwatch()

Constructor for use by subclasses.

Method Detail

createFromSystemTimerback to summary
public static RateLimiter.SleepingStopwatch createFromSystemTimer()
readMicrosback to summary
protected abstract long readMicros()
sleepMicrosUninterruptiblyback to summary
protected abstract void sleepMicrosUninterruptibly(long micros)