Top Description Inners Fields Constructors Methods
org.eclipse.jetty.server.handler

public Class QoSHandler

extends Abstract
Class Inheritance
Annotations
@ManagedObject
Imports
java.time.Duration, java.util.Comparator, .Iterator, .Map, .Queue, .Set, java.util.concurrent.ConcurrentHashMap, .ConcurrentLinkedQueue, .ConcurrentSkipListSet, .TimeoutException, java.util.concurrent.atomic.AtomicInteger, org.eclipse.jetty.http.HttpStatus, org.eclipse.jetty.io.CyclicTimeouts, org.eclipse.jetty.server.Handler, .Request, .Response, org.eclipse.jetty.util.Callback, .NanoTime, .ProcessorUtils, org.eclipse.jetty.util.annotation.ManagedAttribute, .ManagedObject, org.eclipse.jetty.util.thread.Scheduler, .ThreadPool, org.slf4j.Logger, .LoggerFactory

A quality of service Handler that conditionally limits the number of concurrent requests, to provide more predictable end-user experience in case descendant Handlers have limited capacity.

This Handler limits the number of concurrent requests to the number configured via setMaxRequestCount(int). If more requests are received, they are suspended (that is, not forwarded to the child Handler) and stored in a priority queue. Priorities are determined via getPriority(Request), that should return values between 0 (the lowest priority) and positive numbers, typically in the range 0-10.

When a request that is being processed completes, the suspended request that current has the highest priority is resumed.

This Handler is ideal to avoid contending on slow/limited resources such as a JDBC connection pool, avoiding the situation where all server threads blocked contending on the limited resource, therefore leaving threads free to process other requests that do not require access to the limited resource.

Requests are resumed in priority order, so that when the server is under load, and there are many requests suspended to be processed, high priority request are processed first. For example, load balancer "ping" requests may have the highest priority, followed by requests performed by admin users, etc. so that regardless of the load, "ping" and "admin" requests will always be able to access the web application.

Nested and Inner Type Summary

Modifier and TypeClass and Description
private class
QoSHandler.Entry

private class
QoSHandler.Timeouts

Field Summary

Modifier and TypeField and Description
private static final String
EXPIRED_ATTRIBUTE_NAME

private static final Logger
LOG

Hides org.eclipse.jetty.server.handler.ConditionalHandler.LOG.

private int
maxRequests

private Duration
maxSuspend

private final Set<Integer>
priorities

private final Map<Integer, Queue<QoSHandler.Entry>>
queues

private final AtomicInteger
state

private CyclicTimeouts<QoSHandler.Entry>
timeouts

Constructor Summary

AccessConstructor and Description
public
QoSHandler()

public
QoSHandler(Handler handler)

Method Summary

Modifier and TypeMethod and Description
protected void
doStart()

Overrides org.eclipse.jetty.server.handler.ConditionalHandler.doStart.

Starts the managed lifecycle beans in the order they were added.
protected void
doStop()

Overrides org.eclipse.jetty.server.Handler.Abstract.doStop.

Stops the managed lifecycle beans in the reverse order they were added.
protected void
failSuspended(Request
the request to fail
request, Response
the response to fail
response, Callback
the callback to complete
callback, int
the failure status code
status, Throwable
the failure
failure)

Fails the given suspended request/response with the given error code and failure.

public int

Returns:

the max number of concurrent requests
getMaxRequestCount()

public Duration

Returns:

the max duration of time a request may stay suspended
getMaxSuspend()

Get the max duration of time a request may stay suspended.

protected int

Returns:

the priority of the given suspended request, a value >= 0
getPriority(Request
the suspended request to compute the priority for
request)

Returns the priority of the given suspended request, a value greater than or equal to 0.

public long
getSuspendedRequestCount()

private boolean
handleWithPermit(Request request, Response response, Callback callback)

private static void
notAvailable(Response response, Callback callback)

public boolean
onConditionsMet(Request
The request to handle
request, Response
The response to generate
response, Callback
The callback for completion
callback)

Implements abstract org.eclipse.jetty.server.handler.ConditionalHandler.onConditionsMet.

Handle a request that has met the conditions.
protected boolean
onConditionsNotMet(Request
The request to handle
request, Response
The response to generate
response, Callback
The callback for completion
callback)

Implements abstract org.eclipse.jetty.server.handler.ConditionalHandler.onConditionsNotMet.

This method is called when the request has not met the conditions and is not to be handled by this handler.
private void
resume(Throwable x)

private boolean
resumeSuspended()

public void
setMaxRequestCount(int
the max number of concurrent requests
maxRequests)

Sets the max number of concurrent requests.

public void
setMaxSuspend(Duration
the max duration of time a request may stay suspended
maxSuspend)

Sets the max duration of time a request may stay suspended.

private void
suspend(Request request, Response response, Callback callback)

Field Detail

EXPIRED_ATTRIBUTE_NAMEback to summary
private static final String EXPIRED_ATTRIBUTE_NAME
LOGback to summary
private static final Logger LOG

Hides org.eclipse.jetty.server.handler.ConditionalHandler.LOG.

maxRequestsback to summary
private int maxRequests
maxSuspendback to summary
private Duration maxSuspend
prioritiesback to summary
private final Set<Integer> priorities
queuesback to summary
private final Map<Integer, Queue<QoSHandler.Entry>> queues
stateback to summary
private final AtomicInteger state
timeoutsback to summary
private CyclicTimeouts<QoSHandler.Entry> timeouts

Constructor Detail

QoSHandlerback to summary
public QoSHandler()
QoSHandlerback to summary
public QoSHandler(Handler handler)

Method Detail

doStartback to summary
protected void doStart() throws Exception

Overrides org.eclipse.jetty.server.handler.ConditionalHandler.doStart.

Doc from org.eclipse.jetty.util.component.ContainerLifeCycle.doStart.

Starts the managed lifecycle beans in the order they were added.

Annotations
@Override
Exceptions
Exception:

Doc from org.eclipse.jetty.util.component.AbstractLifeCycle.doStart.

If there was a problem starting. Will cause a transition to FAILED state

doStopback to summary
protected void doStop() throws Exception

Overrides org.eclipse.jetty.server.Handler.Abstract.doStop.

Doc from org.eclipse.jetty.util.component.ContainerLifeCycle.doStop.

Stops the managed lifecycle beans in the reverse order they were added.

Annotations
@Override
Exceptions
Exception:

Doc from org.eclipse.jetty.util.component.AbstractLifeCycle.doStop.

If there was a problem stopping. Will cause a transition to FAILED state

failSuspendedback to summary
protected void failSuspended(Request request, Response response, Callback callback, int status, Throwable failure)

Fails the given suspended request/response with the given error code and failure.

This method is called only for suspended requests, in case of timeout while suspended, or in case of failure when trying to handle a resumed request.

Parameters
request:Request

the request to fail

response:Response

the response to fail

callback:Callback

the callback to complete

status:int

the failure status code

failure:Throwable

the failure

getMaxRequestCountback to summary
public int getMaxRequestCount()
Returns:int

the max number of concurrent requests

Annotations
@ManagedAttribute:The maximum number of concurrent requests
readonly:true
getMaxSuspendback to summary
public Duration getMaxSuspend()

Get the max duration of time a request may stay suspended.

Returns:Duration

the max duration of time a request may stay suspended

getPriorityback to summary
protected int getPriority(Request request)

Returns the priority of the given suspended request, a value greater than or equal to 0.

Priority 0 is the lowest priority.

The set of returned priorities should be stable over time, typically constrained in the range 0-10.

Parameters
request:Request

the suspended request to compute the priority for

Returns:int

the priority of the given suspended request, a value >= 0

getSuspendedRequestCountback to summary
public long getSuspendedRequestCount()
Annotations
@ManagedAttribute:The number of suspended requests
handleWithPermitback to summary
private boolean handleWithPermit(Request request, Response response, Callback callback) throws Exception
notAvailableback to summary
private static void notAvailable(Response response, Callback callback)
onConditionsMetback to summary
public boolean onConditionsMet(Request request, Response response, Callback callback) throws Exception

Implements abstract org.eclipse.jetty.server.handler.ConditionalHandler.onConditionsMet.

Doc from org.eclipse.jetty.server.handler.ConditionalHandler.onConditionsMet.

Handle a request that has met the conditions. Typically, the implementation will provide optional handling and then call the nextHandler(Request, Response, Callback) method to continue handling.

Parameters
request:Request

The request to handle

response:Response

The response to generate

callback:Callback

The callback for completion

Returns:boolean

True if this handler will complete the callback

Annotations
@Override
Exceptions
Exception:
If there is a problem handling
onConditionsNotMetback to summary
protected boolean onConditionsNotMet(Request request, Response response, Callback callback) throws Exception

Implements abstract org.eclipse.jetty.server.handler.ConditionalHandler.onConditionsNotMet.

Doc from org.eclipse.jetty.server.handler.ConditionalHandler.onConditionsNotMet.

This method is called when the request has not met the conditions and is not to be handled by this handler. Implementations may return false; send an error response; or handle the request differently.

Parameters
request:Request

The request to handle

response:Response

The response to generate

callback:Callback

The callback for completion

Returns:boolean

True if this handler will complete the callback

Annotations
@Override
Exceptions
Exception:
If there is a problem handling
resumeback to summary
private void resume(Throwable x)
resumeSuspendedback to summary
private boolean resumeSuspended()
setMaxRequestCountback to summary
public void setMaxRequestCount(int maxRequests)

Sets the max number of concurrent requests.

A negative or zero value indicates to calculate a value based on heuristics, drawn from the number the size of the server thread pool and/or the number of CPU cores.

Parameters
maxRequests:int

the max number of concurrent requests

setMaxSuspendback to summary
public void setMaxSuspend(Duration maxSuspend)

Sets the max duration of time a request may stay suspended.

Once the duration expires, the request is failed with an HTTP status of 503 Service Unavailable.

Duration#ZERO means that the request may stay suspended forever.

Parameters
maxSuspend:Duration

the max duration of time a request may stay suspended

suspendback to summary
private void suspend(Request request, Response response, Callback callback)
org.eclipse.jetty.server.handler back to summary

private Class QoSHandler.Entry

extends Object
implements Expirable, Runnable
Class Inheritance
All Implemented Interfaces
java.lang.Runnable, org.eclipse.jetty.io.CyclicTimeouts.Expirable

Field Summary

Modifier and TypeField and Description
private final Callback
callback

private final long
expireNanoTime

private final int
priority

private final Request
request

private final Response
response

Constructor Summary

AccessConstructor and Description
private
Entry(Request request, Response response, Callback callback, int priority)

Method Summary

Modifier and TypeMethod and Description
private void
expire()

public long
getExpireNanoTime()

Implements org.eclipse.jetty.io.CyclicTimeouts.Expirable.getExpireNanoTime.

Returns the expiration time in nanoseconds.

public void
run()

Implements java.lang.Runnable.run.

Runs this operation.
Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

callbackback to summary
private final Callback callback
expireNanoTimeback to summary
private final long expireNanoTime
priorityback to summary
private final int priority
requestback to summary
private final Request request
responseback to summary
private final Response response

Constructor Detail

Entryback to summary
private Entry(Request request, Response response, Callback callback, int priority)

Method Detail

expireback to summary
private void expire()
getExpireNanoTimeback to summary
public long getExpireNanoTime()

Implements org.eclipse.jetty.io.CyclicTimeouts.Expirable.getExpireNanoTime.

Doc from org.eclipse.jetty.io.CyclicTimeouts.Expirable.getExpireNanoTime.

Returns the expiration time in nanoseconds.

The value to return must be calculated taking into account the current nanoTime, for example:

expireNanoTime = NanoTime.now() + timeoutNanos

Returning Long#MAX_VALUE indicates that this entity does not expire.

Returns:long

the expiration time in nanoseconds, or Long#MAX_VALUE if this entity does not expire

Annotations
@Override
runback to summary
public void run()

Implements java.lang.Runnable.run.

Doc from java.lang.Runnable.run.

Runs this operation.

Annotations
@Override
org.eclipse.jetty.server.handler back to summary

private Class QoSHandler.Timeouts

extends CyclicTimeouts<QoSHandler.Entry>
Class Inheritance

Constructor Summary

AccessConstructor and Description
private
Timeouts(Scheduler scheduler)

Method Summary

Modifier and TypeMethod and Description
protected Iterator<QoSHandler.Entry>
iterator()

Implements abstract org.eclipse.jetty.io.CyclicTimeouts.iterator.

protected boolean
onExpired(QoSHandler.Entry
the entity that is expired
entry)

Implements abstract org.eclipse.jetty.io.CyclicTimeouts.onExpired.

Invoked during the iteration when the given entity is expired.

Inherited from org.eclipse.jetty.io.CyclicTimeouts:
destroyschedule

Constructor Detail

Timeoutsback to summary
private Timeouts(Scheduler scheduler)

Method Detail

iteratorback to summary
protected Iterator<QoSHandler.Entry> iterator()

Implements abstract org.eclipse.jetty.io.CyclicTimeouts.iterator.

Returns:Iterator<QoSHandler.Entry>

Doc from org.eclipse.jetty.io.CyclicTimeouts.iterator.

the entities to iterate over when this instance expires

Annotations
@Override
onExpiredback to summary
protected boolean onExpired(QoSHandler.Entry entry)

Implements abstract org.eclipse.jetty.io.CyclicTimeouts.onExpired.

Doc from org.eclipse.jetty.io.CyclicTimeouts.onExpired.

Invoked during the iteration when the given entity is expired.

This method may be invoked multiple times, and even concurrently, for the same expirable entity and therefore the expiration of the entity, if any, should be an idempotent action.

When false is returned, the implementation should adjust the Expirable expiration, so that a call to Expirable#getExpireNanoTime() after this method has returned yields a new expiration nanoTime.

Parameters
entry:QoSHandler.Entry

the entity that is expired

Returns:boolean

whether the entity should be removed from the iterator via Iterator#remove()

Annotations
@Override