Top Description Inners Fields Constructors Methods
org.apache.catalina.valves

public Class CrawlerSessionManagerValve

extends ValveBase
Class Inheritance
Imports
java.io.IOException, .Serializable, java.util.Enumeration, .Map, java.util.concurrent.ConcurrentHashMap, java.util.regex.Pattern, jakarta.servlet.ServletException, jakarta.servlet.http.HttpSession, .HttpSessionBindingEvent, .HttpSessionBindingListener, org.apache.catalina.Context, .Host, .LifecycleException, org.apache.catalina.connector.Request, .Response, org.apache.juli.logging.Log, .LogFactory

Web crawlers can trigger the creation of many thousands of sessions as they crawl a site which may result in significant memory consumption. This Valve ensures that crawlers are associated with a single session - just like normal users - regardless of whether or not they provide a session token with their requests.

Nested and Inner Type Summary

Modifier and TypeClass and Description
private static class

Field Summary

Modifier and TypeField and Description
private final Map<String, String>
private String
private String
private Pattern
private boolean
private boolean
private static final Log
private final Map<String, String>
private int
private Pattern
Inherited from org.apache.catalina.valves.ValveBase:
asyncSupportedcontainercontainerLognextsm

Constructor Summary

AccessConstructor and Description
public
CrawlerSessionManagerValve()

Specifies a default constructor so async support can be configured.

Method Summary

Modifier and TypeMethod and Description
private String
getClientIdentifier(Host host, Context context, String clientIp)

public Map<String, String>
public String

Returns:

The current regular expression being used to match IP addresses.
getCrawlerIps
()

public String

Returns:

The current regular expression being used to match user agents.
getCrawlerUserAgents
()

public int

Returns:

The current timeout in seconds
getSessionInactiveInterval
()

protected void
initInternal()

Overrides org.apache.catalina.valves.ValveBase.initInternal.

Sub-classes wishing to perform additional initialization should override this method, ensuring that super.initInternal() is the first call in the overriding method.

public void
invoke(Request
The servlet request to be processed
request
,
Response
The servlet response to be created
response
)

Implements org.apache.catalina.Valve.invoke.

Perform request processing as required by this Valve.

public boolean
public boolean
public void
setContextAware(boolean isContextAware)

public void
setCrawlerIps(String
The regular expression using Pattern
crawlerIps
)

Specify the regular expression (using Pattern) that will be used to identify crawlers based on their IP address.

public void
setCrawlerUserAgents(String
The regular expression using Pattern
crawlerUserAgents
)

Specify the regular expression (using Pattern) that will be used to identify crawlers based in the User-Agent header provided.

public void
setHostAware(boolean isHostAware)

public void
setSessionInactiveInterval(int
The new timeout for crawler sessions
sessionInactiveInterval
)

Specify the session timeout (in seconds) for a crawler's session.

Inherited from org.apache.catalina.valves.ValveBase:
backgroundProcessgetContainergetDomainInternalgetNextgetObjectNameKeyPropertiesisAsyncSupportedsetAsyncSupportedsetContainersetNextstartInternalstopInternaltoString

Field Detail

clientIdSessionIdback to summary
private final Map<String, String> clientIdSessionId
crawlerIpsback to summary
private String crawlerIps
crawlerUserAgentsback to summary
private String crawlerUserAgents
ipPatternback to summary
private Pattern ipPattern
isContextAwareback to summary
private boolean isContextAware
isHostAwareback to summary
private boolean isHostAware
logback to summary
private static final Log log

Hides org.apache.catalina.util.LifecycleMBeanBase.log.

sessionIdClientIdback to summary
private final Map<String, String> sessionIdClientId
sessionInactiveIntervalback to summary
private int sessionInactiveInterval
uaPatternback to summary
private Pattern uaPattern

Constructor Detail

CrawlerSessionManagerValveback to summary
public CrawlerSessionManagerValve()

Specifies a default constructor so async support can be configured.

Method Detail

getClientIdentifierback to summary
private String getClientIdentifier(Host host, Context context, String clientIp)
getClientIpSessionIdback to summary
public Map<String, String> getClientIpSessionId()
getCrawlerIpsback to summary
public String getCrawlerIps()
Returns:String

The current regular expression being used to match IP addresses.

See Also
setCrawlerIps(String)
getCrawlerUserAgentsback to summary
public String getCrawlerUserAgents()
Returns:String

The current regular expression being used to match user agents.

See Also
setCrawlerUserAgents(String)
getSessionInactiveIntervalback to summary
public int getSessionInactiveInterval()
Returns:int

The current timeout in seconds

See Also
setSessionInactiveInterval(int)
initInternalback to summary
protected void initInternal() throws LifecycleException

Overrides org.apache.catalina.valves.ValveBase.initInternal.

Doc from org.apache.catalina.util.LifecycleMBeanBase.initInternal.

Sub-classes wishing to perform additional initialization should override this method, ensuring that super.initInternal() is the first call in the overriding method.

Annotations
@Override
Exceptions
LifecycleException:

Doc from org.apache.catalina.util.LifecycleBase.initInternal.

If the initialisation fails

invokeback to summary
public void invoke(Request request, Response response) throws IOException, ServletException

Implements org.apache.catalina.Valve.invoke.

Doc from org.apache.catalina.Valve.invoke.

Perform request processing as required by this Valve.

An individual Valve MAY perform the following actions, in the specified order:

  • Examine and/or modify the properties of the specified Request and Response.
  • Examine the properties of the specified Request, completely generate the corresponding Response, and return control to the caller.
  • Examine the properties of the specified Request and Response, wrap either or both of these objects to supplement their functionality, and pass them on.
  • If the corresponding Response was not generated (and control was not returned, call the next Valve in the pipeline (if there is one) by executing getNext().invoke().
  • Examine, but not modify, the properties of the resulting Response (which was created by a subsequently invoked Valve or Container).

A Valve MUST NOT do any of the following things:

  • Change request properties that have already been used to direct the flow of processing control for this request (for instance, trying to change the virtual host to which a Request should be sent from a pipeline attached to a Host or Context in the standard implementation).
  • Create a completed Response AND pass this Request and Response on to the next Valve in the pipeline.
  • Consume bytes from the input stream associated with the Request, unless it is completely generating the response, or wrapping the request before passing it on.
  • Modify the HTTP headers included with the Response after the getNext().invoke() method has returned.
  • Perform any actions on the output stream associated with the specified Response after the getNext().invoke() method has returned.
Parameters
request:Request

The servlet request to be processed

response:Response

The servlet response to be created

Annotations
@Override
Exceptions
IOException:
if an input/output error occurs, or is thrown by a subsequently invoked Valve, Filter, or Servlet
ServletException:
if a servlet error occurs, or is thrown by a subsequently invoked Valve, Filter, or Servlet
isContextAwareback to summary
public boolean isContextAware()
isHostAwareback to summary
public boolean isHostAware()
setContextAwareback to summary
public void setContextAware(boolean isContextAware)
setCrawlerIpsback to summary
public void setCrawlerIps(String crawlerIps)

Specify the regular expression (using Pattern) that will be used to identify crawlers based on their IP address. The default is no crawler IPs.

Parameters
crawlerIps:String

The regular expression using Pattern

setCrawlerUserAgentsback to summary
public void setCrawlerUserAgents(String crawlerUserAgents)

Specify the regular expression (using Pattern) that will be used to identify crawlers based in the User-Agent header provided. The default is ".*GoogleBot.*|.*bingbot.*|.*Yahoo! Slurp.*"

Parameters
crawlerUserAgents:String

The regular expression using Pattern

setHostAwareback to summary
public void setHostAware(boolean isHostAware)
setSessionInactiveIntervalback to summary
public void setSessionInactiveInterval(int sessionInactiveInterval)

Specify the session timeout (in seconds) for a crawler's session. This is typically lower than that for a user session. The default is 60 seconds.

Parameters
sessionInactiveInterval:int

The new timeout for crawler sessions

org.apache.catalina.valves back to summary

private Class CrawlerSessionManagerValve.CrawlerHttpSessionBindingListener

extends Object
implements HttpSessionBindingListener, Serializable
Class Inheritance
  • java.lang.Object
  • org.apache.catalina.valves.CrawlerSessionManagerValve.CrawlerHttpSessionBindingListener
All Implemented Interfaces
java.io.Serializable, jakarta.servlet.http.HttpSessionBindingListener, java.util.EventListener

Field Summary

Modifier and TypeField and Description
private final transient String
private final transient Map<String, String>
private static final long

Constructor Summary

AccessConstructor and Description
private
CrawlerHttpSessionBindingListener(Map<String, String> clientIdSessionId, String clientIdentifier)

Method Summary

Modifier and TypeMethod and Description
public void
valueUnbound(HttpSessionBindingEvent
the event that identifies the session
event
)

Overrides default jakarta.servlet.http.HttpSessionBindingListener.valueUnbound.

Notifies the object that it is being unbound from a session and identifies the session.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

clientIdentifierback to summary
private final transient String clientIdentifier
clientIdSessionIdback to summary
private final transient Map<String, String> clientIdSessionId
serialVersionUIDback to summary
private static final long serialVersionUID

Constructor Detail

CrawlerHttpSessionBindingListenerback to summary
private CrawlerHttpSessionBindingListener(Map<String, String> clientIdSessionId, String clientIdentifier)

Method Detail

valueUnboundback to summary
public void valueUnbound(HttpSessionBindingEvent event)

Overrides default jakarta.servlet.http.HttpSessionBindingListener.valueUnbound.

Doc from jakarta.servlet.http.HttpSessionBindingListener.valueUnbound.

Notifies the object that it is being unbound from a session and identifies the session. The default implementation is a NO-OP.

Parameters
event:HttpSessionBindingEvent

the event that identifies the session

Annotations
@Override