Tuning keep-alive connections
As mentioned in our introduction, keep-alive connections
is the name for HTTP connections kept open between requests.
In principle, keeping connections open between requests saves resources on the server
and can make web pages more responsive. Each connection needs resources on the server,
plus incurs a time delay or latency in setting up the connection.
With keep-alive connections a notable improvement comes where, for example, a web page has various embedded images.
But they do lead to a server tuning issue.
Every time a server accepts a connection from a client, the connection takes up
resources on the server. But what resources it takes up when is quite a subtle issue.
What resources do connections use... and when?
Connections use at least the following resources on the server. As mentioned, crucially,
they use different resources at different stages of their "life cycle":
- Setting up the connection in the first place inevitably uses network resources for the "handshake"
plus some CPU time for the server to prepare the connection (e.g. memory allocation, initiating or
allocating an available thread to administer the connection...),
although this is admittedly much more significant in the case of HTTPS.
- Each currently open connection will generally be
served by a separate thread (or worse, a separate process on
some servers) and consume some amount of memory space.1
- If you're using a separate web server and Servlet runner
(such as Apache with Tomcat), then each remote connection will actually use an additional— and
possibly two— local connections (as the web server and
Servlet runner talk to each other over a socket).
- Subtly, even closed connections continue to "hang around" and consume resources
for a period of about 1 to 4 minutes after they are closed.2
Because each connection, both alive and dead, takes up certain resources, part of
tuning a server is to make connections keep alive for the right amount of
time. At one extreme, if every request needed a separate connection, then we burn network and CPU time
on the 'administrative' task of setting up connections, plus end
up with a large number of connections sitting in the TIME_WAIT state.
At the other extreme,
if connections are held open for too long, then we'll have more simultaneously open
connections and hence more threads to handle them.
Keep-alive tuning parameters
Now, which of the various resources is more limited depends on the server environment.
- If you have the luxury of a dedicated server, then you'll probably be
able to service hundreds of simultaneous connections, and having even tens of thousands of
connections sitting in the TIME_WAIT state may not be a problem;
- If you have a virtual server or VPS3,
then there may well be constraints on the number of processes, threads
and connections (either alive or in the TIME_WAIT state) that you are allowed to have
going. And if not, you will certainly have a constraint on available memory, which will
limit all of these resources to some extent or other.
You may need to consult with your hosting company to find out what their policy is.
1. Of course, while data is actually being transferred, resources will be used: CPU time will be used transferring the
data from kernel space to memory space and vice versa; the Servlet itself will presumably want to do something useful
such as composing the page to send back. But this resource usage is the bit that's "fair game".
We want to optimise the "wasted" or "administrative" resource usage that happens around the essential
part of processing the actual web content.
2. This is the TCP TIME_WAIT delay. During a period after a connection is closed— typically
between 1 and 4 minutes depending on the OS— the "dead" connection sits in the TIME_WAIT state.
In this state, the connection should not be hogging a thread, but it may well be hogging other resources
consumed by TCP connections within the OS, such as memory space. If your environment has some upper
limit on the number of TCP connections, the TIME_WAIT connection may well count towards this limit.
3. A virtual (private) server or VPS is where a single physical server is divided
into several "virtual" servers that to the naked eye behave almost as though they were
separate servers. This model of hosting is apparently becoming more and more common, as
space in data centres becomes more scarce, and servers are powerful enough that dedicating a single
server to many types of application or web site is actually a waste of resources.