HTTP Headers And Caching

Caching is one of the lesser understood concepts of HTTP. So this article is going to attempt to clarify some of the confusing concepts of web caching in order for you to improve your site's performance. If you set up your site to be cached correctly, your site will appear to display faster and you will reduce the load on your server. This is achieved by reducing the number of requests and responses sent to and from the server.

A web cache sits between the web server and a client in the request/response chain. It listens to and saves requests made by clients. When the cache sees a request for an object that it has saved, it will attempt to validate the object, then send back either it's version or the one on the server. In general, if the cache determines that the requested object is still fresh (i.e. the copy it has is up to date), it will send back the copy it has. Otherwise, the cache gets a new copy from the origin server and sends it back to client.

In order for the cache to determine when to send back it's copy and when to pass the request along to the origin server, rules - or caching policies - are followed. Some general rules are:

• If the object's headers say not to keep the object, then the cache shouldn't keep the object.
• If the object does not have a validator, most caches will mark the object as uncacheable.
• If the object is authenticated or secure, the cache won't keep the object.
• The object will be considered fresh if:
  1. It the object's Expires and/or other age-controlling header directive is set, and it is still within the fresh period.
  2. If a browser cache has already seen the object, and has been set to check for freshness once a session.
  3. If a proxy cache has seen the object recently, and it was modified relatively long ago.
• If an object is stale, the origin server will be asked to validate the object (tell the cache if the copy it has is still good).

Freshness and validation are the core concepts of how a cache works. If an object is deemed fresh, it will be instantly available from the cache. An object that is stale, but validated, will avoid sending the entire object again.

A cache doesn't only use the Expires header to determine object freshness. The Cache-Control header also helps to define object cachability. Some of the Cache-Control header values are:

• no-cache - a cache must not return a cached copy without successful revalidation with the origin server.
• no-store - This directive is the best way to make an object uncachable, because it prevents even temporary storage of the object.
• no-transform - certain caches will attempt to change, or transform, objects (like compress text). This directive prevents caches from doing so.
• public - makes normally uncachable objects cachable by both browser caches and proxy (shared) caches.
• private - allows browser caches to cache normally uncachable objects, however, for proxy (shared) caches, normally cachable objects become uncachable.
• max-age=[seconds] - specifies the maximum amount of time that an object will be considered fresh, where [seconds] is the number of seconds after the time of the request that the object will be fresh.
• s-maxage=[seconds] - similar to max-age, except that it only applies to proxy (shared) caches.
• must-revalidate - The HTTP spec allows for intentional return of stale responses in certain cases (like poor network connectivity). This directive is intended to prevent this from happening.
• proxy-revalidate - similar to must-revalidate, except that it only applies to proxy (shared) caches.

Related Links
W3C RFC 2616 Section on Caching
CacheRight is a great product for IIS to help administer caching policies.
Apache Servers have many modules for controlling cache-related headers.