Re: [squid-users] max-age and Last-Modified header

Henrik Nordstrom wrote on 11/28/2006, 3:38 PM:

> What do you want Squid to do in such case? For all what it knows the
> object is dynamically generated on each request with an unique response
> and can not be validated.

Henrik, Thanks for replying. Here's what we are trying to achieve. Your
responses are really appreciated!

    We are seeing if Squid supports ETag based revalidation of highly
dynamic content in a reverse proxy cache configuration.

    We cannot use time based caching (Last-Modified and max-age headers)
because of the "resource representation" -- same resource (page) but
different representation (the HTML). There are two different types of
users. Readers vs Posters. 99% of the users are Readers and will see the
same page but the page is represented differently for the 1% of the
users who are actually Posters.

    We've identified two issues that seem to make Squid unsuitable for this
use.

1. Squid doesnot cache the page if the response does not include a
Last-Modified or a max-age (with a value greater than 0) header.
Note : We can't return Last-Modified: because it tickles IE6 caching
bugs. We can't add a max-age>0 for shared caches because the content
may (1% of the time) be different for different users as described above.

2. Even if we add max-age>0 or Last-Modified, squid does not revalidate
the page with the origin server using ETags (If-None-Match). It seems
to only support If-Modified-Since.

Here's a more detailed scenario:

1. A request to a URL
(http://russia.office.aol.net:3128/cachetest?age=70&resp=200) is made
for the first time via Squid.
---------------------Request headers sent by curl client------------
GET /cachetest?age=70&resp=200 HTTP/1.1
User-Agent: curl/7.10.6 (i386-redhat-linux-gnu) libcurl/7.10.6
OpenSSL/0.9.7a ipv6 zlib/1.1.4
Host: russia:3128
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
-----------------------------------------------

2. Squid doesnot have it in its cache and forwards this request to the
Origin server.
-------------------Request headers received by Origin server------------
header=user-agent=curl/7.10.6 (i386-redhat-linux-gnu) libcurl/7.10.6
OpenSSL/0.9.7a ipv6 zlib/1.1.4
header=host=russia.nscp.aoltw.net:3128
header=accept=image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
header=Via=1.1 russia.nscp.aoltw.net:3128 (squid/2.6.STABLE4)
header=X-Forwarded-For=10.169.155.156
header=Cache-Control=max-age=259200
header=connection=keep-alive
header=content-length=0
header=Etag="1164838155266-ow:c=1"
header=Cache-Control=max-age=70,must-revalidate, proxy-revalidate
header=Content-Length=130
------------------------------------------------------------------------------

3. Origin server responds with 200 OK and includes the following in the
header.
---------------------Response from Origin server------------
HTTP/1.1 200 OK
Date: Wed, 29 Nov 2006 23:01:01 GMT
Server: Apache/2.0.54 (Unix) mod_jk/1.2.14
Etag: "1164841261679-ow:c=1"
Cache-Control: max-age=70,must-revalidate, proxy-revalidate
Content-Length: 130
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/plain
-------------------------------------------------------------

4. Squid caches the response and sends the same to the client.
------------Response from Squid------------------------------
HTTP/1.0 200 OK
Date: Wed, 29 Nov 2006 22:09:15 GMT
Server: Apache/2.0.54 (Unix) mod_jk/1.2.14
ETag: "1164838155266-ow:c=1"
Cache-Control: max-age=70,must-revalidate, proxy-revalidate
Content-Length: 130
Content-Type: text/plain
X-Cache: MISS from russia.nscp.aoltw.net
Via: 1.0 russia.nscp.aoltw.net:3128 (squid/2.6.STABLE4)

Connection: close
------------------------------------------------------------

5. A second request is made for the same URL
(http://russia.office.aol.net:3128/cachetest?age=70&resp=200)
-----------------Second request for the same URL------------
GET /cachetest?age=70&resp=200 HTTP/1.1
User-Agent: curl/7.10.6 (i386-redhat-linux-gnu) libcurl/7.10.6
OpenSSL/0.9.7a ipv6 zlib/1.1.4
Host: russia:3128
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
--------------------------------------------------------------

6. Squid finds this entry in the cache to be stale and sends an
If-Modified-Since header to the Origin server.
-------------------Headers sent to Origin server-----------------
header=If-Modified-Since=Wed, 29 Nov 2006 22:17:14 GMT
header=user-agent=curl/7.10.6 (i386-redhat-linux-gnu) libcurl/7.10.6
OpenSSL/0.9.7a ipv6 zlib/1.1.4
header=host=russia.nscp.aoltw.net:3128
header=accept=image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
header=Via=1.1 russia.nscp.aoltw.net:3128 (squid/2.6.STABLE4)
header=X-Forwarded-For=10.169.155.156
header=Cache-Control=max-age=259200
header=connection=keep-alive
header=content-length=0
header=Etag="1164838718763-ow:c=1"
header=Cache-Control=max-age=70,must-revalidate, proxy-revalidate
header=Content-Length=130
--------------------------------------------------------------------

    As seen, Squid is doing an end-to-end revalidation, but not using
If-None-Match with its cached ETag, which would be much more efficient
in our particular case where the dynamically generated page is expensive
to generate but cheap to validate against and ETag. T
    The semantics we're interested in are the ones supported by Firefox,
IE6, IE7, and the AOL Traffic Server caches. Is this currently supported
in Squid, or would this require additional changes to the source; and if
the latter, is someone already making these changes in a post-2.6 code base?

Thanks

Mohan Rao
Received on Wed Nov 29 2006 - 16:09:43 MST