Re: [squid-users] bad file caching

Volodymyr Kostyrko wrote:
> I got one file stuck into my cache for two month despite site having
> newer version:
>
> 1219244034.580 3 192.168.8.18 TCP_HIT/200 50599 GET
> http://www.freebsd.org/ports/auditfile.tbz - NONE/-
> application/x-bzip-compressed-tar
>
> > fetch -vvvvv http://www.freebsd.org/ports/auditfile.tbz
> scheme: [http]
> user: []
> password: []
> host: [www.freebsd.org]
> port: [0]
> document: [/ports/auditfile.tbz]
> ---> www.freebsd.org:80
> looking up www.freebsd.org
> connecting to www.freebsd.org:80
> requesting http://www.freebsd.org/ports/auditfile.tbz
> >>> GET /ports/auditfile.tbz HTTP/1.1
> >>> Host: www.freebsd.org
> >>> User-Agent: fetch libfetch/2.0
> >>> Connection: close
> >>>
> <<< HTTP/1.0 200 OK
> <<< Content-Type: application/x-bzip-compressed-tar
> <<< Accept-Ranges: bytes
> <<< ETag: "-943519291"
> <<< Last-Modified: Wed, 04 Jun 2008 14:10:03 GMT
> last modified: [2008-06-04 14:10:03]
> <<< Content-Length: 50198
> content length: [50198]
> <<< Date: Wed, 04 Jun 2008 14:25:15 GMT
> <<< Server: httpd/1.4.x LaHonda
> <<< Age: 16663
> <<< X-Cache: HIT from utwig.xim.bz
> <<< X-Cache-Lookup: HIT from utwig.xim.bz:3128
> <<< Via: 1.0 utwig.xim.bz (squid/3.0.STABLE8)
> <<< Proxy-Connection: close
> <<<
> offset 0, length -1, size -1, clength 50198
> local size / mtime: 50198 / 1212588603
> remote size / mtime: 50198 / 1212588603
> auditfile.tbz 100% of 49 kB 21 MBps

Mmmm. No expiry information, so Squid has to do a best-effort
approach. Let's learn about refresh patterns
(http://www.squid-cache.org/Versions/v3/3.0/cfgman/refresh_pattern.html).
The default for Squid includes...

refresh_pattern . 0 20% 4320

...which would match the request mentioned. So what does this mean? In
the absence of expiry information, Squid should use the age (gathered
from the Last Modified date) to infer how long the object will be
fresh. The first number specifies there should be no lower limit on the
freshness of the object. If the freshness calculation concludes that
the object is only fresh for 30 seconds, so be it. The last number
states that the maximum object freshness is 4320 minutes (3 days). Even
if the freshness calculation states the object could be fresh for
another year, we'll verify freshness every 3 days. The middle number is
where the calculation comes in. The cached object was last modified on
June 4, 2008 at 14:10:03 GMT. As the object gets older, it's assumed
that it is less likely to change (we are predicting the future based on
past performance), so after an hour of no changes, we assume that the
object is not going to change in the next 12 minutes (60 * 20%). After
a day of no changes, we assume the object will not change for around 5
hours. At 15 days of no changes we hit the ceiling on freshness (15 *
20%) and our freshness calculation becomes superfluous.

Obviously the object has changed, so you have a few options:
* Use the PURGE method with squidclient
* force a refresh with your browser (hold down shift or control when you
press the refresh or reload button, use the -r switch with squidclient
or --cache=off for wget. fetch does not appear to have a method of
forcing a refresh.)
* Add a cache deny for this domain
* Wait for the freshness calculation to expire (3 days at the most)

>
> > fetch -vvvvv http://www.freebsd.org/ports/auditfile.tbz
> scheme: [http]
> user: []
> password: []
> host: [www.freebsd.org]
> port: [0]
> document: [/ports/auditfile.tbz]
> ---> www.freebsd.org:80
> looking up www.freebsd.org
> connecting to www.freebsd.org:80
> requesting http://www.freebsd.org/ports/auditfile.tbz
> >>> GET /ports/auditfile.tbz HTTP/1.1
> >>> Host: www.freebsd.org
> >>> User-Agent: fetch libfetch/2.0
> >>> Connection: close
> >>>
> <<< HTTP/1.1 200 OK
> <<< Connection: close
> <<< Content-Type: application/x-bzip-compressed-tar
> <<< Accept-Ranges: bytes
> <<< ETag: "-1888419386"
> <<< Last-Modified: Wed, 20 Aug 2008 14:40:01 GMT
> last modified: [2008-08-20 14:40:01]
> <<< Content-Length: 51141
> content length: [51141]
> <<< Date: Wed, 20 Aug 2008 14:56:35 GMT
> <<< Server: httpd/1.4.x LaHonda
> <<<
> offset 0, length -1, size -1, clength 51141
> remote size / mtime: 51141 / 1219243201
> auditfile.tbz 100% of 49 kB 28 kBps
>
> config:
> cache_replacement_policy heap LFUDA
> memory_replacement_policy heap GDSF
> refresh_pattern ^ftp: 1440 20% 1440 refresh-ims
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern . 0 20% 4320
>

Chris
Received on Wed Aug 20 2008 - 19:29:18 MDT