Re: [squid-users] AVG Updates not being cached with squid 2.6?

Richard Chapman wrote:
> Amos Jeffries wrote:
>> Richard Chapman wrote:
>>> Amos Jeffries wrote:
>>>> Richard Chapman wrote:
>>>>> I have a more or less default configured squid 2.6 proxy on a
>>>>> centos 5.4 server.
>>>>> I have configured AVG 9 network edition (Virus scanner) to use the
>>>>> squid proxy (as opposed to the avg proxy) - and it appears to be
>>>>> doing so.
>>>>> However - checking the usage logs - it appears that different
>>>>> client machines download identical update (.bin) files within a
>>>>> few hours of each other - but do not appear to get a cache hit..
>>>>>
>>>>> Can anyone suggest why these update files are not being cached (or
>>>>> at least not getting cache hits) - and whether there is anything I
>>>>> can do to encourage them to be cached?
>>>>>
>>>>> I have checked the Squid FAQ and searched the archive - and found
>>>>> a similar request from 2005. The suggestion there was that the AVG
>>>>> server might be using the
>>>>>
>>>>> "Pragma: no-cache" HTTP header
>>>>
>>>> To be sure take the URL that should be a HIT and enter it at
>>>> redbot.org.
>>>> The whole problems should be easily visible there.
>>>>
>>>>>
>>>>> And that at that time there was no suggestion on how to override
>>>>> this. Can anyone confirm that this is the reason for the
>>>>> apparently unnecessary cache misses - and if so - is there
>>>>> anything new in squid to allow us to override?
>>>>>
>>>>
>>>> Squid which do not ignore "Pragma: no-cache" treat it the same as
>>>> "Cache-Control: no-cache"
>>>>
>>>> Amos
>>> Thanks Amos
>>> I tried redbot as you suggested - and this is a url which I think
>>> SHOULD have been a hit - though it is hard to be sure. The stats
>>> show that NONE of the avg updates come from cache - and I assume
>>> they should all have similar headers... Hopefully someone more
>>> knowledgeable than I can make more sense of this;
>>>
>>> http://redbot.org/?uri=http://aa.avg.com/softw/90/update/u7iavi2551u2550qp.bin
>>>
>>>
>>>
>>> It looks to me that it should be cacheable - but the only suspicious
>>> thing is the statement I get when I hover over the "This response is
>>> stale". I think it says that it has a "Freshnes lifetime of 0" -
>>> which sounds like it will always be considered stale. I'm not sure
>>> why they would do this as each update has a unique file name - and
>>> could therefore be considered fresh indefinitely couldn't it?
I never checked redbot, but upon getting similar end results while
trying to cache updates for other av programs (eg. Malwarebytes
Antimalware) I ended up doing the following:

refresh_pattern http://mbam-cdn.malwarebytes.org/.* 1440 100% 1441
ignore-reload override-lastmod override-expire

Without that, I always got TCP_REFRESH_MISS. What this line does is
force squid to consider the cached copy fresh for 24 hours (1440
minutes). With the malwarebytes updates, the filename is always the
same. In order to actually get updates, I had to let it re-download
every day. You could adjust the 1440 and 1441 to much higher numbers if
every AVG update file has it's own unique file name.

It seems like these AV programs do a lot to try make sure things aren't
getting cached by mistake, but it looks like your AVG setup is supposed
to allow caching. I'm not sure what they're trying to do here. They
may just be trying to make you use their cache. ;)

Just a note: the above line is one I'm using in squid 2.7 and 3.x. I
don't know if the syntax of refresh_pattern was exactly the same in 2.6
You may want to check.
>>>
>>> Can anyone confirm my interpretation - and/or suggest a way to treat
>>> the updates more rationally?
>>>
>>> Richard.
>>>
>>>
>>>
>>> A cache considers a HTTP response stale when its age (here, 0) is
>>> equal to or exceeds its freshness lifetime (in this case, 0)
>>>
>>> A A cache considers a HTTP response stale when its age (here, 0) is
>>> equal to or exceeds its freshness lifetime (in this case, 0).cache
>>> considers a HTTP response stale when its age (here, 0) is equal to
>>> or exceeds its freshness lifetime (in this case, 0).
>>
>> Hmm, something strange there.
>>
>> AFAIK the object looks like with the L-M header + the Date should
>> have both non-zero freshness (Date - LM) and an age (now - Date).
>>
>> Amos
>
> Thanks Amos
> Can you shed ANY light on what might be going on here? I presume you
> are seeing the same "odd" freshness and age numbers as I am.
> Can you enlighten me on what "LM" means or stands for?
> Are you suggesting that the AVG website is doing something odd to
> concoct the strange age and freshness numbers?
> Where could an inconsistency arise to give zero for both these numbers
> - as seen by redbot?
>
> I am very new to this stuff - and need help interpreting the
> data...:-) This all started from the observation that all AVG updates
> seem to be cache misses - even when the same update file is downloaded
> several times in a few hours.
>
>
> Richard.
>
>
>
>
>
>
Received on Tue Dec 08 2009 - 15:40:43 MST