Re: [squid-users] Is ignore-private option from refresh_pattern broken?

On 12/09/2013 10:51 p.m., Pavel Kazlenka wrote:
> Hi gentlemen,
>
> I'm trying to cache youtube videos following
> http://wiki.squid-cache.org/Features/StoreID guide.
> But seems like squid rejects to cache the content because original
> server returns header 'Cache-control:private' and 'refresh pattern ...
> ignore-private' doesn't take effect. Here is debug log entries that
> are interesting here (I think):
>
> 2013/09/11 19:24:24.842 kid1| helper.cc(419) helperSubmit:
> buf[975]=http://r16---sn-4g57ln7e.c.youtube.com/videoplayback?ratebypass=yes&itag=43&ip=212.98.189.159&key=yt1&upn=6KCvBVLs-yM&mt=1378980214&fexp=919118%2C924606%2C929117%2C929121%2C929906%2C929907%2C929922%2C929127%2C929129%2C929131%2C929930%2C936403%2C92
> 5726%2C936310%2C925720%2C925722%2C925718%2C925714%2C929917%2C906945%2C929933%2C920302%2C906842%2C913428%2C920605%2C919811%2C935020%2C935021%2C913563%2C919373%
>
> 2C930803%2C908536%2C932211%2C938701%2C931924%2C934005%2C936308%2C909549%2C900816%2C912711%2C904494%2C904497%2C939903%2C900375%2C900382%2C934507%2C907231%2C936
>
> 312%2C906001&id=2c5e89c0af6c8804&expire=1379004381&sver=3&ipbits=8&cp=U0hWTlBLUl9NS0NONl9IRVZEOjNVQko4ZGFaMGcz&ms=au&source=youtube&sparams=cp%2Cid%2Cip%2Cipbits%2Citag%2Cratebypass%2Csource%2Cupn%2Cexpire&mv=m&cpn=Ph7LcSRYt1STlsoQ&signature=BB797D0EFC5182670EF89E95EFBB6E5D12F49B8F.6404F37419A96004F8DDCA2CAB901101A
>
> 30082CA&ptk=youtube_none&pltype=contentugc 192.168.10.8/- - GET
> myip=192.168.10.245 myport=3128
> ...
> 2013/09/11 19:24:24.843 kid1| helper.cc(919) helperHandleRead:
> accumulated[77]=OK
> store-id=http://video-srv.youtube.com.squid.internal/43&2c5e89c0af6c8804"
> 2013/09/11 19:24:24.843 kid1| store_dir.cc(786) get: none of 1
> cache_dirs have 028E8844ECA93A634459175C8C0D463D
> 2013/09/11 19:24:24.843 kid1| store.cc(840) storeCreateEntry:
> storeCreateEntry:
> 'http://video-srv.youtube.com.squid.internal/43&2c5e89c0af6c8804'
> 2013/09/11 19:24:25.032 kid1| http.cc(396) cacheableReply: NO because
> server reply Cache-Control:private
>
> My squid.conf:
>
> acl rewritedoms dstdomain .c.youtube.com
>
> store_id_program /home/tester/squid/libexec/storeid_file_rewrite
> /home/tester/squid/db.txt
> store_id_children 40 startup=10 idle=5 concurrency=0
> store_id_access allow rewritedoms
> store_id_access deny all
> ...
> refresh_pattern ^http://video-srv\.youtube\.squid\.internal/.* 10080
> 80% 79900 override-lastmod override-expire ignore-reload
> ignore-must-revalidate ignore-private
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> #refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
> refresh_pattern . 0 20% 4320
>
> Squid version is squid-3.4.0.1-20130910-r12989.
>
> So my questions are:
>
> 1) Am I right that refresh_pattern ... ignore-private should work here
> and help to cache reply with CC:Private?

Yes the exact code is:
  if ( Cache-Control is present AND contains "private" AND
ignore-private is *absent*)
    display that "NO because server reply Cache-Control:private" message
etc.

> 2) Is there another (may be more correct) way to force squid to cache
> replies with CC:Private?

There is no _correct_ way to abuse the RFC protocol standard. This
ignore-private explicitly forbidden behaviour for any cache shared by
more than 1 person.
It is officially only made available in Squid to allow single-person
installations to operate a proxy between multiple devices.

> 3) How can I check that 'refresh_pattern' config string is parsed
> correctly and all options are remembered by squid?

To check it is parsed correctly use:
   squid -k parse

To check what the running config is you can produce a config file dump
using the cache manager interface:

* ensure that you have a cachemgr_passwd (or "none") explicitly defined
for either the "config" or the "all" reports.
   http://www.squid-cache.org/Doc/config/cachemgr_passwd/

* ensure that your http_access rules involving "manager" ACL permit you
access to the proxy management interface.

* fetch http://example.com/squid-internal-mgr/config in your browser

The result should be a TXT format listing of all the squid.conf settings
(including the defaults) which Squid is using.

Amos
Received on Thu Sep 12 2013 - 11:53:22 MDT