Re: [squid-users] Cache URL with "?"

>> On tis, 2008-07-22 at 23:37 +1200, Amos Jeffries wrote:
>>
>> Only one issue has come up: under sibling relationships, squids method
>> of
>> relaying requests to sibling is slightly bad and may break the proper
>> expected
>> behavior. All other setups should be fine.
>>
>>
>>> Also seen in parent relations.
>>>
>>> Regards Henrik
>
> Hi,
>
> What is the nature of the problems with dynamic caching and
> sibling/parent relationships? I tried googling for more info but
> couldn't find anything.

In general dynamic pages should be cachable. There are two cases known to
me where that is false.

a) page is dynamic, without cache-control info and meant to be non-cachable.

  This is the reason we add the 0 0% 0 refresh pattern to replace the old
QUERY acl. It means consider all indeterminate dynamic requests as
non-cacheble. Just to be safe.

b) the page is one of the above but has been passed to a peer by squid.

  This is our fault, squid adds an otherwise unnoticeable cache-control
header to requests passed in peering. This breaks the refresh_pattern
safety check for (a).

None yet has coded the patch for (b). So dynamic caching works in squid,
but not on requests received from peers.

>
> I'm using 4 squid nodes [2.7STABLE3] as accelerators and encountered
> some odd behavior recently after enabling dynamic caching. There are
> sibling relationships among the 4 nodes using ICP and cache digests.
> Under medium load I've been able to successfully cache and serve
> dynamic URLs:
>
> GET http://contentreal.foobar.com/clown_widget.swf?version=20080820
> STORE_OK IN_MEMORY SWAPOUT_NONE PING_NONE
> CACHABLE,VALIDATED
>
> But under heavier load [20-50mbit per node] I get multiple objects [20-30]
> like:
>
> GET http://contentreal.foobar.com/clown_widget.swf?version=20080820
> STORE_PENDING NOT_IN_MEMORY SWAPOUT_NONE PING_DONE
> RELEASE_REQUEST,DISPATCHED,PRIVATE,VALIDATED
>
> GET http://contentreal.foobar.com/clown_widget.swf?version=20080820
> STORE_OK NOT_IN_MEMORY SWAPOUT_NONE PING_DONE
> RELEASE_REQUEST,DISPATCHED,PRIVATE,VALIDATED,ABORTED
>
> and requests for this URL timeout after about 25-50% of the download is
> done.
>
> Headers are being set via apache/mod_expires:
>
> [root_at_mongo ~]# curl -I
> http://content5.foobar.com/clown_widget.swf?version=20080820
> HTTP/1.0 200 OK
> Date: Thu, 21 Aug 2008 09:53:50 GMT
> Server: Apache
> Last-Modified: Fri, 08 Aug 2008 06:07:10 GMT
> ETag: "35b0f"
> Accept-Ranges: bytes
> Content-Length: 219919
> Cache-Control: max-age=86400
> Expires: Fri, 22 Aug 2008 09:53:50 GMT
> Content-Type: application/x-shockwave-flash
> X-Cache: MISS from squid03.foobar.com
> X-Cache-Lookup: MISS from squid03.foobar.com:80
> Via: 1.1 squid03.foobar.com:80 (squid)
> Connection: close
>
> I followed http://wiki.squid-cache.org/ConfigExamples/DynamicContent,
> acl QUERY is disabled and the swf is < maximum_object_size_in_memory.
> Any assistance would be greatly appreciated.
>
> Thanks to all who maintain squid, this is great and extremely useful
> software.
>
> murray
>
> ##### squid.conf #####
> http_port 80 defaultsite=contentreal.foobar.com
> cache_peer 110.0.2.99 parent 80 0 no-query no-digest originserver
> cache_peer 110.0.2.76 sibling 80 3130
> cache_peer 110.0.2.77 sibling 80 3130 proxy-only
> cache_peer 110.0.2.78 sibling 80 3130 proxy-only
> digest_generation on
> #hierarchy_stoplist cgi-bin ?
> #acl QUERY urlpath_regex cgi-bin \?
> #cache deny QUERY
> shutdown_lifetime 10 seconds
> strip_query_terms off
> acl apache rep_header Server ^Apache
> broken_vary_encoding allow apache
> cache_mem 5200 MB
> maximum_object_size_in_memory 500 KB
> ipcache_size 4096
> fqdncache_size 4096
> cache_dir null /tmp
> logformat squid %tl.%03tu %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt
> access_log /usr/local/squid/var/logs/access.log squid
> negative_ttl 30 seconds
> cache_access_log /dev/null
> cache_store_log /dev/null
> buffered_logs on
> client_db off
> refresh_pattern html$ 0 20% 1
> refresh_pattern txt$ 0 20% 1
> refresh_pattern xml$ 0 20% 5
> refresh_pattern swf$ 0 20% 5
> refresh_pattern flv$ 0 20% 10
> refresh_pattern . 0 20% 4320
> refresh_pattern (cgi-bin|\?) 0 0% 0
> #refresh_pattern ^ftp: 1440 20% 10080
> #refresh_pattern ^gopher: 1440 0% 1440
> acl N1 src 110.0.2.76
> acl N2 src 110.0.2.77
> acl N3 src 110.0.2.78
> icp_access allow N1
> icp_access allow N2
> icp_access allow N3
> acl all src 0.0.0.0/0.0.0.0
> acl manager proto cache_object
> acl localhost src 127.0.0.1/255.255.255.255
> acl to_localhost dst 127.0.0.0/8
> acl SSL_ports port 443
> acl Safe_ports port 80 # http
> acl CONNECT method CONNECT
> acl purge method PURGE
> http_access allow purge localhost
> http_access deny purge
> http_access allow manager localhost
> http_access deny manager
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_access allow localhost
> http_access allow all
> http_reply_access allow all
> cache_mgr webmaster_at_foobar.com
> cache_effective_user squidmaster
> httpd_suppress_version_string on
> coredump_dir /usr/local/squid/var/cache
>
> ##### config.log #####
> ./configure --enable-storeio=null,aufs,diskd,ufs --enable-cache-digests
>
Received on Fri Aug 22 2008 - 03:11:04 MDT