Re: [squid-users] YouTube and other streaming media (caching)

The problem with caching Youtube (and other CDN content) is that
the same content is found at lots of different URLs/hosts. This
unfortunately means you'll end up caching multiple copies of the
same content and (almost!) never see hits.

Squid-2.7 -should- be quite stable. I'd suggest just running it from
source. Hopefully Henrik will find some spare time to roll 2.6.STABLE19
and 2.7.STABLE1 soon so 2.7 will appear in distributions.

Adrian

On Wed, Apr 16, 2008, Ray Van Dolson wrote:
> Hello all, I'm beginning to implement a Squid setup and am in
> particular looking to cache Youtube as it is a significant chunk of our
> traffic and we don't want to outright block it (yet).
>
> I'm using squid-2.6.STABLE6 from RHEL 5.1 (latest errata). I've been
> reading around a lot and am seeking a bit of clarification on the
> current status of caching youtue and potentially other streaming media.
> Specifically:
>
> * Adrian mentions support for Youtube caching in 2.7 -- which seems
> to correspond with this changeset:
>
> http://www.squid-cache.org/Versions/v2/2.7/changesets/11905.patch
>
> Which would seem to be only a configuration file change. Is there
> any reason Youtube caching won't work correctly in my 2.6 version
> with a similar setup (and the rewriting script as well I guess)?
>
> * If there are additional changes to 2.7 codebase that make youtube
> caching possible, are they insignificant enough that they could
> easily be backported to 2.6? I'm trying to decide how I will
> convince Red Hat to incorporate this as I doubt they'll want to
> move to 2.7. Alternate of course is to build from source which I
> am open to.
>
> My config file is as follows:
>
> http_port 3128
> append_domain .esri.com
> acl apache rep_header Server ^Apache
> broken_vary_encoding allow apache
> maximum_object_size 4194240 KB
> maximum_object_size_in_memory 1024 KB
> access_log /var/log/squid/access.log squid
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern . 0 20% 4320
>
> acl all src 0.0.0.0/0.0.0.0
> acl esri src 10.0.0.0/255.0.0.0
> acl manager proto cache_object
> acl localhost src 127.0.0.1/255.255.255.255
> acl to_localhost dst 127.0.0.0/8
> acl SSL_ports port 443
> acl Safe_ports port 80 # http
> acl Safe_ports port 21 # ftp
> acl Safe_ports port 443 # https
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 1025-65535 # unregistered ports
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
> acl CONNECT method CONNECT
> # Some Youtube ACL's
> acl youtube dstdomain .youtube.com .googlevideo.com .video.google.com .video.google.com.au
> acl youtubeip dst 74.125.15.0/24 64.15.0.0/16
> cache allow youtube
> cache allow youtubeip
> cache allow esri
>
> http_access allow manager localhost
> http_access deny manager
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_access allow localhost
> http_access allow esri
> http_access deny all
> http_reply_access allow all
> icp_access allow all
> coredump_dir /var/spool/squid
>
> # YouTube options.
> refresh_pattern -i \.flv$ 10080 90% 999999 ignore-no-cache override-expire ignore-private
> quick_abort_min -1 KB
>
> # This will block other streaming media. Maybe we don't want this, but using
> # it for now.
> hierarchy_stoplist cgi-bin ?
> acl QUERY urlpath_regex cgi-bin \?
> cache deny QUERY
>
> I see logfile entries (and cached objects) that indicate my youtube
> videos are being saved to disk. However they are never "HIT" even when
> the same server is used. I wonder if the refresh_pattern needs to be
> updated? The GET requests for the video do not have a .flv in their
> filename.... What does refresh_pattern search for a match? The request
> URL? The resulting MIME type?
>
> That's it for now. :) Thanks in advance.
>
> Ray

-- 
- Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support -
- $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -