Re: [squid-users] cache hierarchy question

On sön, 2008-08-10 at 20:25 -0700, Rob Williams wrote:

> HTCP, Cache Digest, whatever), do I need all requests to my array /
> mesh to come to a 'master' squid acting as a router/load balancer? Or
> do I put a load balancer in front of the array and distribute http
> requests randomly to the squids in the array?

I would use a load balancer..

> The problem I'm having is if I put a load balancer in front of my
> squid array then each squid must be aware of all other peers in order
> for the squid array to act as a large cache.

Yes..

> This results in
> forwarding loops and other problems when using ICP and carp that I've
> been unable to get around so far.

There is no need for ICP when using CARP.

You should not see forwarding loops in a correct CARP setup.

In a crossed sibling relation you may see forwarding loops, and the
workaround is to use cache_peer_access to deny siblings to be used if
the request was received from a sibling (any sibling),

There is no intra-array CARP routing in Squid, only CARP forwarding. A
quite efficient CARP setup looks like:

1. A simple load balancer distributing the connections among the caches.
Or even DNS round-robin if you like (the load balancer mainly makes it
easier if you need to take one server down for service or if there is a
problem..)

2. A CARP squid layer, with no disk cache but a reasonable cache_mem.
This receives requests from the load balancer and acts as hot object
cache and distributed CARP HTTP load balancer.

3. A caching Squid layer with the disk cache, used as a parent CARP
cache_peer relation by the CARP squids. These have the origin servers as
parents.

2 & 3 runs as two Squid instances on the same servers, with each server
having a CARP squid listening on the load balanced port and a caching
squid listening on another port only used for the CARP forwarding (or
another IP if you prefer..)

On all Squids a relatively short connect_timeout is set.

ICP is not used. (no-query on all cache_peer)

On multi-core servers >2 cores there may be benefit to run more than one
CARP instance per server.

> instance as it assumes a linear array instead of a hierarchy, but at
> this point I'd be happy to see ANY cache protocol example that would
> distribute the cached objects in a squid cluster.

The simplest setup is ICP + proxy-only. But it's not as efficient as
CARP and will result in noticeably increased duplication of content over
time.

Regards
Henrik