RE: [squid-users] CPU saturation?

> -----Original Message-----
> From: James Vanns [mailto:jimv@canterbury.ac.uk]
> Sent: Wednesday, November 16, 2005 3:49 AM
> To: squid-users@squid-cache.org
> Cc: Paul Mills (paulm@canterbury.ac.uk); Benjamin Tanner
> Subject: Re: [squid-users] CPU saturation?
>
>
> On Wed, 2005-11-16 at 12:41 +0000, James Vanns wrote:
> > <snip>
> > > > With approximately 3000 open file descriptors
> (configured at compile
> > > > time and set at runtime with ulimit -HSn 16384) and an
> equivalent no. of
> > > > client/server TCP connections we see a constant CPU
> usage of ~95-100%.
> > > > Is this normal!? The hardware is as follows:
> > >
> > > I don't think so. Try strace and ltrace on running squid.
> >
> > Yeah I have actually run squid through strace and the only alarming
> > thing is the huge amount of bind() calls on 0.0.0.0 and
> port htons (0).
> > And by huge amount I mean practically every lookup (I guess
> somehow this
> > is related to the internal DNS lookups).
>
> Sorry I should have said 'squid's internal resolver' to avoid
> confusion ;)
>
> > Surely you don't need to call
> > bind() for every query!? Or am I missing something here?
> Admittedly I
> > didn't spend much time analysing the strace output.
> >
> > > > Dual Intel(R) Xeon(TM) CPU 3.20GHz (Linux of course
> manages the CPU
> > > > affinity as we know squid isn't multi-threaded)
> > > > 6G of memory
> > >
> > > IIUC one squid will load only one CPU, how do you get
> "95-100%" load
> > > on both?
> >
> > You misunderstand (I think). That percentage I gave was per CPU e.g.
> > 95-100% usage on CPU0 not across all (0-3) processors.
> >
> > Jim
> >
> > > --
> > > vda
> > --
>
>

I know that it's not officially supported, I don't know how hard it would be to patch 2.5STABLE3, but the epoll patch (http://devel.squid-cache.org/projects.html#epoll) did absolute wonders for my CPU utilization on Linux (from about 75% average down to less than 20% peak). For what it's worth, here are some statistics from someone who took the time to do some comparison testing:

http://www.squid-cache.org/mail-archive/squid-users/200504/0422.html

Henrik Nordstrom (one of the Squid Devs) has stated that epoll support might "reduce CPU usage when there is many open filedescriptors" (http://www.squid-cache.org/mail-archive/squid-users/200509/0244.html).

It's been stable as a rock for me for the time I've been using it (~24 days). Vital stats: about 100 req/s peak, 1500 kBytes/s peak using 2.5STABLE11 on Centos 4.1. For me this works out to about 500 open file descriptors. There are two squid instances on this box (to take advantage of dual CPUs - Intel Xeon 3.00GHz), and the stated stats are per-instance.

On another note, with null caching, and multiple CPUs running more than one instance of squid becomes a fair bit easier. See http://squidwiki.kinkie.it/squidwiki/MultipleInstances for more details.

Chris
Received on Wed Nov 16 2005 - 17:07:47 MST