Re: [squid-users] Load Balance Authenticators

On 28/05/11 02:11, unjc email wrote:
> Amos / Kinkie, thanks for your replies.
>
> From what I see, I don’t see how increasing the number of helpers
> could help resolving the problem as I see there are even idle helpers
> at the time when the squid crashes. Actually I am doing a capacity
> test on our local squid box, I am curious to know how I can avoid the
> crash in order to get a consistent result. I ran the same ramp test
> twice, but the crashes occurred at different times. Any advice or
> suggestion will be appreciated.

Capacity testing?

  Um, well.. what Squid is trying to say by "Too many queued **
requests". Is that you have hit the capacity for those helpers. To reach
a higher parallel/concurrent request rate you need to increase the
helper numbers.

At the point Squid reports FATAL there are >200 brand new TCP
connections waiting to be authenticated against just 40 helpers
*simultaneously*. The queue limit being 5x the number of helpers available.

I suspect your testing involves a step opening something like 500
concurrent TCP links. Doing so in under one second Squid will hit the
limit and fail immediately.

Negotiate/Kerberos has no helper limit, so bump it up as high as you
want. Then re-test.

> On Fri, May 27, 2011 at 5:04 AM, Amos Jeffries wrote:
>> On 27/05/11 08:54, unjc email wrote:
>>>
>>> Hello there,
>>>
>>> I encounter a performance issue. I found that the squid proxy crashes
>>> occasionally due to "Too many queued negotiateauthenticator requests".
>>> When I monitor the server by querying "squidclient
>>> mgr:negotiateauthenticator", I discovered that not all authenticators
>>> are busy at the same time. It seems like only the authenticators on
>>> the top of the list are busy with queued requests; but many of others
>>> down at the bottom of the list are basically idle. It looks like a
>>> load-balancing problem among the authenticators. Please advise if
>>> there is any setting for load-balancing authenticators in squid.conf.
>>
>> Yes. There is no point "balancing" helpers. Each is pinned 1:1 with a client
>> login for the entire time it takes to operate. When the first is pinned the
>> seconds gets used, etc. Once the first helper is released, it can
>> immediately be used for another client without swapping it in/out of memory
>> just to reach one that has not been used yet.
>>
>>> cache.log:
>>>
>>> 2011/05/25 03:04:44| WARNING: up to 199 pending requests queued
>>> 2011/05/25 03:04:44| Consider increasing the number of
>>> negotiateauthenticator processes to at least 239 in your config file.
>>> 2011/05/25 03:05:01| storeDirWriteCleanLogs: Starting...
>>> 2011/05/25 03:05:01| Finished. Wrote 0 entries.
>>> 2011/05/25 03:05:01| Took 0.0 seconds ( 0.0 entries/sec).
>>> FATAL: Too many queued negotiateauthenticator requests (201 on 40)
>>> Squid Cache (Version 2.7.STABLE4): Terminated abnormally.
>>>
>>
>> Upgrade to 2.7.STABLE9. There are several negotiate and helper fixes in that
>> version. Maybe an underlying problem is resolved already.
>>
>> If it continues there, you could try following Squids advice which was
>> displayed at your peak load:
>>
>>> 2011/05/25 03:04:44| Consider increasing the number of
>>> negotiateauthenticator processes to at least 239 in your config file.
>>
>>
>> Amos
>> --
>> Please be using
>> Current Stable Squid 2.7.STABLE9 or 3.1.12
>> Beta testers wanted for 3.2.0.7 and 3.1.12.1
>>

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.12
   Beta testers wanted for 3.2.0.7 and 3.1.12.1