Re: [squid-users] Squid3 and lots of FIN_WAIT1

David B. wrote:
> Hi Amos,
>
> Amos Jeffries a écrit :
>> On Mon, 16 Nov 2009 16:40:16 +0100, "David B." <haazeloud_at_gmail.com>
>> wrote:
>>
>> [Snip]
>>
>> I have not seen FIN_WAIT1 before, but often see FIN_WAIT like this when
>> Squid is receiving a lot of connections.
>>
>> I think its due to squid using sockets for short times (non-persistent
>> connections) and moving on. The system TCP timeouts are much longer.
>>
> Maybe, it's quite strange.
>
>> [Snip]
>> Maybe yes, maybe no.
>> I assume the LVS works on TCP-link connections so one persistent
>> connection is not broken up. Thats the only breakage that could make this
>> issue worse.
>>
>> Normal LVS would be helping a bit by spreading the load off the problem
>> Squid boxes.
>>
> Sorry i haven't give you all details.
> In fact, i've got an LVS in front, acting as a TCP/IP load balancer, no
> persistent session there, it's a round robin.
>
> Then, TCP request is transmitted to a squid box. This box will answer to
> the client directly (DSR or direct routing mode).
>
> I'm just wondering if activate "session" on the LVS to force someone to
> join the same squid boxe will help.
>>
>>> Perhaps my load is too high and i need to tune kernel via sysctl, but i
>>> can't figure what to do. For now, i've tried several things and i can't
>>> solved this issue.
>>>
>>
>> You may want to check:
>> * persistent connections is turned on (squid.conf)
>>
> On by défault i think, but i will force this to on to test.
>> * system socket timeouts
>>
> System socket or squid related like persistent_request_timeout ?

System socket. I'm guessing the FIN_WAIT* is created once Squid sends
its close signal to the system.

>
>> * half-closed clients is off. (squid.conf)
>>
> We've tried on and off on several boxes, no change for now.
>> Hopefully someone with real finger-time at high performance systems will
>> be able to point at specific tuning knobs which are helpful.
>>
> I hope too.
> I can give the squid conf if needed. :)
>
>>
>>> Here's some squid stats :
>>> HTTP/1.0 200 OK
>>> Server: squid/3.0.STABLE8
>>> Mime-Version: 1.0
>>> Date: Mon, 16 Nov 2009 15:35:35 GMT
>>> Content-Type: text/plain
>>> Expires: Mon, 16 Nov 2009 15:35:35 GMT
>>> Last-Modified: Mon, 16 Nov 2009 15:35:35 GMT
>>> X-Cache: MISS from ww.xx.com
>>> X-Cache-Lookup: MISS from ww.xx.com:80
>>> Connection: close
>>>
>>> Squid Object Cache: Version 3.0.STABLE8
>>> Start Time: Mon, 09 Nov 2009 09:03:56 GMT
>>> Current Time: Mon, 16 Nov 2009 15:35:35 GMT
>>> Connection information for squid:
>>> Number of clients accessing cache: 254750
>>> Number of HTTP requests received: 337655902
>>> Number of ICP messages received: 0
>>> Number of ICP messages sent: 0
>>> Number of queued ICP replies: 0
>>> Number of HTCP messages received: 0
>>> Number of HTCP messages sent: 0
>>> Request failure ratio: 0.00
>>> Average HTTP requests per minute since start: 32244.7
>>>
>>
>> avg 537 req/sec. Nice :)
>>
>> Are you able to provide CPU details sop we can add this to
>> http://wiki.squid-cache.org/KnowledgeBase/Benchmarks ?
>>
> Squid box is a VM under Xen.
> CPU : 2 vcpus (xen), like 2 cores of an intel Xéon E5430 2.66 Ghz
> Memory : 10 Go
> Disks : VM storage on a hardware RAID 5 controller.
> FS for cache : XFS with noatime,nodev,nosuid,noexec
>
> Under peak, CPU is low : (max is 200%, 100% per core)
> idle : 160%
> iowait : 30%
> system & user : 10%
>
> I hope this will help.
>
> David.

Wonderful. Thanks.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE7 or 3.0.STABLE20
   Current Beta Squid 3.1.0.14