Re: [squid-users] Re: squid 3.2.0.14 with TPROXY => commBind: Cannot bind socket FD 773 to xxx.xxx.xxx.xx: (98) Address

On 25/08/2013 3:12 a.m., Niki Gorchilov wrote:
> Hi, Amos,
>
> I'm working on the same project with Plamen.
>
>>> squidclient mgr:info |grep HTTP
>>> HTTP/1.1 200 OK
>>> Number of HTTP requests received: 1454792
>>> Average HTTP requests per minute since start: 116719.5
>>
>> Nice. With stats like these would you mind supplying the data necessary for
>> an entry in this page?
>> http://wiki.squid-cache.org/KnowledgeBase/Benchmarks
>> (see section 2 for how to calculate the datum).
> The moment we manage to fix this issue and are able to run squid for
> more than few minutes without commBind problem, I promise to submit
> benchmarks for two times bigger setup. Just we have to iron out this
> issue. :-)
>
>> Unfortunately all I can do is point you at the known reasons for the
>> message.
>> The things to figure out is whether there is some limit in TPROXY kernel
>> code itself (the socket match module is the critical point I think) about
>> how many sockets it can manage. Or if some of the traffic is coming an
>> excessive amounts from any particular IPs and reducing the amount of
>> outgoing connections that can be used for it.
> Before digging deeper into the TPROXY kernel code, I'd like to clarify
> one aspect of squid's behaviour. Do you pass a port number (anything >
> 0) in inaddr.ai_addr during the bind call? Sorry, I couldn't trace it
> myself, as I didn't do much C/C++ programming since early 90's :-)
>
> Is it Squid or the kernel who decides what port to be used?

We pass the destination port:IP to connect() and then try to bind() to
the client IP on port 0 for source. The kernel decides which port is
available, then we retrieve its decision with getsockname().

> I believe the kernel will return EADDRNOTAVAIL in case of exhausted
> ports for the specific IP. And the commBind errors will cite one and
> the same IP, which is not the case. All random IPs are there in the
> log. Very few IP's has more (100-200) error log lines. Most IPs will
> be mentioned just 1,2,3 times.
>
> EADDRINUSE error is a clear indication that this same IP:port pair is
> already in use. Or someone else listens to 0.0.0.0:<same_port>.
> It'll be of great help if we manage to log the port number together
> with the address in order too look for possible collisions with other
> processes running on the machine (incl all other squid workers).
>
> Thank you in advance for your support!

You can add a debugs line on src/comm.cc where it call commBind() from
comm_apply_flags() to display the "addr" variable. The if-statements
above it have some examples.

However the existing log line shodul be dumping out the full IP:port
details when a port >0 is used, so you should be seeing a port there if
a port was sent to bind(). Your obfuscation indicates only an IPv4 was used.

Amos
Received on Sun Aug 25 2013 - 01:50:51 MDT