A solution for problems seen when listening on specific IP address

[If anyone else who has seen the problems described below tries my suggested
solution, I'd be interested in confirmation that it works. It seems to
work for me, and looks like it's the right solution, but maybe there are
subtleties that I've missed!]

Background:

A few weeks ago, I asked about how to configure Squid to listen (only) on a
specific IP address, of several available on the system. I specifically
didn't want user requests to be accepted on other addresses as they'd then
complain if Squid was moved to a different system and access failed because
they'd been using "the wrong name" (one of the system's other addresses).

The problem was that the default squid.conf includes comments that
udp_incoming_address and udp_outgoing_address cannot be the same - which
seemed "unhelpful". The tcp addresses can be defined to be the same.

I received the suggestion that I should configure udp_outgoing_address
explicitly (that *must* be the address that UCP peers believe they are
using, so responses are considered valid) and let the incoming listener
default to all available addresses. It worked, but with some problems.
Judging by responses when I queried the problems, other people had used a
similar configuration and were seeing the same problems, but no-one offered
any solutions. I believe that I've now found the solution...

The problems, seen only when configured for a specific address:

(1) "icpHandleUdp: FD 16 recvfrom: (11) Resource temporarily unavailable"
in cache.log during shutdown and after HUP (the named FD not being either of
the TCP or ICP FDs mentioned in "Closing xxx connection" messages).
Apparently harmless (but see 2), though worrying.

(2) During shutdown or after HUP, Squid *always* waited the full (default)
30 second shutdown_lifetime timeout before completing shutdown or
reconfiguration, even when it was totally idle. With hindsight, this is
probably another symptom of the underlying cause of problem (1).

(3) Sending Squid a HUP signal to reconfigure it mostly (or maybe always)
caused it to respond

ftpget server failing too rapidly
WARNING: FTP access is disabled!

The solution:
-------------

While looking around the source code for an explanation for the icpHandleUdp
message, I found what appears to be the solution.

In summary: define udp_incoming_address as required, and leave
udp_outgoing_address undefined. Squid then automatically uses the incoming
address (same socket, in fact) for outgoing UDP. And the problems vanish...
(for me, so far).

In source file main.c, the function serverConnectionsOpen deals with opening
both HTTP and ICP (tcp and udp) sockets, taking into consideration any
specific addresses defined in the configuration file. It sets up the
incoming udp (ICP) listener socket first, then the outgoing udp socket.

The interesting bits (line numbers for Squid 1.NOVM.16's main.c) are at
line 388:

            if ((addr = Config.Addrs.udp_outgoing).s_addr != no_addr.s_addr) {

and lines 407-409:

            } else {
                theOutIcpConnection = theInIcpConnection;
            }

So, Squid checks if the configured outgoing UDP address was defined
explicitly, and if so then tries to set it up. If no outgoing address
was configured, lines 407-9 ensure that it will automatically be configured
to use the same address(es) - could be all by default, or a specific one -
as for incoming UDP!!! Just what we wanted all along, but as far as I'm
aware, totally undocumented - and certainly not hinted at in the sample
configuration file, which says only

# The defaults behaviour is to not bind to any specific address.
#
# NOTE, udp_incoming_address and udp_outgoing_address can not have
# the same value since they both use port 3130.

Those comments are both misleading and bizarre.

Misleading because they can *use* the same address, simply by defining
udp_incoming_address and leaving udp_outgoing_address undefined - but it
implies they cannot be the same, and certainly does not say that setting an
incoming addresses implicitly sets the outgoing address!

Bizarre in the sense that the only reason it doesn't "just work" if you
define both incoming and outgoing UDP with the same address is that Squid
doesn't bother checking for that case. I haven't yet tried doing it, but
it looks like it should have taken little more time to add the check
for incoming and outgoing addresses being the same (and handle it like the
incoming defined, outgoing undefined case) than to add the comments in the
configuration file implying it was impossible! [Though maybe it *was*
impossible at one time, the code changed later, and the comments didn't get
updated...]

I would suggest that the least-confusion option would be

 * add check and handle identical definitions of incoming and outgoing
   UDP address properly
 * replace the existing NOTE in the sample config with a note that
   configuring them to be identical does work (for the benefit of people
   who've seen the old warning and might assume it still applied if not
   mentioned) and also note that outgoing UDP defaults to the same
   address(es) as incoming UDP - it needs to, for things to work, but
   someone might expect it to behave differently (using all addresses
   outgoing by default even though incoming address defined).

The least-effort option would be to amend the comments in the sample
configuration file to indicate that the right way to get incoming and
outgoing UDP on the same explicit address is to define the incoming
address and leave the outgoing address unspecified, when it will default
to using the same address (and indeed, same socket) as for incoming UDP.

As for the problems that result from doing it the other way around,
defining the outgoing UDP address (superficially more important than the
incoming one) and leaving incoming undefined, I've no idea why ftpget is
confused by a HUP signal - it's not obvious why UDP sockets should have any
relevance to it!

The long pause/timeout on HUP or shutdown when idle is more readily
explicable, at least in a hand-waving way - I haven't traced through exactly
what happens. serverConnectionsClose() in main.c handles closing
UDP sockets specially, depending on the situation. If the same socket is
used for incoming and outgoing UDP, it is flagged as closed for incoming
data *but* without actually closing it down - a comment notes "don't close
outgoing ICP connection, we need to write to it during shutdown".

If the incoming UDP socket is different from the outgoing socket, it really
*is* shut down. My guess is that with incoming UDP using an explicit address
and outgoing UDP using that address independently/implicitly along with all
others (hence appearing "different"), outgoing access gets closed down (or
at least disrupted) when the incoming socket is closed down, leading to the
icpHandleUdp error and the long timeout.

                                John Line

-- 
University of Cambridge WWW manager account (usually John Line)
Send general WWW-related enquiries to webmaster@ucs.cam.ac.uk