Re: [squid-users] Unneeded DNS lookups for cache_peer selection

Am 30.01.2014 10:45, schrieb Amos Jeffries:
> On 30/01/2014 9:25 p.m., Amon Ott wrote:
>> Am 29.01.2014 21:51, schrieb Amos Jeffries:
>>> On 2014-01-30 03:48, Amon Ott wrote:
>>>> We have a setup with a single parent proxy that shall be used for all
>>>> requests. "never_direct allow all" ensures that the parent is not to be
>>>> bypassed. Still, every request leads to an extra unnecessary (and
>>>> failing) DNS lookup. We enjoyed several complaints because of unwanted
>>>> DNS traffic inside a firewalled area. As all results are negative, even
>>>> the DNS caching does not really help.
>>>>
>>>> I have traced the problem into peerSelectDnsPaths() in
>>>> src/peer_select.cc. It seems that despite the clear fact that only one
>>>> cache_peer can ever be selected (and has to be selected), the code still
>>>> makes a DNS request for the hostname in the requested URL to find the
>>>> best proxy. It would be greatly appreciated if we could suppress these
>>>> lookups without loosing DNS access for parent proxy lookup by DNS name.
>>>> The lookup does not make any sense to me even with multiple parents, if
>>>> the parent selection is never based on the request target host.
>>>>
>>>> The only workaround we have found so far is to specify the parent proxy
>>>> by IP address and tell Squid with "dns_nameservers localhost" to use a
>>>> not existing DNS server. However, this way we loose the DNS based load
>>>> balancing at the parent proxy, which we cannot influence ourselves.
>>>>
>>>> This looks like a similar problem to the one discussed in
>>>> http://www.squid-cache.org/mail-archive/squid-users/201309/0364.html
>>>
>>>
>>> It does indeed sound like a similar problem. The issue is likely the
>>> same, "bad" configuration requiring DNS lookups to happen.
>>> There are two solutions indicated in my response to that referenced post:
>>> 1) fixing the ACL tests and config options requiring DNS
>>
>> I have not found any, please do have a look yourself.
>>
>>> 2) using /etc/hosts (or a localhost DNS resolver)
>>
>> This is no good option, because the uplink proxy's IP address can change
>> at any time, so we need to use its DNS name. That DNS domain is not
>> under our control, so a DNS server at localhost does not help. In worst
>> case I would have to run a cron job updating the host entry every minute
>> or such - sounds horrible.
>
> Well then. There is the answer.
>
> Squid has to find the IP address that name points at as often as the TTL
> says to re-check for changes. Without the destination IP address it
> cannot send TCP packets.

It needs to recheck the IP of the parent proxy, but not for the URLs the
users want to see. The parent proxy has to take care of that, and
external DNS does not work in the internal DNS setup anyway. It creates
lots of bogus extra packets in the internal net without any good reason.
If we can get that fixed, it will also reduce network traffic and DNS
server load for many other users with parent proxies, too.

>> Since the DNS lookup fails anyway, the solution could be as simple as
>> having a simple switch to forbid URL target host DNS lookups completely.
>> Parent proxy selection could still continue like after failed DNS lookup.
>
> If the DNS lookup fails there Squid has no idea where the cache_peer has
> been moved to when it changes. And never_direct prevents alternative IPs
> being looked for.

I seem to have been unclear here: I want to get rid of DNS lookups of
request URL hosts, not of DNS lookups for the parent proxy. ATM, every
single Web request over the proxy seems to trigger a DNS lookup for that
host. tcpdump clearly shows tons of requests for external DNS names to
our DNS server.

It seems to me that the parent selection algorithm always looks up the
host in the user request URL at the place mentioned above, even though
it is only needed with certain per-parent ACLs or if the proxies might
get bypassed for speed. Maybe the lookup could be delayed until it is
really needed for a decision. AFAICS, the new "-n" parameter to ACLs in
3.4 has a similar idea, it avoids DNS traffic.

> Your choices are exactly thse above, or possibly also configuring the IP
> explicitly into squid.conf and reconfiguring whenever it changes. But
> finding out when that change happens will be even more of a problem for
> the process reconfiguring Squid than it is for a localhost DNS server.

As said above: regular DNS lookups for the parent proxy work and are
fine, but not for the Internet hosts users want to visit.

>>> If you are willing to post your config file (without any # comment
>>> lines) I am happy to check it like I did that earlier one.
>>
>> The stripped squid.conf is attached. The outgoing address settings are
>> workarounds for https URL problems without IPv6, which is disabled in
>> our kernel. We have tested with Squid 3.3.11 and 3.4.2.
>
> PS. in 3.4 the tcp_outgoing_address hack is not doing anything.

Thanks, will remove that for 3.4.

Amon.
Received on Thu Jan 30 2014 - 10:52:29 MST