Re: [squid-users] Well, this is what I concluded about using squid!

On 3/09/2013 11:02 a.m., Golden Shadow wrote:
> Hi Amos, Kinkie and Eliezer
>
>
> @Kinkie: It's very interesting to me to know about the companies that provide commercial support for squid on:(http://www.squid-cache.org/Support/services.html). Thanks for your reply and for providing this link.
>
> @Eliezer: Thanks for your reply. I'm already using Centos but yeah, I think I need to consider faster disk drives, perhaps SSDs.
>
> @Amos:
> Thanks a lot for your detailed explanation, you made several things clearer to me.
>
> By "configure options", I meant the options that should be used with the configure script that is run before "make". Is there any good documentation about their meanings and what options to use with what values based on a certain environment and/or hardware? Take for example --enable-async-io=, should I use it? What is the best value to use? I don't think there is good documentation about those configure options.

The --help option on ./configure itself has the most detailed
documentation for each particular option we have at present. We are
trying to document them well there.

--enable-async-io does not exist and there is no mention of it being
removed in any of the 3.x series.

Did you mean --with-aio=N ?

> As for: http://wiki.squid-cache.org/Features/Tproxy4
> Although it takes care of most things a beginner would need to implement a TPROXY, but it looks summarized and in my opinion does not remove the confusion caused by the many old and obsolete articles about how to implement a TPROXY. When I first started, I was puzzled whether I should compile the kernel from source to use the TPROXY patches or just use the kernel that comes with my Centos 6.4 kernel.

Which is what that page seeks to clarify in the first section under
requirements and differences between TPROXYv4 and TPROXYv2 (the old
patched-in code). They are not compatible versions with each other.

> Moreover, the following section really confused me:
>
> Use DIVERT to prevent existing connections going through TPROXY twice:
> iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
>
> Based on my understanding of this iptables rule, I think it is intended for reply packets coming from web servers to squid (with the spoofed IP address), right?! Saying that its purpose is to "prevent existing connections going through TPROXY twice" really confused me and I still can't understand what this means!

I'm not sure myself. It is part of the magics which the iptables people
passed to me.

As far as I understand "-m socket" matches any packet which has a local
process with sockets sending/recieving. So it _probably_ causes the
TPROXY rule to operate only on SYN,SYN-ACK packets from the client and
ignore the packet on the squid->server connection *and* packets on
ESTABLISHED state for already caught connections. Although I am not sure
of that ESTABLISHED ivolvement. "DIVERT" is just the name of a local
custom chain with the operations to perform on the "-m socket" matched
packets.

As far as Squid goes this is system level voodoo to be added to your
iptables rules. The netfilter help mailing lists is the place to go for
explanations if you want to know the deeper meanings. I can help you wit
hthe Squid side, such as it is, and history behind the wiki page since I
am the developer for TPROXYv4 in Squid - but it has a long history
before I gt anywhere near it.

> I know there is a lot of documentation online about tuning file descriptors and similar things, and yeah I think it's not the job of squid documentation to talk about how to do those things. What I meant is that it would perhaps be much easier for newbies to be notified that they may need to tune file descriptors, that they may get SYN floods, that they may have page faults, and all other things that are waiting for them down the way, all in a single document. This will also reduce the number of duplicate questions on this list, I guess.

Hmm. That is just the thing. All of what you describe above is actually
advanced system tuning details, not beginner or newbie details at all.
You will find most OS provide such a document specifically for admin
using their systems. By the time you get to fiddling with features
which need them you are expected to be a knowledgable sysadmin, or at
least reading a document specific to that feature - *that* document has
the notes and a Troubleshooting section about potential issues known.

For beginners Squid-3 is quite capable of pulling out the system default
settings for just about everything and using them. So there is _no_
tuning required and our release packages are expected to install and be
a usable proxy immediately (although some distros [Debian/Ubuntu/RHEL]
disable that capability).

Even that seemingly ridiculous 1024 FD default can easily handle up to
100 concurrent clients for SME installations. Although once they get
serious on performance tuning FD is one thing they start looking for and
find the documents about Squid vs ulimit.

> Thanks once again and I really appreciate your support.

Thank you.

Amos
Received on Tue Sep 03 2013 - 11:37:30 MDT