Re: [squid-users] Squid and Squidguard using high disk IO

Hey,

Notes inside.

On 11/09/2013 05:58 PM, Kaya Saman wrote:
>
> What can I do to improve performance with this?
>
>
> Is this line too high: url_rewrite_children 500
YES!!

> or simply have a misconfigured something?

> I additionally have 'c-icap' running with squidclamav coupled to clamd
> in case that is of importance - not using the squidGuard line in the
> squidclamav.conf file!!!
>
> Basically how can I get the IO usage down and get the system to work again?
For how many users exactly?
Just a note that I am not in a favor of any OS by default but I would
feel better Using Linux.

>
> - the logs don't indicate anything outside of 'starting squidGuard
> process' many times.
The basic assumption of using 500 child process is that you have atleast
100 CPUs.
SquidGuard was design for performance which is lots of urls per sec.
It can be tested just to clear the point out.
for example in a rate of 1500k requests per second you should not have a
need in more then 40-50 children.
In practice it works a bit different speed since there is a speed limit
on STDIN and STDOUT which slows down the speed of squid and squidguard
communication blocking the whole squid instance(in a way).

If you need basic url filtering you can use ICAP which has an option to
run as a standalone service outside of squid settings and machine.

I have written in the past a small ICAP service for the favor of
requests manipulation and filtering.
I have never finished it in a level I was happy with but the basic code
can be seen here:
https://github.com/elico/echelon

I know for a fact that ICAP interface adds concurrency by the "nature"
of it using TCP.

This is not the place to ask about concurrency in squidguard which can
allow the usage of square less processes(children) for more requests.

In order to find the right number of children start with 40 and see if
it fits you and then see what is the bottle neck in the whole setup.

Eliezer
Received on Sat Nov 09 2013 - 17:24:06 MST