[squid-users] Optimizing Squid for huge traffic

Hi everybody !

I'm actually trying to use a squid to handle a large amount of
connections... And I'm facing some performance problems I don't
understand... If someone has an idea about where the problem can be ...

I would appreciate any return :)

Typically, I have the following case:

 - Squid receives requests in the proxy form (tits 'GET
   http://host/uri') from 6 apache mod_proxy servers (actually needed
   because of features and patches apache-only)

 - Average HTTP req/s is 300 (bursts at 400)

 - Squid redirects to a very restricted subset of servers: 7 apache
   servers serving only distinct files between 10k and 200k (gifs, jpeg,
   etc, constated to be ~20Ko average)...

 - I use /etc/hosts for resolving (--disable-internal-dns).

 - The "interesting" parts of config file is joined...

 - I'm using the 'diskd' method (tested with ufs, not much differences
   except that it is worse than diskd :( perhaps should I use async-io
   ?)

 - I've disabled the persistent connections (speeds a little).

And I have the following problem:

 - Response time is very long... In place of the 100ms / 500ms usual
   timing I get an average response time of 3-10s, sometimes 30s !!!

Precisions:

 - The server load is ridiculous (something like 1.5) and the config is
   a bi-PIII 1GHz + 2 Go RAM + Ultra160 drives on a DAC1100 controller.
   (raid5) (a VAlinux-2251 in fact), using RedHat 7.1 (not MY choice :-(

 - kernel 2.2.19 (experienced some memory leak problem with 2.4,
   completely unusable in my case...)

 - I've tried a 'single instance' (just one squid) and a split (2 apache
   back-end by squid, 3 squid running) without much differences, just
   the triple instance is a little faster...

 - I've tried to modify the refresh_pattern, and in fact squid is faster
   when doing _no http request at all_ when object is already in the
   cache (. 360 100% 4320)... But this is not possible, cause there is a
   lot of gifs/jpegs moving very frequently and needing to be as fresh
   as possible...

 - Some changes have been done on the system in such a way to provide a
   lot of file descriptor to Squid, and better network performances:
   - changed /usr/include/bits/types.h to set __FD_SETSIZE to 65536
     instead of 1024
   - ulimit -HSn 32768 right before the ./configure runtime, and before
     running the squid
   - changed in /proc:
     - fs.file-max = 32768
     - net.ipv4.tcp_max_syn_backlog = 16384
     - net.ipv4.ip_local_port_range = "1024 65535"

 - Here is the config file excerpt:
#######################################################################
[...snip...]
cache_dir diskd /ctk/var/squid/spool 40960 64 256
store_avg_object_size 40 KB

# not higher because of squid eating much more than this setting.
cache_mem 500 MB

cache_swap_low 95
cache_swap_high 98

#faster when pconn are off.
client_persistent_connections off
server_persistent_connections off

#We use mod_tsunami on back apaches. So we need a very little ttl for
#errors.
negative_ttl 3 seconds

#32 is huge for just resolving 7 hosts using /etc/hosts, but minus isn't
#ok according to cache_log (too much requests queued...)
dns_children 32

#for all urls (.) :
#We want a Head/IMS request very frequently (20%)
#But no more than every min (60)
#Force 'reload' if not modif since 3 days (4320)
refresh_pattern . 60 20% 4320

#We need to accept proxy requests (GET http://...)
httpd_accel_with_proxy on

httpd_accel_uses_host_header off
redirect_rewrites_host_header off
httpd_accel_host virtual
#######################################################################

Cheers,

--
Helios de Creisquer <helios@balios.org>