[squid-users] ICAP protocol error

Hi guys,

We are currently using Squid 3.1.12 (old, I know) on RHEL 5.8 64bit
(HP ProLiant DL380 G7 with 16 CPU and 28GB RAM)
Squid Cache: Version 3.1.12
configure options: '--enable-ssl' '--enable-icap-client'
'--sysconfdir=/etc/squid' '--enable-async-io' '--enable-snmp'
'--enable-poll' '--with-maxfd=32768' '--enable-storeio=aufs'
'--enable-removal-policies=heap,lru' '--enable-epoll'
'--disable-ident-lookups' '--enable-truncate'
'--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid'
'--with-default-user=squid' '--prefix=/opt/squid' '--enable-auth=basic
digest ntlm negotiate'
'-enable-negotiate-auth-helpers=squid_kerb_auth'
--with-squid=/home/squid/squid-3.1.12 --enable-ltdl-convenience

As ICAP server, we are using McAfee Webwasher 6.9 (old too, I know).
Up until recently we hardly had problems with this environment.
Squid is doing authentication via Kerberos and passing the username to
the Webwasher, which is doing a LDAP lookup to find the users groups
and assign a policy based on group membership.
We have multiple Squids and multiple Webwasher with a hardware
loadbalancer, approx 15k users.

Since a couple of weeks, we almost daily get an ICAP server error
message, similar to:
http://support.kaspersky.com/2723
Unfortunately, I cannot figure out why. In blame the webwasher, but I
am not 100% sure.

This is my ICAP configuration:
#ICAP
icap_enable on
icap_send_client_ip on
icap_send_client_username on
icap_preview_enable on
icap_preview_size 30
icap_uses_indirect_client off
icap_persistent_connections on
icap_client_username_encode on
icap_client_username_header X-Authenticated-User
icap_service service_req reqmod_precache bypass=0
icap://10.122.125.48:1344/wwreqmod
adaptation_access service_req deny favicon
adaptation_access service_req deny to_localhost
adaptation_access service_req deny from_localnet
adaptation_access service_req deny whitelist
adaptation_access service_req deny dst_whitelist
adaptation_access service_req deny icap_bypass_src
adaptation_access service_req deny icap_bypass_dst
adaptation_access service_req allow all
icap_service service_resp respmod_precache bypass=0
icap://10.122.125.48:1344/wwrespmod
adaptation_access service_resp deny favicon
adaptation_access service_resp deny to_localhost
adaptation_access service_resp deny from_localnet
adaptation_access service_resp deny whitelist
adaptation_access service_resp deny dst_whitelist
adaptation_access service_resp deny icap_bypass_src
adaptation_access service_resp deny icap_bypass_dst
adaptation_access service_resp allow all

Could an upgrade (either to 3.2 or to 3.3) solve this problem (There
are more icap options in recent squid versions available)?
Unfortunately, this is a rather complex organisational process, that's
why I did not do that yet.
I do have a test machine, but this ICAP error is not reproducible,
only in production. Server load and IO-througput are ok, there is
nothing suspicious on the server. I recently activated icap debug
option 93 and found following message:
2013/06/12 15:32:15| suspending ICAP service for too many failures
2013/06/12 15:32:15| essential ICAP service is suspended:
icap://10.122.125.48:1344/wwrespmod [down,susp,fail11]
2013/06/12 15:35:15| essential ICAP service is up:
icap://10.122.125.48:1344/wwreqmod [up]
2013/06/12 15:35:15| essential ICAP service is up:
icap://10.122.125.48:1344/wwrespmod [up]
I don't know why this check failed, but it usually does not occur when
clients are getting the icap protocol error page.

Another possibility would be the ICAP bypass, but our ICAP server is
doing anti-Malware-checking and that's why I don't want to activate
this feature.

Does anybody have other ideas?

Thanks!
Peter
Received on Wed Jun 12 2013 - 13:57:41 MDT