Re: [squid-users] WCCP

Ross Kovelman wrote:
>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>> Date: Fri, 30 Oct 2009 14:08:23 +1300
>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>> Subject: Re: [squid-users] WCCP
>>
>> Ross Kovelman wrote:
>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>> Date: Tue, 27 Oct 2009 12:17:12 +1300
>>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>> Subject: Re: [squid-users] WCCP
>>>>
>>>> On Wed, 21 Oct 2009 12:20:00 -0400, Ross Kovelman
>>>> <rkovelman_at_gruskingroup.com> wrote:
>>>>>> From: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>>> Date: Mon, 19 Oct 2009 22:35:36 -0400
>>>>>> To: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>>>> Subject: Re: [squid-users] WCCP
>>>>>>
>>>>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>>>> Date: Tue, 20 Oct 2009 13:20:27 +1300
>>>>>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>>>>> Subject: Re: [squid-users] WCCP
>>>>>>>
>>>>>>> On Mon, 19 Oct 2009 20:06:55 -0400, Ross Kovelman
>>>>>>> <rkovelman_at_gruskingroup.com> wrote:
>>>>>>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>>>>>> Date: Tue, 20 Oct 2009 12:40:02 +1300
>>>>>>>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>>>>>>> Subject: Re: [squid-users] WCCP
>>>>>>>>>
>>>>>>>>> On Mon, 19 Oct 2009 18:26:18 -0400, Ross Kovelman
>>>>>>>>> <rkovelman_at_gruskingroup.com> wrote:
>>>>>>>>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>>>>>>>> Date: Tue, 20 Oct 2009 11:04:42 +1300
>>>>>>>>>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>>>>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>>>>>>>>> Subject: Re: [squid-users] WCCP
>>>>>>>>>>>
>>>>>>>>>>> On Mon, 19 Oct 2009 14:21:44 -0400, Ross Kovelman wrote:
>>>>>>>>>>>>> From: Amos Jeffries
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ross Kovelman wrote:
>>>>>>>>>>>>> From: Amos Jeffries:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ross Kovelman wrote:
>>>>>>>>>>>>> I am going to be using WCCP. I did another reconfigure with
>>>>>>>>>>>>> the
>>>>>>>>>>>>> --enable
>>>>>>>>>>>>> WCCP option. How can I check that it is on and running? The
>>>>>>> next
>>>>>>>>>>>>> step I
>>>>>>>>>>>>> need to do is upgrade to version 2 since the Cisco only
>>>>>>>>> communicates
>>>>>>>>>>>>> on
>>>>>>>>>>>>> version 2. I tried to do the patch < upgrade patch but then
>>>> I
>>>>>>> get
>>>>>>>>> a
>>>>>>>>>>>>> response with path to upgrade and I am not sure where the
>>>> file
>>>>>>> is
>>>>>>>>> I
>>>>>>>>>>>>> need
>>>>>>>>>>>>> patch.
>>>>>>>>>>>>> There is zero need to patch for support WCCPv2. It's been
>>>> built
>>>>>>>>> into
>>>>>>>>>>>>> Squid for many years now.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Run "./configure --help".
>>>>>>>>>>>>> * If it lists "--disable-wccpv2" there is no need to do
>>>>>>> anything.
>>>>>>>>>>>>> * If it lists "--enable-wccpv2" , add that to your build
>>>>>>> options.
>>>>>>>>>>>>> * If it does not mention "wccpv2" at all upgrade your Squid
>>>>>>>>>>> version.
>>>>>>>>>>>>> Then setup squid.conf with the relevant wccp2_* options.
>>>>>>>>>>>>>
>>>>>>>>>>>>> http://www.squid-cache.org/Doc/config/ or the wiki example
>>>>>>> configs
>>>>>>>>>>> have
>>>>>>>>>>>>> details on those.
>>>>>>>>>>>>> Thanks again.
>>>>>>>>>>>>> Running the ./configure --help only says this:
>>>>>>>>>>>>> --disable-wccp Disable Web Cache Coordination V1
>>>>>>> Protocol
>>>>>>>>>>>>> --disable-wccpv2 Disable Web Cache Coordination V2
>>>>>>> Protocol
>>>>>>>>>>>>> When I did the install I ran the ./configure --enable wccp
>>>>>>>>>>>>> option.
>>>>>>> I
>>>>>>>>>>>>> didn't
>>>>>>>>>>>>> say --enable-wccpv2, does this matter? I also have this in the
>>>>>>>>>>> config:
>>>>>>>>>>>>> wccp2_router 192.168.16.1
>>>>>>>>>>>>> wccp2_forwarding_method 1
>>>>>>>>>>>>> wccp2_return_method 1
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am running Squid Web Proxy 2.7.STABLE5.
>>>>>>>>>>>>> Okay. Thats fine.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The ./configure results mean that both WCCP versions are built
>>>>>>>>>>>>> into
>>>>>>>>>>>>> Squid by default unless you explicitly say --disable. Nothing
>>>>>>>>>>>>> extra
>>>>>>>>>>>>> needed to build them.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The config options you have there are already WCCPv2-only
>>>> options
>>>>>>> for
>>>>>>>>>>>>> Cisco. Nothing new needed there either.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If thats not working its a config error somewhere.
>>>>>>>>>>>>>
>>>>>>>>>>>> I am getting this in my cache log:
>>>>>>>>>>>>
>>>>>>>>>>>> Accepting proxy HTTP connections at 0.0.0.0, port 3128, FD 20.
>>>>>>>>>>>> commBind: Cannot bind socket FD 21 to *:3128: (48) Address
>>>> already
>>>>>>> in
>>>>>>>>>>> use
>>>>>>>>>>>> Accepting proxy HTTP connections at 0.0.0.0, port 80, FD 21.
>>>>>>>>>>>> commBind: Cannot bind socket FD 22 to *:80: (48) Address already
>>>> in
>>>>>>>>> use
>>> http://wiki.squid-cache.org/SquidFaq/TroubleShooting#Cannot_bind_socket_FD_NN
>>>>> _
>>>>>>>>>>> to_.2A:8080_.28125.29_Address_already_in_use
>>>>>>>>>>>
>>>>>>>>>>> I would suspect this as part of the problem. The WCCP router will
>>>> be
>>>>>>>>>>> trying to contact whatever software is already running on port
>>>> 3128,
>>>>>>>>> not
>>>>>>>>>>> the Squid you are starting with WCCP config.
>>>>>>>>>>>
>>>>>>>>>>>> Accepting ICP messages at 0.0.0.0, port 3130, FD 22.
>>>>>>>>>>>> WCCP Disabled.
>>>>>>>>>>>> Accepting WCCPv2 messages on port 2048, FD 23.
>>>>>>>>> To answer your earlier question:
>>>>>>>>> the above two lines means WCCPv1 is disabled, WCCPv2 is being
>>>> used.
>>>>>>>>>>>> Initialising all WCCPv2 lists
>>>>>>>>>>>>
>>>>>>>>>>>> As from my other posting I need WCCP enabled but it is showing
>>>>>>>>> disabled.
>>>>>>>>>>>> Any reason why? How can I resolve this. Below is my lines in
>>>>>>> config
>>>>>>>>>>>> wccp2_router 192.168.16.1
>>>>>>>>>>>> wccp2_forwarding_method 1
>>>>>>>>>>>> wccp2_return_method 1
>>>>>>>>>>> The above are only the config of how squid sends packets to the
>>>>>>> Cisco.
>>>>>>>>>>> WCCP requires configuration Cisco, the squid box OS and firewall,
>>>>>>>>>>> and
>>>>>>>>>>> routing tables. Any one of which could be the problem.
>>>>>>>>>>> The tutorials and troubleshooting info we have at present is a
>>>>>>>>>>> little
>>>>>>>>>>> spread out and disjointed. What how-to are you working from?
>>>>>>>>>>>
>>>>>>>>>>> Amos
>>>>>>>>>> Amos,
>>>>>>>>>> I just did a TCP dump and I think my problem is the GRE packet. It
>>>>>>>>>> is
>>>>>>>>>> being
>>>>>>>>>> listed I think as unknown. Shouldn't squid be able to pick the
>>>>>>>>>> packet
>>>>>>>>> up
>>>>>>>>>> and open it? The Cisco sees squid and relays the information good
>>>>>>>>>> but
>>>>>>>>> it
>>>>>>>>>> is
>>>>>>>>>> stopping at the squid box. Any ideas? I am just google'ing around
>>>> no
>>>>>>>>> set
>>>>>>>>>> how to.
>>>>>>>>> Okay. I've polished up our exemplar configs a little:
>>>>>>>>> http://wiki.squid-cache.org/Features/Wccp2
>>>>>>>>> (some way to go though).
>>>>>>>>>
>>>>>>>>> There are four parts to WCCP systems:
>>>>>>>>>
>>>>>>>>> 1) WCCP capture and redirect
>>>>>>>>>
>>>>>>>>> 2) gre tunnel between the Cisco and Squid boxes
>>>>>>>>>
>>>>>>>>> 3) squid box firewall settings and NAT capture of received gre
>>>>>>>>> packets
>>>>>>>>>
>>>>>>>>>
>>> http://wiki.squid-cache.org/ConfigExamples/Intercept#Traffic_Interception_cap
>>>>> t
>>>>>>>>> ure_into_Squid
>>>>>>>>>
>>>>>>>>> 4) squid.conf settings to make Squid contact the cisco router
>>>>>>>>>
>>>>>>>>> Amos
>>>>>>>>>
>>>>>>>> From what I have read and what you show only for the PIX and ASA
>>>> should
>>>>>>> be
>>>>>>>> the same. The Pix is actually correct for the ASA, although that is
>>>>>>> what
>>>>>>>> Cisco told me to do.
>>>> Hmm, I was worried a bit by this. Then realized what the problem was.
>>>> The difference appears to have been only a security ACL added to the ASA
>>>> config and the screwy wrapping.
>>>>
>>>> Thanks for that hint.
>>>>
>>>>>>>> As far as:
>>>>>>>> wccp2_router - My cisco router address
>>>>>>>> wccp2_forwarding_method - I took this out of my config as GRE is
>>>>>>>> default
>>>>>>>> wccp2_return_method - same as forward
>>>>>>>> wccp2_assignment_method - nothing in config
>>>>>>>> wccp2_service - nothing in config
>>>>>>>>
>>>>>>>> Am I missing something? If I have my cisco config turned on for WCCP
>>>>>>> and
>>>>>>>> squid running no one can browse the web. If I turn squid off and
>>>> leave
>>>>>>>> wccp
>>>>>>>> running on the Cisco browsing web is perfect. No issues. Anything
>>>> else
>>>>>>> to
>>>>>>>> check?
>>>>>>> ... rp_filter settings on the Squid box are turned off.
>>>>>>>
>>>>>>> ... iptables does REDIRECT or DNAT capture of the packets to the Squid
>>>>>>> http_port marked with "transparent"
>>>>>>>
>>>>>>>> bert:~ administrator$ sudo tcpdump -n -i en1 ip proto gre
>>>>>>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>>>>>>> decode
>>>>>>>> listening on en1, link-type EN10MB (Ethernet), capture size 96 bytes
>>>>>>>> 15:00:33.599161 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 60:
>>>>>>>> gre-proto-0x883e
>>>>>>>> 15:00:34.715585 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 60:
>>>>>>>> gre-proto-0x883e
>>>>>>>> 15:00:34.805734 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 56:
>>>>>>>> gre-proto-0x883e
>>>>>>>> 15:00:34.808181 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 56:
>>>>>>>> gre-proto-0x883e gre-proto-0x883e
>>>>>>>> 15:00:34.805734 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 56:
>>>>>>>> gre-proto-0x883e
>>>>>>>> 15:00:34.808181 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 56:
>>>>>>>> gre-proto-0x883e
>>>>>>>>
>>>>>>>> Does that help? Let me know what you need from me so we can resolve
>>>>>>> this.
>>>>>>>> I did mask off my IP but the IP prior to the > is the ASA and the
>>>>>>> numbers
>>>>>>>> after is the squid server
>>>>>>>>
>>>>>>>> Thanks
>>>>>> Amos,
>>>>>>
>>>>>> I have this in my sysctl config:
>>>>>> net.ipv4.ip_forward =1
>>>>>> net.ipv4.conf.all.rp_filter = 0
>>>>>>
>>>>>> That should take care of the rp_filter. Although how can I check that
>>>> I
>>>>>> don't know. I am also running transparent so I assume that iptables
>>>>>> thing
>>>>>> you wrote I do not need to do?
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>>
>>>>> I am starting to look more into this and what I see is this on the
>>>> firewall
>>>>> log:
>>>>> Oct 21 12:03:37 bert ipfw: 12313 Accept P:47 192.168.xx.1
>>>> 192.168.xx.xxx
>>>>> in
>>>>> via en1
>>>>>
>>>>> P47 is GRE so I can see that the GRE packet from the ASA is passed and
>>>>> accepted to the squid server. I do not think Squid knows how to either
>>>>> decipher the GRE packet and or when it tries to send the information
>>>> back
>>>>> out its not going back to the client or ASA. How can I resolve this?
>>>> Aha, you have an ASA. Somehow I missed that detail earlier. This is the
>>>> specific ASA config details we have so far:
>>>> http://wiki.squid-cache.org/ConfigExamples/Intercept/CiscoAsaWccp2
>>>>
>>>> Check that you have the squid bypass in the config. Thats one of the
>>>> critical parts.
>>>>
>>>> Good tracking so far.
>>>>
>>>> It's the OS business to unwrap the GRE packet into a normal TCP packet
>>>> before passing it to Squid. I'm not sure how ipfw ensures that. modprobe
>>>> ip_gre?
>>>>
>>>> The next bit will be to see if Squid receives the packet at all. With
>>>> debug_options ALL,6 or so cache.log should record a connection accepted
>>>> from the client and show what happens to it.
>>>>
>>>> Amos
>>>>
>>> Amos,
>>>
>>> Got it working, but I am having some timeout issues when browsing all
>>> websites. Do you know why or know what I can look for? I do see the ASA
>>> and squid server communicating now.
>>>
>>> Thanks
>> Not a clue. I'd guess some delays on the network. Perhaps during DNS
>> lookups.
>>
>> Amos
>> --
>> Please be using
>> Current Stable Squid 2.7.STABLE7 or 3.0.STABLE19
>> Current Beta Squid 3.1.0.14
>
> Amos,
>
> I got cisco on the phone and did some packet debugging. What we found is
> the ASA and squid communicate. The problem lies where for some reason the
> clients browser will report a "connection time out". Do you know of
> anything that should be done, IPTables or Firewall so the packet does go
> correctly to the client. I was told WCCP communicates with the ASA and
> Squid and then from there the request is between squid and the client. I
> have done everything here:
> http://www.sublime.com.au/squid-wccp/

Sorry for the delay. Looks like you understand it all so far.

I just have a few Qs to clarify your situation in my mind:

The architecture piece labeled "10/100BT Layer-2 Segment"...
   that is a switch right?
   so that squid+client are able to directly wire-connect without going
through the cisco ASA again?
   with both also setup to route local network range packets directly to
each other? (again without involving the cisco ASA)

> Besides compile of the wccp module and kernal rebuilding. I know this is
> not squid issue, or at least I do not think so, so an Fedora help you can
> give would be appreciated. I have also followed the book from O'reilly
> called squid. I am going to dig a lil deeper on the server side doing dumps
> to see what else I find. The only other clue I see is the packet going from
> the server to the ASA:
> ICMP Destinatoon unreachable (Host administratively prohibited)

Hmm, indicates a firewall problem. Possibly preventing part of the
traffic between squid and the client or between squid and server.

Worth getting a low-level trace and finding out whats generating it.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE7 or 3.0.STABLE20
   Current Beta Squid 3.1.0.14