Re: [squid-users] WCCP

On Nov 3, 2009, at 3:22 PM, Ross Kovelman wrote:

>
> On Nov 3, 2009, at 12:07 AM, Amos Jeffries wrote:
>
>> Ross Kovelman wrote:
>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>> Date: Fri, 30 Oct 2009 14:08:23 +1300
>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>> Subject: Re: [squid-users] WCCP
>>>>
>>>> Ross Kovelman wrote:
>>>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>>> Date: Tue, 27 Oct 2009 12:17:12 +1300
>>>>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>>>> Subject: Re: [squid-users] WCCP
>>>>>>
>>>>>> On Wed, 21 Oct 2009 12:20:00 -0400, Ross Kovelman
>>>>>> <rkovelman_at_gruskingroup.com> wrote:
>>>>>>>> From: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>>>>> Date: Mon, 19 Oct 2009 22:35:36 -0400
>>>>>>>> To: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>>>>>> Subject: Re: [squid-users] WCCP
>>>>>>>>
>>>>>>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>>>>>> Date: Tue, 20 Oct 2009 13:20:27 +1300
>>>>>>>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-
>>>>>>>>> cache.org>
>>>>>>>>> Subject: Re: [squid-users] WCCP
>>>>>>>>>
>>>>>>>>> On Mon, 19 Oct 2009 20:06:55 -0400, Ross Kovelman
>>>>>>>>> <rkovelman_at_gruskingroup.com> wrote:
>>>>>>>>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>>>>>>>> Date: Tue, 20 Oct 2009 12:40:02 +1300
>>>>>>>>>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>>>>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org
>>>>>>>>>>> >
>>>>>>>>>>> Subject: Re: [squid-users] WCCP
>>>>>>>>>>>
>>>>>>>>>>> On Mon, 19 Oct 2009 18:26:18 -0400, Ross Kovelman
>>>>>>>>>>> <rkovelman_at_gruskingroup.com> wrote:
>>>>>>>>>>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>>>>>>>>>> Date: Tue, 20 Oct 2009 11:04:42 +1300
>>>>>>>>>>>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>>>>>>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org
>>>>>>>>>>>>> >
>>>>>>>>>>>>> Subject: Re: [squid-users] WCCP
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, 19 Oct 2009 14:21:44 -0400, Ross Kovelman wrote:
>>>>>>>>>>>>>>> From: Amos Jeffries
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ross Kovelman wrote:
>>>>>>>>>>>>>>> From: Amos Jeffries:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ross Kovelman wrote:
>>>>>>>>>>>>>>> I am going to be using WCCP. I did another
>>>>>>>>>>>>>>> reconfigure with
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> --enable
>>>>>>>>>>>>>>> WCCP option. How can I check that it is on and
>>>>>>>>>>>>>>> running? The
>>>>>>>>> next
>>>>>>>>>>>>>>> step I
>>>>>>>>>>>>>>> need to do is upgrade to version 2 since the Cisco only
>>>>>>>>>>> communicates
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> version 2. I tried to do the patch < upgrade patch
>>>>>>>>>>>>>>> but then
>>>>>> I
>>>>>>>>> get
>>>>>>>>>>> a
>>>>>>>>>>>>>>> response with path to upgrade and I am not sure where
>>>>>>>>>>>>>>> the
>>>>>> file
>>>>>>>>> is
>>>>>>>>>>> I
>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>> patch.
>>>>>>>>>>>>>>> There is zero need to patch for support WCCPv2. It's
>>>>>>>>>>>>>>> been
>>>>>> built
>>>>>>>>>>> into
>>>>>>>>>>>>>>> Squid for many years now.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Run "./configure --help".
>>>>>>>>>>>>>>> * If it lists "--disable-wccpv2" there is no need to do
>>>>>>>>> anything.
>>>>>>>>>>>>>>> * If it lists "--enable-wccpv2" , add that to your build
>>>>>>>>> options.
>>>>>>>>>>>>>>> * If it does not mention "wccpv2" at all upgrade your
>>>>>>>>>>>>>>> Squid
>>>>>>>>>>>>> version.
>>>>>>>>>>>>>>> Then setup squid.conf with the relevant wccp2_* options.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://www.squid-cache.org/Doc/config/ or the wiki
>>>>>>>>>>>>>>> example
>>>>>>>>> configs
>>>>>>>>>>>>> have
>>>>>>>>>>>>>>> details on those.
>>>>>>>>>>>>>>> Thanks again.
>>>>>>>>>>>>>>> Running the ./configure --help only says this:
>>>>>>>>>>>>>>> --disable-wccp Disable Web Cache Coordination
>>>>>>>>>>>>>>> V1
>>>>>>>>> Protocol
>>>>>>>>>>>>>>> --disable-wccpv2 Disable Web Cache Coordination
>>>>>>>>>>>>>>> V2
>>>>>>>>> Protocol
>>>>>>>>>>>>>>> When I did the install I ran the ./configure --enable
>>>>>>>>>>>>>>> wccp
>>>>>>>>>>>>>>> option.
>>>>>>>>> I
>>>>>>>>>>>>>>> didn't
>>>>>>>>>>>>>>> say --enable-wccpv2, does this matter? I also have
>>>>>>>>>>>>>>> this in the
>>>>>>>>>>>>> config:
>>>>>>>>>>>>>>> wccp2_router 192.168.16.1
>>>>>>>>>>>>>>> wccp2_forwarding_method 1
>>>>>>>>>>>>>>> wccp2_return_method 1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I am running Squid Web Proxy 2.7.STABLE5.
>>>>>>>>>>>>>>> Okay. Thats fine.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The ./configure results mean that both WCCP versions
>>>>>>>>>>>>>>> are built
>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>> Squid by default unless you explicitly say --disable.
>>>>>>>>>>>>>>> Nothing
>>>>>>>>>>>>>>> extra
>>>>>>>>>>>>>>> needed to build them.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The config options you have there are already WCCPv2-
>>>>>>>>>>>>>>> only
>>>>>> options
>>>>>>>>> for
>>>>>>>>>>>>>>> Cisco. Nothing new needed there either.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If thats not working its a config error somewhere.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I am getting this in my cache log:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Accepting proxy HTTP connections at 0.0.0.0, port 3128,
>>>>>>>>>>>>>> FD 20.
>>>>>>>>>>>>>> commBind: Cannot bind socket FD 21 to *:3128: (48)
>>>>>>>>>>>>>> Address
>>>>>> already
>>>>>>>>> in
>>>>>>>>>>>>> use
>>>>>>>>>>>>>> Accepting proxy HTTP connections at 0.0.0.0, port 80,
>>>>>>>>>>>>>> FD 21.
>>>>>>>>>>>>>> commBind: Cannot bind socket FD 22 to *:80: (48)
>>>>>>>>>>>>>> Address already
>>>>>> in
>>>>>>>>>>> use
>>>>> http://wiki.squid-cache.org/SquidFaq/TroubleShooting#Cannot_bind_socket_FD_NN
>>>>>>> _
>>>>>>>>>>>>> to_.2A:8080_.28125.29_Address_already_in_use
>>>>>>>>>>>>>
>>>>>>>>>>>>> I would suspect this as part of the problem. The WCCP
>>>>>>>>>>>>> router will
>>>>>> be
>>>>>>>>>>>>> trying to contact whatever software is already running
>>>>>>>>>>>>> on port
>>>>>> 3128,
>>>>>>>>>>> not
>>>>>>>>>>>>> the Squid you are starting with WCCP config.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Accepting ICP messages at 0.0.0.0, port 3130, FD 22.
>>>>>>>>>>>>>> WCCP Disabled.
>>>>>>>>>>>>>> Accepting WCCPv2 messages on port 2048, FD 23.
>>>>>>>>>>> To answer your earlier question:
>>>>>>>>>>> the above two lines means WCCPv1 is disabled, WCCPv2 is
>>>>>>>>>>> being
>>>>>> used.
>>>>>>>>>>>>>> Initialising all WCCPv2 lists
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As from my other posting I need WCCP enabled but it is
>>>>>>>>>>>>>> showing
>>>>>>>>>>> disabled.
>>>>>>>>>>>>>> Any reason why? How can I resolve this. Below is my
>>>>>>>>>>>>>> lines in
>>>>>>>>> config
>>>>>>>>>>>>>> wccp2_router 192.168.16.1
>>>>>>>>>>>>>> wccp2_forwarding_method 1
>>>>>>>>>>>>>> wccp2_return_method 1
>>>>>>>>>>>>> The above are only the config of how squid sends packets
>>>>>>>>>>>>> to the
>>>>>>>>> Cisco.
>>>>>>>>>>>>> WCCP requires configuration Cisco, the squid box OS and
>>>>>>>>>>>>> firewall,
>>>>>>>>>>>>> and
>>>>>>>>>>>>> routing tables. Any one of which could be the problem.
>>>>>>>>>>>>> The tutorials and troubleshooting info we have at
>>>>>>>>>>>>> present is a
>>>>>>>>>>>>> little
>>>>>>>>>>>>> spread out and disjointed. What how-to are you working
>>>>>>>>>>>>> from?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Amos
>>>>>>>>>>>> Amos,
>>>>>>>>>>>> I just did a TCP dump and I think my problem is the GRE
>>>>>>>>>>>> packet. It
>>>>>>>>>>>> is
>>>>>>>>>>>> being
>>>>>>>>>>>> listed I think as unknown. Shouldn't squid be able to
>>>>>>>>>>>> pick the
>>>>>>>>>>>> packet
>>>>>>>>>>> up
>>>>>>>>>>>> and open it? The Cisco sees squid and relays the
>>>>>>>>>>>> information good
>>>>>>>>>>>> but
>>>>>>>>>>> it
>>>>>>>>>>>> is
>>>>>>>>>>>> stopping at the squid box. Any ideas? I am just
>>>>>>>>>>>> google'ing around
>>>>>> no
>>>>>>>>>>> set
>>>>>>>>>>>> how to.
>>>>>>>>>>> Okay. I've polished up our exemplar configs a little:
>>>>>>>>>>> http://wiki.squid-cache.org/Features/Wccp2
>>>>>>>>>>> (some way to go though).
>>>>>>>>>>>
>>>>>>>>>>> There are four parts to WCCP systems:
>>>>>>>>>>>
>>>>>>>>>>> 1) WCCP capture and redirect
>>>>>>>>>>>
>>>>>>>>>>> 2) gre tunnel between the Cisco and Squid boxes
>>>>>>>>>>>
>>>>>>>>>>> 3) squid box firewall settings and NAT capture of received
>>>>>>>>>>> gre
>>>>>>>>>>> packets
>>>>>>>>>>>
>>>>> http://wiki.squid-cache.org/ConfigExamples/Intercept#Traffic_Interception_cap
>>>>>>> t
>>>>>>>>>>> ure_into_Squid
>>>>>>>>>>>
>>>>>>>>>>> 4) squid.conf settings to make Squid contact the cisco
>>>>>>>>>>> router
>>>>>>>>>>>
>>>>>>>>>>> Amos
>>>>>>>>>>>
>>>>>>>>>> From what I have read and what you show only for the PIX
>>>>>>>>>> and ASA
>>>>>> should
>>>>>>>>> be
>>>>>>>>>> the same. The Pix is actually correct for the ASA,
>>>>>>>>>> although that is
>>>>>>>>> what
>>>>>>>>>> Cisco told me to do.
>>>>>> Hmm, I was worried a bit by this. Then realized what the
>>>>>> problem was.
>>>>>> The difference appears to have been only a security ACL added
>>>>>> to the ASA
>>>>>> config and the screwy wrapping.
>>>>>>
>>>>>> Thanks for that hint.
>>>>>>
>>>>>>>>>> As far as:
>>>>>>>>>> wccp2_router - My cisco router address
>>>>>>>>>> wccp2_forwarding_method - I took this out of my config as
>>>>>>>>>> GRE is
>>>>>>>>>> default
>>>>>>>>>> wccp2_return_method - same as forward
>>>>>>>>>> wccp2_assignment_method - nothing in config
>>>>>>>>>> wccp2_service - nothing in config
>>>>>>>>>>
>>>>>>>>>> Am I missing something? If I have my cisco config turned
>>>>>>>>>> on for WCCP
>>>>>>>>> and
>>>>>>>>>> squid running no one can browse the web. If I turn squid
>>>>>>>>>> off and
>>>>>> leave
>>>>>>>>>> wccp
>>>>>>>>>> running on the Cisco browsing web is perfect. No issues.
>>>>>>>>>> Anything
>>>>>> else
>>>>>>>>> to
>>>>>>>>>> check?
>>>>>>>>> ... rp_filter settings on the Squid box are turned off.
>>>>>>>>>
>>>>>>>>> ... iptables does REDIRECT or DNAT capture of the packets to
>>>>>>>>> the Squid
>>>>>>>>> http_port marked with "transparent"
>>>>>>>>>
>>>>>>>>>> bert:~ administrator$ sudo tcpdump -n -i en1 ip proto gre
>>>>>>>>>> tcpdump: verbose output suppressed, use -v or -vv for full
>>>>>>>>>> protocol
>>>>>>>>> decode
>>>>>>>>>> listening on en1, link-type EN10MB (Ethernet), capture size
>>>>>>>>>> 96 bytes
>>>>>>>>>> 15:00:33.599161 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0,
>>>>>>>>>> length 60:
>>>>>>>>>> gre-proto-0x883e
>>>>>>>>>> 15:00:34.715585 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0,
>>>>>>>>>> length 60:
>>>>>>>>>> gre-proto-0x883e
>>>>>>>>>> 15:00:34.805734 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0,
>>>>>>>>>> length 56:
>>>>>>>>>> gre-proto-0x883e
>>>>>>>>>> 15:00:34.808181 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0,
>>>>>>>>>> length 56:
>>>>>>>>>> gre-proto-0x883e gre-proto-0x883e
>>>>>>>>>> 15:00:34.805734 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0,
>>>>>>>>>> length 56:
>>>>>>>>>> gre-proto-0x883e
>>>>>>>>>> 15:00:34.808181 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0,
>>>>>>>>>> length 56:
>>>>>>>>>> gre-proto-0x883e
>>>>>>>>>>
>>>>>>>>>> Does that help? Let me know what you need from me so we
>>>>>>>>>> can resolve
>>>>>>>>> this.
>>>>>>>>>> I did mask off my IP but the IP prior to the > is the ASA
>>>>>>>>>> and the
>>>>>>>>> numbers
>>>>>>>>>> after is the squid server
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>> Amos,
>>>>>>>>
>>>>>>>> I have this in my sysctl config:
>>>>>>>> net.ipv4.ip_forward =1
>>>>>>>> net.ipv4.conf.all.rp_filter = 0
>>>>>>>>
>>>>>>>> That should take care of the rp_filter. Although how can I
>>>>>>>> check that
>>>>>> I
>>>>>>>> don't know. I am also running transparent so I assume that
>>>>>>>> iptables
>>>>>>>> thing
>>>>>>>> you wrote I do not need to do?
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>>
>>>>>>> I am starting to look more into this and what I see is this on
>>>>>>> the
>>>>>> firewall
>>>>>>> log:
>>>>>>> Oct 21 12:03:37 bert ipfw: 12313 Accept P:47 192.168.xx.1
>>>>>> 192.168.xx.xxx
>>>>>>> in
>>>>>>> via en1
>>>>>>>
>>>>>>> P47 is GRE so I can see that the GRE packet from the ASA is
>>>>>>> passed and
>>>>>>> accepted to the squid server. I do not think Squid knows how
>>>>>>> to either
>>>>>>> decipher the GRE packet and or when it tries to send the
>>>>>>> information
>>>>>> back
>>>>>>> out its not going back to the client or ASA. How can I
>>>>>>> resolve this?
>>>>>> Aha, you have an ASA. Somehow I missed that detail earlier.
>>>>>> This is the
>>>>>> specific ASA config details we have so far:
>>>>>> http://wiki.squid-cache.org/ConfigExamples/Intercept/
>>>>>> CiscoAsaWccp2
>>>>>>
>>>>>> Check that you have the squid bypass in the config. Thats one
>>>>>> of the
>>>>>> critical parts.
>>>>>>
>>>>>> Good tracking so far.
>>>>>>
>>>>>> It's the OS business to unwrap the GRE packet into a normal TCP
>>>>>> packet
>>>>>> before passing it to Squid. I'm not sure how ipfw ensures that.
>>>>>> modprobe
>>>>>> ip_gre?
>>>>>>
>>>>>> The next bit will be to see if Squid receives the packet at
>>>>>> all. With
>>>>>> debug_options ALL,6 or so cache.log should record a connection
>>>>>> accepted
>>>>>> from the client and show what happens to it.
>>>>>>
>>>>>> Amos
>>>>>>
>>>>> Amos,
>>>>>
>>>>> Got it working, but I am having some timeout issues when
>>>>> browsing all
>>>>> websites. Do you know why or know what I can look for? I do
>>>>> see the ASA
>>>>> and squid server communicating now.
>>>>>
>>>>> Thanks
>>>> Not a clue. I'd guess some delays on the network. Perhaps during
>>>> DNS
>>>> lookups.
>>>>
>>>> Amos
>>>> --
>>>> Please be using
>>>> Current Stable Squid 2.7.STABLE7 or 3.0.STABLE19
>>>> Current Beta Squid 3.1.0.14
>>> Amos,
>>> I got cisco on the phone and did some packet debugging. What we
>>> found is
>>> the ASA and squid communicate. The problem lies where for some
>>> reason the
>>> clients browser will report a "connection time out". Do you know of
>>> anything that should be done, IPTables or Firewall so the packet
>>> does go
>>> correctly to the client. I was told WCCP communicates with the
>>> ASA and
>>> Squid and then from there the request is between squid and the
>>> client. I
>>> have done everything here:
>>> http://www.sublime.com.au/squid-wccp/
>>
>>
>> Sorry for the delay. Looks like you understand it all so far.
>>
>> I just have a few Qs to clarify your situation in my mind:
>>
>> The architecture piece labeled "10/100BT Layer-2 Segment"...
>> that is a switch right?
>> so that squid+client are able to directly wire-connect without
>> going through the cisco ASA again?
>> with both also setup to route local network range packets directly
>> to each other? (again without involving the cisco ASA)
>>
>>
>>> Besides compile of the wccp module and kernal rebuilding. I know
>>> this is
>>> not squid issue, or at least I do not think so, so an Fedora help
>>> you can
>>> give would be appreciated. I have also followed the book from
>>> O'reilly
>>> called squid. I am going to dig a lil deeper on the server side
>>> doing dumps
>>> to see what else I find. The only other clue I see is the packet
>>> going from
>>> the server to the ASA:
>>> ICMP Destinatoon unreachable (Host administratively prohibited)
>>
>> Hmm, indicates a firewall problem. Possibly preventing part of the
>> traffic between squid and the client or between squid and server.
>>
>> Worth getting a low-level trace and finding out whats generating it.
>>
>>
>> Amos
>> --
>> Please be using
>> Current Stable Squid 2.7.STABLE7 or 3.0.STABLE20
>> Current Beta Squid 3.1.0.14
>
> Amos,
>
> The "10/100" statement on the website I am unsure of as I used it
> more for the configuration then his network diagram. To explain
> further I did compare the 2 network layouts but I strictly wanted to
> see his config vs mine. On my network all clients and servers
> eventually go through the ASA there is no way around it. I am not
> sure at this current time how I can not involve the ASA as the
> server squid is on is used now for other functionalities.
>
> I will try to run some packet stuff by the end of day and report
> back my findings. Anything specific?
>
> Thanks
>
>
>
Amos,

Here is the outputs

[root_at_cache Administrator]# tcpdump -n -i eth0 ip proto gre
tcpdump: verbose output suppressed, use -v or -vv for full protocol
decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
14:43:57.241246 IP 192.168.16.1 > 192.168.16.217: GREv0, length 56:
gre-proto-0x883e
14:43:57.674427 IP 192.168.16.1 > 192.168.16.217: GREv0, length 56:
gre-proto-0x883e
14:43:57.737913 IP 192.168.16.1 > 192.168.16.217: GREv0, length 56:
gre-proto-0x883e
14:43:58.089966 IP 192.168.16.1 > 192.168.16.217: GREv0, length 56:
gre-proto-0x883e
14:43:58.942476 IP 192.168.16.1 > 192.168.16.217: GREv0, length 56:
gre-proto-0x883e
14:43:59.705716 IP 192.168.16.1 > 192.168.16.217: GREv0, length 56:
gre-proto-0x883e
14:44:00.194796 IP 192.168.16.1 > 192.168.16.217: GREv0, length 56:
gre-proto-0x883e

GRE is working

netstat -s | grep unknown
3 packets to unknown port received.

That seems to be ok, number does not increase

netstat -i
Kernel Interface table
Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-
DRP TX-OVR Flg
eth0 1500 0 846693 0 0 0 70715 0
0 0 BMRU
gre1 1476 0 0 0 0 0 0 0
0 0 OPRU
lo 16436 0 103 0 0 0 103 0
0 0 LRU

any ideas?

Thanks