[squid-users] WCCP2+tproxy and Cisco LNS

Hello List

My first post here but have been using squid for a while.

Trying to implement a transparent proxy for some of our DSL users.
I've setup a test LNS on a Cisco 2821, the connections come in via the
standard PPPoA and are sent via L2TP from the provider. Standard stuff
which works. WCCPv2 is setup and working OK, I can see the packets
arriving on the box. The trouble I'm having is that the packets are
arriving on the squid box but don't seem to be diverted into squid
daemon.

Details

LNS = Cisco 2821, (C2800NM-SPSERVICESK9-M), Version 12.4(3b). LNS is
acting as a router on a stick (1 active interface)

(IP's changed to protect the guilty. NAT is not used in this network)

LNS IP = 172.16.254.253 /30
LNS GW = 172.16.254.254 /30
DSL user IP = 10.10.254.254 /32

interface GigabitEthernet0/0
 description TEST-LNS
 ip address 172.16.254.253 255.255.255.252
 ip wccp redirect exclude in
 duplex auto
 speed auto
 no cdp enable
 no mop enabled

interface Virtual-Template99
 ip unnumbered GigabitEthernet0/0
 ip wccp 80 redirect in
 ip wccp 90 redirect out
 peer default ip address pool dsl
 ppp authentication pap chap dsl
 ppp accounting dsl
end

ip wccp web-cache
ip wccp 80 redirect-list 100
ip wccp 90 redirect-list 100

access-list 100 permit ip 10.0.0.0 0.255.255.255 any
access-list 100 permit ip any 10.0.0.0 0.255.255.255

test-lns#sh ip wccp
Global WCCP information:
    Router information:
        Router Identifier: 172.16.254.253
        Protocol Version: 2.0

    Service Identifier: web-cache
        Number of Cache Engines: 0
        Number of routers: 0
        Total Packets Redirected: 1180
        Process: 0
        Fast: 0
        CEF: 1180
        Redirect access-list: -none-
        Total Packets Denied Redirect: 0
        Total Packets Unassigned: 48
        Group access-list: -none-
        Total Messages Denied to Group: 0
        Total Authentication failures: 0
        Total Bypassed Packets Received: 0

    Service Identifier: 80
        Number of Cache Engines: 1
        Number of routers: 1
        Total Packets Redirected: 1003
        Process: 0
        Fast: 0
        CEF: 1003
        Redirect access-list: 100
        Total Packets Denied Redirect: 0
        Total Packets Unassigned: 12
        Group access-list: -none-
        Total Messages Denied to Group: 0
        Total Authentication failures: 0
        Total Bypassed Packets Received: 0

    Service Identifier: 90
        Number of Cache Engines: 1
        Number of routers: 1
        Total Packets Redirected: 0
        Process: 0
        Fast: 0
        CEF: 0
        Redirect access-list: 100
        Total Packets Denied Redirect: 0
        Total Packets Unassigned: 11
        Group access-list: -none-
        Total Messages Denied to Group: 0
        Total Authentication failures: 0
        Total Bypassed Packets Received: 0

Squid box = Debian Wheezy, iptables v1.4.14, Squid Cache: Version
3.1.20 (also one interface)

root_at_squid:~# uname -a
Linux squid 3.2.0-3-amd64 #1 SMP Thu Jun 28 09:07:26 UTC 2012 x86_64 GNU/Linux

eth0 = 172.16.254.2 /30
gw = 172.16.254.1 /30
wccp0 = 172.16.254.2 /32
rp_filter = disabled
forwarding = enabled

egrep -v "^#|^$" /etc/squid3/squid.conf
acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
acl localnet src 10.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localnet
http_access allow localhost
http_access deny all
http_port 3128
http_port localhost:3129 tproxy tcpkeepalive=60,10,6
disable-pmtu-discovery=transparent
hierarchy_stoplist cgi-bin ?
coredump_dir /var/spool/squid3
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
always_direct allow all
wccp2_router 172.16.254.253
wccp2_forwarding_method gre
wccp2_return_method gre
wccp2_service dynamic 80
wccp2_service_info 80 protocol=tcp flags=src_ip_hash priority=240 ports=80
wccp2_service dynamic 90
wccp2_service_info 90 protocol=tcp flags=dst_ip_hash,ports_source
priority=240 ports=80

iptables rules

# Generated by iptables-save v1.4.14 on Fri Jul 13 12:18:56 2012
*mangle
:PREROUTING ACCEPT [842:129900]
:INPUT ACCEPT [1741:198753]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [1305:232713]
:POSTROUTING ACCEPT [1305:232713]
:DIVERT - [0:0]
-A PREROUTING -p tcp -m socket -j DIVERT
-A PREROUTING -i wccp0 -p tcp -m tcp --dport 80 -j TPROXY --on-port
3129 --on-ip 0.0.0.0 --tproxy-mark 0x1/0x1
-A DIVERT -j MARK --set-xmark 0x1/0xffffffff
-A DIVERT -j ACCEPT
COMMIT
# Completed on Fri Jul 13 12:18:56 2012
# Generated by iptables-save v1.4.14 on Fri Jul 13 12:18:56 2012
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [9:1268]

root_at_squid:~# ip rul show
0: from all lookup local
32765: from all fwmark 0x1 lookup 100
32766: from all lookup main
32767: from all lookup default

Packet traces

traffic from dsl connection directed via wccp to squid

root_at_squid:~# !tcpdump
tcpdump -niwccp0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wccp0, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
12:19:54.287278 IP 10.10.254.254.46360 > 80.239.148.170.80: Flags [S],
seq 975284290, win 13600, options [mss 1360,sackOK,TS val 2009935 ecr
0,nop,wscale 4], length 0
12:19:54.445694 IP 10.10.254.254.46361 > 80.239.148.170.80: Flags [S],
seq 1791319806, win 13600, options [mss 1360,sackOK,TS val 2009975 ecr
0,nop,wscale 4], length 0
12:19:55.285531 IP 10.10.254.254.46360 > 80.239.148.170.80: Flags [S],
seq 975284290, win 13600, options [mss 1360,sackOK,TS val 2010185 ecr
0,nop,wscale 4], length 0
12:19:55.445826 IP 10.10.254.254.46361 > 80.239.148.170.80: Flags [S],
seq 1791319806, win 13600, options [mss 1360,sackOK,TS val 2010225 ecr
0,nop,wscale 4], length 0

I have followed several guides on the wiki, tried different distro's,
DNAT without Tproxy and now with Tproxy. Any pointers on where I'm
going wrong will be helpful as I've been at this for 3 days now. If I
set this up in a "normal" network with LAN, WAN and squid being the
gateway device it works in non-transparent and transparent modes. This
feels like a issue with the DSL connections being rejected by squid or
iptables but I'm at a loss to explain where or how.

When tested using the DNAT method the packets were routed via the
squid box although still bypassed the squid daemon, the packets would
return from the webserver but were then dropped. Using the Tproxy
method shows the packets never getting to squid and not leaving the
box to the webserver.

Do I require multiple interfaces on the squid box and maybe use
ebtables or is what I'm trying to achieve possible on 1 interface ?

Thanks for reading

Wayne
Received on Fri Jul 13 2012 - 11:34:02 MDT