Re: [squid-users] transparent integration with proxy on router

On Tue, 29 Sep 2009 13:22:58 +1300, Todd Nine <todd_at_spidertracks.co.nz>
wrote:
> Hi Amos,
> Here is my squid.conf. I've just used the defaults and added a single
> rule. We're pushing a lot of throughput (several gigs a day). I've
> disabled writing to disk as we actually run from a USB appliance, and
> set the cache size to 1 GB (1024M) of RAM. My main use of squid is not
> caching, but rather http redirection to save us money on our usage fees
> from our ISPs.

In which case you probably do want to look at caching. Since that can have
between 20% and 40% reduction in HTTP traffic going to your ISPs.

What version of squid is this?

>
> Thanks again for the help!
>
> File:
> # Do not edit manually !
> http_port 10.0.1.1:3128
> http_port 10.0.2.1:3128
> http_port 127.0.0.1:80 transparent
> icp_port 0
>
> pid_filename /var/run/squid.pid
> cache_effective_user proxy
> cache_effective_group proxy
> error_directory /usr/local/etc/squid/errors/English
> icon_directory /usr/local/etc/squid/icons
> visible_hostname router
> cache_mgr admin_at_localhost
> access_log /var/squid/log/access.log
> cache_log /var/squid/log/cache.log
> cache_store_log none
> shutdown_lifetime 3 seconds
> # Allow local network(s) on interface(s)
> acl localnet src 10.0.1.0/255.255.255.0 10.0.2.0/255.255.255.0

Please use CIDR masks:
acl localnet src 10.0.1.0/24 10.0.2.0/24

> uri_whitespace strip
>
> cache_dir aufs /var/squid/cache 100 16 256

Huh? you said you disabled writing to disk. That above is using a 100MB
cache on the disk.

To disable disk caching use the 'cache_dir null ' storage type in Squid
older than 3.1, or remove all cache_dir from Squid-3.1+.

> cache_mem 1024 MB
> maximum_object_size 4 KB

1GB worth of 4KB objects is a LOT of objects. If you have the memory to
spare 1GB for caching its probably best to allow moderate sized objects to
be cached in RAM. Setting Max size to 1MB should do. Though depending on
the popularity of video sites with your users they may also benefit from
10MB max object size (video causes a bump at 2MB-8MB apparently).

> minimum_object_size 0 KB
> cache_replacement_policy heap LFUDA
> memory_replacement_policy heap GDSF
> offline_mode off
> dns_children 32

The above is only relevant with the obsolete 'dnsserver' helper. If you are
still using that you would get a great deal of performance boots by
changing to the internal DNS (requires a recompile).

> cache_swap_low 90
> cache_swap_high 95
> acl donotcache dstdomain "/var/squid/acl/donotcache.acl"
> cache deny donotcache
> # No redirector configured
>
>
>
> # Setup some default acls
> acl all src 0.0.0.0/0.0.0.0
> acl localhost src 127.0.0.1/255.255.255.255

Some more CIDR benefits:
acl all src all
acl localhost src 127.0.0.1

> acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 1111
> 3128 1025-65535
> acl sslports port 443 563 1111
> acl manager proto cache_object
> acl purge method PURGE
> acl connect method CONNECT
> acl dynamic urlpath_regex cgi-bin \?
> cache deny dynamic

More speed boost by dropping the above QUERY

> http_access allow manager localhost
>
> http_access deny manager
> http_access allow purge localhost
> http_access deny purge
> http_access deny !safeports
> http_access deny CONNECT !sslports
>
> # Always allow localhost connections
> http_access allow localhost
>
> request_body_max_size 0 KB
> reply_body_max_size 0 allow all
> delay_pools 1
> delay_class 1 2
> delay_parameters 1 -1/-1 -1/-1
> delay_initial_bucket_level 100
> delay_access 1 allow all
>
> # Allow local network(s) on interface(s)
> http_access allow localnet
> # Custom options
> #Set up our ACL for high throughput sites
> acl high_throughput dstdomain .amazonaws.com .rapidshare.com
> #Bind high throughput to the wireless interface
> tcp_outgoing_address 116.90.140.xx high_throughput
>
> # Default block all to be sure
> http_access deny all
>
>
>
> Amos Jeffries wrote:
>> On Tue, 29 Sep 2009 09:32:49 +1300, Todd Nine <todd_at_spidertracks.co.nz>
>> wrote:
>>
>>> Thanks for the help! I read over the rules and it was quite easy to
set
>>>
>>> up what I needed once I had the right directive. I simply set up the
>>> following.
>>>
>>> #Set up our ACL for high throughput sites
>>> acl high_throughput dstdomain .amazonaws.com
>>>
>>> #Bind high throughput to the wireless interface
>>> tcp_outgoing_address 116.90.140.xx high_throughput
>>>
>>> However we're having a side effect issue. Our router box is a bit old
>>> (an old P4), and we can't keep up with the squid demands due to the
>>> number of users with 2 GB of ram. Is there a directive that I can tell

>>> squid not to proxy connections unless they meet the "high_throughput"
>>> acl? I looked and couldn't find any bypass directives that met what I
>>> needed.
>>>
>>> Thanks,
>>> Todd
>>>
>>
>> Once connections have already entered Squid its too late to not send
them
>> to Squid.
>>
>> I have run Squid on P4s routers with 256MB of RAM for hundreds of
domains
>> and dozens of clients without having the box run up much of a sweat.
What
>> is your load (both CPU box load, and visitor rates, bandwidth) like?
>> Also check your other configuration and access controls are using
>> efficient
>> methods, if you don't know what those are already I'm happy to give
>> configs
>> an audit and point things that need adjusting out.
>>
>> Amos
>>
>>
>>> Amos Jeffries wrote:
>>>
>>>> On Mon, 28 Sep 2009 16:21:16 +1300, Todd Nine
<todd_at_spidertracks.co.nz>
>>>> wrote:
>>>>
>>>>
>>>>> Hi all,
>>>>> I'm using squid on a pfSense router we've built. We have 2
>>>>> connections, one we pay for usage (DSL) and one we do not (Wireless).

>>>>>
>>>>> We use Amazon S3 extensively at work. We've been attempting to route

>>>>> all traffic over the wireless via an IP range, but as S3 can change
>>>>>
>> IPs,
>>
>>>>> this doesn't work and we end up with a large bill for our DSL. Is it

>>>>> possible to have squid route connections via a specific interface if
a
>>>>>
>>>>> hostname such as "amazonaws.com" is in the HTTP request header?
>>>>>
>>>>> Thanks,
>>>>> Todd
>>>>>
>>>>>
>>>> Yes you can.
>>>>
>>>> Find an IP assigned to the interface you want traffic to go out. Use
>>>> the
>>>> tcp_outgoing_addr directive and ACLs that match the requests to make
>>>>
>> sure
>>
>>>> all the requests to that domain are assigned that outgoing address.
>>>>
>> Then
>>
>>>> make sure the OS sends traffic from that IP out the right interface.
>>>>
>>>> Amos
>>>>
>>>>
Received on Tue Sep 29 2009 - 00:38:05 MDT