Re: [squid-users] High CPU utilization

Amos Jeffries-2 wrote:
>
> On Mon, 27 Jul 2009 12:14:29 -0700 (PDT), jotacekm <minuzzo_at_viaip.com.br>
> wrote:
>> Hello.
>> Recently we have added a lot more clientes behind a squid proxy, and now
>> cpu
>> utilizitaion is usually 70-95%. The processor is a intel dual core 2160
> @
>> 1.80GHz. Users started complaing about the speed on accessing pages, and
>> the
>> link is fine.
>>
>> Here is squidclient mgr:info:
>>
>> Squid Object Cache: Version 2.6.STABLE5
>> Start Time: Fri, 24 Jul 2009 20:19:16 GMT
>> Current Time: Mon, 27 Jul 2009 19:09:07 GMT
>> Connection information for squid:
>> Number of clients accessing cache: 1561
>> Number of HTTP requests received: 8590404
>> Number of ICP messages received: 0
>> Number of ICP messages sent: 0
>> Number of queued ICP replies: 0
>> Number of HTCP messages received: 0
>> Number of HTCP messages sent: 0
>> Request failure ratio: 0.00
>> Average HTTP requests per minute since start: 2021.3
>
> 2k req/min is getting close to what people report for saturation of older
> 2.6 forward proxies.
> First performance improvement will be simply an upgrade to 2.7.
>
>
>> Average ICP messages per minute since start: 0.0
>> Select loop called: 122560206 times, 2.081 ms avg
>> Cache information for squid:
>> Request Hit Ratios: 5min: 23.0%, 60min: 21.9%
>> Byte Hit Ratios: 5min: 13.3%, 60min: 14.7%
>> Request Memory Hit Ratios: 5min: 16.3%, 60min: 17.0%
>> Request Disk Hit Ratios: 5min: 26.3%, 60min: 29.9%
>> Storage Swap size: 4609784 KB
>> Storage Mem size: 65692 KB
>> Mean Object Size: 15.67 KB
>> Requests given to unlinkd: 455490
>> Median Service Times (seconds) 5 min 60 min:
>> HTTP Requests (All): 1.24267 1.46131
>> Cache Misses: 1.54242 1.81376
>> Cache Hits: 0.28853 0.30459
>> Near Hits: 1.31166 1.62803
>> Not-Modified Replies: 0.23230 0.23230
>> DNS Lookups: 0.29097 0.31806
>> ICP Queries: 0.00000 0.00000
>> Resource usage for squid:
>> UP Time: 254991.358 seconds
>> CPU Time: 56029.670 seconds
>> CPU Usage: 21.97%
>> CPU Usage, 5 minute avg: 94.38%
>> CPU Usage, 60 minute avg: 93.36%
>> Process Data Segment Size via sbrk(): 193008 KB
>> Maximum Resident Size: 0 KB
>> Page faults with physical i/o: 70
>> Memory usage for squid via mallinfo():
>> Total space in arena: 193008 KB
>> Ordinary blocks: 179871 KB 6058 blks
>> Small blocks: 0 KB 0 blks
>> Holding blocks: 1080 KB 2 blks
>> Free Small blocks: 0 KB
>> Free Ordinary blocks: 13136 KB
>> Total in use: 180951 KB 93%
>> Total free: 13136 KB 7%
>> Total size: 194088 KB
>> Memory accounted for:
>> Total accounted: 111416 KB
>> memPoolAlloc calls: 973288506
>> memPoolFree calls: 972077571
>> File descriptor usage for squid:
>> Maximum number of file descriptors: 4096
>> Largest file desc currently in use: 1604
>> Number of file desc currently in use: 1308
>> Files queued for open: 0
>> Available number of file descriptors: 2788
>> Reserved number of file descriptors: 100
>> Store Disk files open: 30
>> IO loop method: epoll
>> Internal Data Structures:
>> 300182 StoreEntries
>> 14733 StoreEntries with MemObjects
>> 14414 Hot Object Cache Items
>> 294106 on-disk objects
>>
>> And here is part of of squid.conf:
>>
>> http_port 3128
>> visible_hostname xxx
>> hierarchy_stoplist cgi-bin ?
>> acl QUERY urlpath_regex cgi-bin \?
>> cache deny QUERY
>
> _after_ upgrading to the most recent 2.6 or a 2.7. Remove the QUERY
> entries
> above.
>
>> acl apache rep_header Server ^Apache
>> broken_vary_encoding allow apache
>> access_log /var/log/squid/access.log squid
>> cache_store_log none
>> hosts_file /etc/hosts
>>
>>
>> #
>>
> ----------------------------------------------------------------------------------------------------------
>> cache_mem 64 MB
>> cache_dir ufs /var/spool/squid 5000 50 256
>
> #1 change that to aufs if using a Linux or variants. diskd if using BSD
> or
> variants. Change to Linux or BSD if using Windows.
>
> #2 up the cache size as much as you can. Local objects are much faster to
> retrieve than external ones.
> The more local objects you can store the faster Squid responds.
> From my experience its around an order of magnitude or more req/sec
> handled
> from cache as from network.
>
> Memory cache is better for speed than disk, but can be lost more easily
> and
> you can get bigger cache total spending RAM on indexes and disk on object
> storage.
>
> After the upgrade to 2.7 you have access to a stable COSS system which is
> very much more efficient for the small files. It's worth a try.
>
>> cache_replacement_policy heap LFUDA
>> maximum_object_size 51200 KB
>
> There is good evidence that the web now has a lot of traffic in the 2MB to
> 250MB range from multimedia objects.
> This will bear investigating in your traffic and see if the limit above is
> still good.
>
>
>> maximum_object_size_in_memory 64 KB
>> memory_replacement_policy heap GDSF
>> logfile_rotate 3
>>
>>
>> #
>>
> ----------------------------------------------------------------------------------------------------------
>> refresh_pattern ^ftp: 1440 20% 10080
>> refresh_pattern ^gopher: 1440 0% 1440
>
> Add:
> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
>
>> refresh_pattern . 0 20% 4320
>> acl all src 0.0.0.0/0.0.0.0
>> acl manager proto cache_object
>> acl localhost src 127.0.0.1/255.255.255.255
>> acl to_localhost dst 127.0.0.0/8
>>
>>
>>
>> max_open_disk_fds 2046
>
> Definitely raise that. Your stats show that many _new_ requests arriving
> every second.
> Each single request eats 2-3 FD and may last from seconds to hours.
>
>>
>> # timeouts
>> connect_timeout 30 seconds
>> shutdown_lifetime 5 seconds
>> forward_timeout 2 minutes
>> pconn_timeout 30 seconds
>
> You _want_ pconn to last a fairly long while. There is no CPU or bandwidth
> overhead.
> Possibly also enabling client and server persistent connections.
>
>> persistent_request_timeout 1 minute
>> request_timeout 2 minute
>>
>> Is there anything that i can do to lower the cpu utilization, or do i
> have
>> to upgrade the hardware?
>
> There are a number of other perfromance optimizations possible within the
> build options and in other squid.conf settings. You don't say what they
> are
> set at, so we can't suggest any right now.
>
> Amos
>
>

Hello,
Thanks for you help. I'll try the modifications you mentioned and report
here if they worked. About the build options, this squid was not compiled,
it was installed via apt-get in a debian etch. And the other squid.conf
options are just acls and allows / denies.

-- 
View this message in context: http://www.nabble.com/High-CPU-utilization-tp24686629p24697787.html
Sent from the Squid - Users mailing list archive at Nabble.com.