Re: [squid-users] top reports twice memory as much as Total in mgr:mem

It is strange that memPoolAlloc is less than memPoolFree; might be
another overflow case.

2010/10/21 Kaiwang Chen <kaiwang.chen_at_gmail.com>:
> Amos,
>
> The cache size is configured around 128GB, as reported by mgr:config:
> cache_dir aufs /export/squid/cache 131072 32 256 IOEngine=DiskThreads
> cache_swap_low 90
> cache_swap_high 95
> cache_mem -1073741824 bytes
> memory_replacement_policy lru
> cache_replacement_policy lru
> memory_pools on
> memory_pools_limit 5242880 bytes
>
> In a VIRT=11.5g  RES=11g instance, mgr:info reports something like:
>        Process Data Segment Size via sbrk(): 1020120 KB
>        Maximum Resident Size: 47922032 KB
>        Storage Swap size:      34233780 KB
>        Storage Swap capacity:  25.5% used, 74.5% free
>        Storage Mem size:       3375668 KB
>        Storage Mem capacity:   46.4% used, 53.6% free
>        memPoolAlloc calls: 21345922346
>        memPoolFree calls:  21401173704
>    Internal Data Structures:
>        451308 StoreEntries
>         48956 StoreEntries with MemObjects
>         48927 Hot Object Cache Items
>        451254 on-disk objects
>
> I guess it is not a mem_pools_limit issue, since memory_pools_limit is
> the default 5MB, which means the malloc library will handle memory
> above 5MB. Not sure whether the malloc library returns freed memory to
> OS or not (guess not).
>
> So the problem might be a rush of requests bring the host to its
> knees, and it won't have a chance to stand on its feet again before
> the squid restarts? Lots of swap in/out and few memory for filesystem
> cache when squid holds above 90 percent of system. The memory
> starvation simply exacerbates system load.
>
> Any solution?
>
> Thanks,
> kc
>
> 2010/10/19 Amos Jeffries <squid3_at_treenet.co.nz>:
>> On 19/10/10 22:19, Kaiwang Chen wrote:
>>>
>>> Hello,
>>>
>>> In a 3.1.6 squid installation on a CentOS 5.5 x86 host with customized
>>> kernel, top reports
>>>
>>> VERT 19.8g, and RES 14g
>>>
>>> while mgr:mem reports
>>> Total Allocated 7.25g (96.4% of which is mem_node) high 7.27g,  In Use
>>> 7.25g %alloc 99.957
>>>
>>> In squid.conf, the following is explicitly configured(mem_pool is
>>> default):
>>> cache_mem 7168 MB
>>> maximum_object_size 51200 KB
>>> maximum_object_size_in_memory 1024 KB
>>>
>>> Probably memory leak.  However, it is not reliably reproducible.
>>> Anyway, a restart will return memory to OS.
>>
>> see question 4 of the memory FAQ:
>> http://wiki.squid-cache.org/SquidFaq/SquidMemory
>>
>>>
>>> Additionally, mgr reports negative values:
>>>
>>
>> Bit of 32-bit signed wrap in the display. Annoying but nothing to worry
>> overly much about. It's been fixed in later releases.
>>
>> Amos
>> --
>> Please be using
>>  Current Stable Squid 2.7.STABLE9 or 3.1.8
>>  Beta testers wanted for 3.2.0.2
>>
>
Received on Thu Oct 21 2010 - 15:21:35 MDT