Re: [squid-users] Repost: Squid 3.1.x uses considerably more memory than 2.7.x or 3.0.x

Ralf Hildebrandt wrote:
> Back in 2009 I was still using 2.7.x:
>
> I switched from 2.7.x to 3.1.0.14 yesterday. Today, my proxy would
> encounter an out of memory condition.
>
> cache_mem was EXPLICITLY set to 128 MB, thus the new default of 256 MB
> would not be relevant.
>
> Attached is a picture of squidstats that show the memory consumption
> before and after the switch.
>
> Note the peak on the right edge of the graph. In the end squid was
> using 1.4GB of RAM.
>
> And a retry in 2010:
>
> I switched from 3.0.x to 3.1.0.x yesterday.
>
> Find some output from
> % squidclient mgr:mem for 3.0
> here http://www.arschkrebs.de/bugs/mgr-mem.3.0.txt.gz
> and from
> % squidclient mgr:mem for 3.1
> here http://www.arschkrebs.de/bugs/mgr-mem.3.1.txt.gz
>
> What I did:
>
> 3.0 was running for a whole day, I then used
> squidclient mgr:mem
>
> Then I replaced 3.0 with 3.1 and restarted.
>
> I then let it run for about 15 minutes and issued
> squidclient mgr:mem
> several times
>
> Can anybody see where the excessive memory usage comes from?
>

3.1 betas had problems with a few things that show the same symptoms as
leaks. But were not real leaks, so the memory tools don't detect them well.

  * on 64-bit boxes due to non-aligned cache_mem chunk sizes being
allocated 2 pages each. Squid only counted the one allocated, so sys
shows 2x usage what squid does.

  * external_acl_type internal result cache not being capped. Use of the
cache=COUNT parameter is needed on high-traffic proxies.

  * still open: bug 2305; auth has some logic errors creating circular
reference loops. Which could hold onto credentials way too long, or
cause real leaks to accumulate under certain request types.

  * still open; Squid's custom memory manager (mempools) is "broken" for
some reason on 64-bit. Not having looked at the one deeply yet I'm not
sure what the impact is.

Can you try your tests on the 3.1.1 (or 3.1.2 in a few days) release
please and see if it was one of the solved issues?

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.1