Re: [squid-users] "Quadruple" memory usage with squid

Linda Messerschmidt wrote:
> Just wanted to update everyone on this.
>
> The log rotation in the child process solution works very well and
> prevents us from needing to "-k rotate" every day, which is great. No
> more daily downtime. That was a wonderful suggestion, and I'm very
> grateful.
>
> However, the primary memory issue remains unsolved. I know a lot of
> squid developers don't take this seriously because they think it's
> just "pretend" virtual address space that's not backed by anything.
> (Similar to, for example, what rpc.statd does on FreeBSD.) But for
> the last several days, we've been running with cache_mem set to 15GB
> on the machine with 24GB, which also has 16GB of swap:
>
> pid 75086 (squid), uid 100, was killed: out of swap space
> pid 81776 (squid), uid 100, was killed: out of swap space
>
> At no time does squid's report of its own memory usage approach
> anything close to 40GB. If this were "empty" address space, it would
> not be getting paged out, and the VM subsystem would not be killing
> it. I am more convinced than ever that there is something seriously
> wrong with squid's memory allocation/usage. (I have web-searched this
> extensively and did find one person who suggested the problem was
> specific to squid using FreeBSD's malloc, but there aren't a lot of
> 64-bit malloc alternatives as far as I know.)
>
> Where do I go from here?
>
> Thanks!

I'd like to assure you we are taking this seriously. Henrik was just
pointing out to us that the superficially perfect vfork() is not as
reliably supported by Linux kernel groups as we would like.

I see the FreeBSD group have declared on Nov 13, 2009 to support it
long-term. Thats good enough for me if it actually works.

Did you get the patch and build instructions I sent you for testing
vfork() and see if it resolved the issues?

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE7 or 3.0.STABLE20
   Current Beta Squid 3.1.0.15