Re: [squid-users] "Quadruple" memory usage with squid

Just wanted to update everyone on this.

The log rotation in the child process solution works very well and
prevents us from needing to "-k rotate" every day, which is great. No
more daily downtime. That was a wonderful suggestion, and I'm very
grateful.

However, the primary memory issue remains unsolved. I know a lot of
squid developers don't take this seriously because they think it's
just "pretend" virtual address space that's not backed by anything.
(Similar to, for example, what rpc.statd does on FreeBSD.) But for
the last several days, we've been running with cache_mem set to 15GB
on the machine with 24GB, which also has 16GB of swap:

pid 75086 (squid), uid 100, was killed: out of swap space
pid 81776 (squid), uid 100, was killed: out of swap space

At no time does squid's report of its own memory usage approach
anything close to 40GB. If this were "empty" address space, it would
not be getting paged out, and the VM subsystem would not be killing
it. I am more convinced than ever that there is something seriously
wrong with squid's memory allocation/usage. (I have web-searched this
extensively and did find one person who suggested the problem was
specific to squid using FreeBSD's malloc, but there aren't a lot of
64-bit malloc alternatives as far as I know.)

Where do I go from here?

Thanks!
Received on Tue Dec 01 2009 - 14:45:16 MST