[squid-users] Réf. : [squid-users] cache dir limits

Hello Victor,

No answers from me, just questions ;-)
you said that your cache became too big, and I see that it is something
like 100G large. Actually, I'm interested here to know how you see that
your cache dir was too slow, how many users you have, and more generaly, I
was wondering if a big cache is good for performance. Does anybody have
some clue about that?

Thanks

                      Victor Ivanov
                      <v0rbiz@icon.bg> Pour : squid-users@squid-cache.org
                                                cc :
                      05/12/2003 13:57 Objet : [squid-users] cache dir limits

Hello,

I'm running squid 2.5 stable 3. Actualy it should be stable4 (freebsd
squid port 2.5_4), it uses lots of patches.
Anyway, the behavior is the same with squid 2.4.

The problem is my cache dir became too big and now it takes about
five minutes to write the cache. In the meantime there's no service.

The other problem is that when squid starts it takes too long to
rebuild the store, and while it rebuilds it there's heavy disk usage.

I'm asking if I've reached the limits of squid on this hardware.
CPU: AMD XP 2200+ (CPU load is low)
RAM: 512M + 1G swap
HDD: 2x Maxtor UDMA133 using RAID0 (that obviously was a mistake)

It seems this raid has awfully low speed.

Anyway, here's some from the log:
(some lines were omitted for readability)

2003/12/05 12:14:50| storeDirWriteCleanLogs: Starting...
2003/12/05 12:14:57| 65536 entries written so far.
.....
2003/12/05 12:16:06| 720896 entries written so far.
.....
2003/12/05 12:20:03| 3014656 entries written so far.
2003/12/05 12:22:53| Finished. Wrote 4594867 entries.
2003/12/05 12:22:53| Took 482.9 seconds (9515.9 entries/sec).

(That was the last shutdown from 2.4STABLE7, after this
 it's 2.5STABLE3 with the patches)

2003/12/05 12:22:54| Store rebuilding is 0.1% complete
2003/12/05 12:23:09| Store rebuilding is 15.9% complete
2003/12/05 12:23:24| Store rebuilding is 30.3% complete
2003/12/05 12:23:39| Store rebuilding is 42.3% complete
2003/12/05 12:23:54| Store rebuilding is 53.9% complete
2003/12/05 12:24:09| Store rebuilding is 65.1% complete
2003/12/05 12:24:24| Store rebuilding is 75.6% complete
2003/12/05 12:24:39| Store rebuilding is 83.3% complete
2003/12/05 12:24:58| Store rebuilding is 84.9% complete
2003/12/05 12:25:20| Store rebuilding is 85.0% complete
2003/12/05 12:25:36| Store rebuilding is 85.3% complete
2003/12/05 12:25:52| Store rebuilding is 85.9% complete
2003/12/05 12:26:08| Store rebuilding is 86.1% complete
2003/12/05 12:26:26| Store rebuilding is 86.3% complete
2003/12/05 12:26:43| Store rebuilding is 86.4% complete
2003/12/05 12:26:59| Store rebuilding is 86.5% complete
2003/12/05 12:27:15| Store rebuilding is 86.6% complete
2003/12/05 12:27:30| Store rebuilding is 86.6% complete
.....and then it gets even slower, while the disk is
at its limits... actually the array is at its limits...
2003/12/05 12:30:07| Store rebuilding is 87.4% complete
.....
2003/12/05 12:54:06| Store rebuilding is 90.0% complete
.....
2003/12/05 13:51:52| Store rebuilding is 96.9% complete
FATAL: xcalloc: Unable to allocate 1 blocks of 4104 bytes!

And then it starts all over. Now this xcalloc error is
probably due to ulimit configuration and I'm going to
extend it. This is not the problem I'm asking about.

I know I misconfigured the whole thing. Any pointers
what should I fix? Here are some lines from the config:

cache_mem 384 MB
maximum_object_size 32768 KB
maximum_object_size_in_memory 32 KB
cache_replacement_policy heap LFUDA
cache_dir ufs /usr/local/squid/cache 100000 128 256

The rest is about the default. There are four delay pools,
and some ...long regex acl's, but the CPU's fast enough :)

P.S. The last used dir in the cache is 46/1F
P.P.S. I don't use diskd anymore. It's more stable this way.
Actually, I don't remember why I turned to direct ufs access,
but it saved me at the time :)

Regards
(See attached file: attc7d17.dat)