[squid-users] cache hit 100%

Hi...
we are running squid 2.6 stable 17 that have been built within the last
6 months.
The problem we are experiencing is that one of our servers has hit 100%
on the proxycache partition. It is configured in its own filesystem, it
shares with nothing... it has a partition of 36Gb and squid is
configured in squid.conf to use 28Gb. Once it has got to 28Gb we see the
following messages in cache.log (some entries have been deleted,
hopefully I have the relevant ones below)

Rebuilding storage in /proxycache (DIRTY)
Store rebuilding is 0.1% complete
2009/01/06 12:36:11| diskHandleWrite: FD 536: disk write error: (28) No
space left on device
2009/01/06 12:36:11| storeAufsWriteDone: got failure (-6)
2009/01/06 12:36:11| storeSwapOutFileClosed: dirno 0, swapfile 00001392,
errflag=-6
(28) No space left on device
2009/01/06 12:36:29| WARNING: newer swaplog entry for dirno 0, fileno
000015D8
2009/01/06 12:44:13| Store rebuilding is 92.2% complete
2009/01/06 12:44:15| WARNING: Disk space over limit: 13552 KB > 13512 KB
2009/01/06 12:44:26| WARNING: Disk space over limit: 13552 KB > 13512 KB
FATAL: xcalloc: Unable to allocate 1 blocks of 28 bytes!
Squid Cache (Version 2.6.STABLE17): Terminated abnormally.
CPU Usage: 531.367 seconds = 482.302 user + 49.066 sys
Maximum Resident Size: 0 KB
Page faults with physical i/o: 0
Memory usage for squid via mallinfo():
total space in arena: -307440 KB
Ordinary blocks: -308814 KB 87176 blks
Small blocks: 0 KB 0 blks
Holding blocks: 25248 KB 6 blks
Free Small blocks: 0 KB
Free Ordinary blocks: 1373 KB
Total in use: -283566 KB 100%
Total free: 1373 KB 0%
2009/01/06 12:44:27| Not currently OK to rewrite swap log.
2009/01/06 12:44:27| storeDirWriteCleanLogs: Operation aborted.
2009/01/06 12:44:38| Store rebuilding is 0.0% complete

And this process repeats continually. We have taken it out of service,
but previously to that it did look like it was processing requests in
the cache.. ie TCP_HIT's were appearing in the log. But surely what is
appearing in the cache log is incorrect.

Squid is configured with the default cache_swap_low and cache_swap_high
and cache_mem 1024 MB, maximum_object_size 16384 KB
maximum_object_size_in_memory 64 KB
cache_replacement_policy heap GDSF
memory_replacement_policy heap GDSF
cache_dir aufs /proxycache 28000 256 256

Can anyone advise why the proxycache hits 100% and it appears to have
problems when it does?

Thanks, K.

Kev Shurmer
Network Analyst - TS Data Networks
AXA Technology Services
kev.shurmer_at_axa-tech.com
Tel. : +44 1 253 68 4652 - Mob. : +44 7974 83 0090

 
Please consider the environment before printing this message

This email originates from AXA Technology Services UK Limited (reg.
no. 1854856) which has its registered office at 5 Old Broad Street,
London EC2N 1AD, England.

This message and any files transmitted with it are confidential and
intended solely for the individual or entity to whom they are addressed.
If you have received this in error, you should not disseminate or copy
this email. Please notify the sender immediately and delete this email
from your system.

Please also note that any opinions presented in this email are solely
those of the author and do not necessarily represent those of The AXA
UK Plc Group.

Email transmission cannot be guaranteed to be secure, or error free as
information could be intercepted, corrupted, lost, destroyed, late in
arriving or incomplete as a result of the transmission process. The
sender therefore does not accept liability for any errors or omissions in
the contents of this message which arise as a result of email
transmission.

Finally, the recipient should check this email and any attachments for
viruses. The AXA UK Plc Group accept no liability for any damage
caused by any virus transmitted by this email.
Received on Wed Jan 07 2009 - 08:28:32 MST