Re: [squid-users] Number of Objects Inside aufs Cache Directory

On 2/09/2013 8:40 a.m., Golden Shadow wrote:
> Hello there!
>
> I've read that the number of first level and second level aufs subdirectories should be selected so that the number of objects inside each second level subdirectory should be no more than few hundreds.

Not sure where that came from. "few hundreds" sounds like advice for
working with FAT-16 formatted disks - or worse. Most of the modern
filesystems can handle several thousands easily.

The real reason for these parameters is that some filesystems start
producing errors. For example; Squid tores 2^24 objects in its cache_dir
but - FAT16 fails with more than 2^14 files in one directory - and IIRC
ext2 and/or ext3 starts giving me trouble around 2^16 files in one
directory. I'm not sure about other OS, I've not hit their limits myself.

> In one of the subdirectories of the following cache_dir on my 3.3.8 squid, there is more than 15000 objects! In other subdirectories, there is ZERO objects, is this normal?!

Yes. It depends entirely on how many objects are in the cache. The
earlier fileno entries fill up first, so the directories where those
fileo map to will show lots of files while later ones do not.

NOTE: when changing these L1/L2 values the entire cache_dir fileno->file
mapping gets screwed up. So you need to erase the contents and use an
empty location/directory to build the new structure shape inside.

> cache_dir aufs /mnt/cachedrive2/small 250000 64 256 min-size=32000 max-size=200000

If you want to increase that spread you can change the 64 to 128. It
should halve the number of files in the fullest directory.
With 250GB storing 32KB-200KB sized objects you are looking at a total
of between 1,310,720 and 8,192,000 objects in that particular cache.

Amos
Received on Mon Sep 02 2013 - 00:33:16 MDT