Storage Filesystems

Collaboration diagram for Storage Filesystems:

Modules
	AUFS Storage Filesystem (UFS Based)
	diskd Storage Filesystem (UFS Based)
	UFS Storage Filesystem
	swap.state File Structure

Classes
class	Rock::StoreFileSystem
	, More...
class	Fs::Ufs::StoreFSufs< TheSwapDir >
class	StoreFileSystem

Detailed Description

Introduction

Traditionally, Squid has always used the Unix filesystem (UFS) to store cache objects on disk. Over the years, the poor performance of UFS has become very obvious. In most cases, UFS limits Squid to about 30-50 requests per second. Our work indicates that the poor performance is mostly due to the synchronous nature of open() and unlink() system calls, and perhaps thrashing of inode/buffer caches.

We want to try out our own, customized filesystems with Squid. In order to do that, we need a well-defined interface for the bits of Squid that access the permanent storage devices. We also require tighter control of the replacement policy by each storage module, rather than a single global replacement policy.

Build structure

The storage types live in src/fs/. Each subdirectory corresponds to the name of the storage type. When a new storage type is implemented configure.ac must be updated to autogenerate a Makefile in src/fs/foo/ from a Makefile.in file.

TODO: DOCS: add template addition to configure.ac for storage module addition. TODO: DOCS: add template Makefile.am for storage module addition.

configure will take a list of storage types through the –enable-store-io parameter. This parameter takes a list of space separated storage types. For example, –enable-store-io="ufs aufs" .

Each storage type must create an archive file in src/fs/foo/.a . This file is automatically linked into squid at compile time.

Each storage filesystem must inherit from StoreFileSystem and provide all virtual function hooks for squid to operate with.

Operation of a Storage Module

Squid understands the concept of multiple diverse storage directories. Each storage directory provides a caching object store, with object storage, retrieval, indexing and replacement.

Each open object has associated with it a storeIOState object. The storeIOState object is used to record the state of the current object. Each storeIOState can have a storage module specific data structure containing information private to the storage module.

Each SwapDir has the concept of a maximum object size. This is used as a basic hint to the storage layer in first choosing a suitable SwapDir. The checkobj function is then called for suitable candidate SwapDirs to find out whether it wants to store a given StoreEntry. A maxobjsize of -1 means 'any size'.