Re: Much wanted feature.

Bill Wichers wrote:

> Oh man... One would need a LOT of disk space for that. Even massive 20+ GB
> caches can't manage to cache a big enough chunk of the Internet to exceed
> about 20-30% hits (well, 20-30% hits for their sibling caches). I have no
> idea how much disk you would need to cache a significant chunk of the
> Internet, but I would guess you'd need well over 100 GB.

I'd suspect you missed by an order of magnitude. I'd guess at 1000 to 2000GB,
myself. This is (of course) a Wild-Assed-Guess(tm). YMMV.

> As for mounting a cache_dir remotly, I think others have had only bad luck
> with this. My guess is that it would be very slow, although I havn't ever
> tried it.

Content-type: verbiage/waffle ; boundary=end-of-waffle

Tried it via NFS over 10MB ethernet. Acceptable. No noticeable delays from the
client end..I mean, heck...The client's probably connected at 28.8K if you're
an ISP. That's about 3.6Kb/s (taking compression into account) at peak. You
don't need a whole lot of responsiveness out of a network mounting to keep up
with those. Even a lot of those.

Nevertheless, we seem to get the best performance by 'pipelining' caches.
client -> CacheA -> CacheB -> CacheC -> world, with (for example) cacheB
pulling from siblings. You can gang them up like this telling A to cache below
a minimum size, B to cache larger things, and C to cache the largest stuff.

There are obvious performance considerations involved here, as well as the
trouble of scads of ICP traffic choking up a broadcast network. If those are
not issues, and cache-space and minimisation of bandwidth _are_ very important
for you, then this sort of setup has it's advantages. Objects tend to settle
out graded for size (although it'd be nice to have a "Don't store anything
smaller than X KB" option to allow better tuning).

Personally, I feel that there must be a better way to do things. But then, I'd
be keen on seeing a persistant TCP connections between
siblings/parents/children down which ICP requests can be funneled. I _like_
the whole ICP thing...I just hate seeing it fall apart just because things get
busy and the UCP/ICP packets get lost.--end-of-waffle

D

--
Note to evil sorcerers and mad scientists: don't ever, ever summon powerful
demons or rip holes in the fabric of space and time. It's never a good idea.
ICQ UIN: 3225440