Re: [squid-users] Issue with Digest and Number of Objects

So I found out why I stopped collecting objects in my cache. It was
because I was hitting the low water mark.

Thanks
  Dusten

On Wed, Dec 30, 2009 at 15:21, Dusten Splan <dsplan_at_myyearbook.com> wrote:
> Yes I agree with the major issue is the peers dropping offline.  Is
> there a way to keep this from happing without just turning off digest
> all together.  Being that it's saving a lot of icp queries.
>
> Algorithm usage:
> Cache Digest: 3602076 ( 97%)
> Icp:           111327 (  3%)
> Total:        3713403 (100%)
>
> Thanks
> Dusten
>
> On Tue, Dec 29, 2009 at 22:03, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
>> On Tue, 29 Dec 2009 14:11:44 -0500, Dusten Splan <dsplan_at_myyearbook.com>
>> wrote:
>>> Hi All,
>>>   So I'm having an issue where squid will not cache more then 3843117
>>> objects.
>>
>> NP: 2^24 objects limit per- cache_dir entry.
>>
>>>  Also on this same box we are seeing the traffic dip every
>>> time it rebuilds the digest file.
>>
>> Some delay is expected while the 50MB digest is generated.
>>
>> I think probably the worse effect is that the peers are disappearing. This
>> will most likely increase the network lag time for each request.
>>
>>>
>>> Sample of log file.
>>>
>>> 2009/12/29 04:15:21.745| storeDigestRebuildStart: rebuild #276
>>> 2009/12/29 04:15:21.745| storeDigestCalcCap: have: 38414447, want
>>> 38414447 entries; limits: [8318
>>> 0325, 103975384]
>>> 2009/12/29 04:15:21.745| storeDigestResize: 81437985 -> 83180325;
>>> change: 1742340 (2%)
>>> 2009/12/29 04:15:21.745| storeDigestResize: small change, will not
>> resize.
>>> 2009/12/29 04:15:21.761| storeDigestRebuildStep: buckets: 8388608
>>> entries to check: 16777220
>>> 2009/12/29 04:15:36.125| storeDigestRebuildStep: buckets: 8388608
>>> entries to check: 16777220
>>> 2009/12/29 04:15:50| Detected DEAD Sibling: mypeer01
>>> 2009/12/29 04:15:50| Detected DEAD Sibling: mypeer02
>>> 2009/12/29 04:15:50| Detected DEAD Sibling: mypeer03
>>> 2009/12/29 04:15:50.710| storeDigestRebuildStep: buckets: 8388608
>>> entries to check: 16777220
>>> 2009/12/29 04:15:54.956| storeDigestRebuildFinish: done.
>>> 2009/12/29 04:15:55.010| storeDigestRewrite: start rewrite #276
>>> 2009/12/29 04:15:55.010| storeDigestRewrite: url:
>>> http://example.com/squid-interna
>>> l-periodic/store_digest key: D412EC3C5799ADE5A3CF229E9BD27A16
>>> 2009/12/29 04:15:55.040| storeDigestRewrite: entry expires on -1
>>> (-1262078156)
>>> 2009/12/29 04:15:55.065| storeDigestSwapOutStep: size: 50898741
>>> offset: 0 chunk: 4096 bytes
>>> 2009/12/29 04:15:55.079| storeDigestSwapOutStep: size: 50898741
>>> offset: 4096 chunk: 4096 bytes
>>> 2009/12/29 04:15:55.088| storeDigestSwapOutStep: size: 50898741
>>> offset: 8192 chunk: 4096 bytes
>>> ...
>>> 2009/12/29 04:15:55.372| storeDigestSwapOutStep: size: 50898741
>>> offset: 50888704 chunk: 4096 bytes
>>> 2009/12/29 04:15:55.372| storeDigestSwapOutStep: size: 50898741
>>> offset: 50892800 chunk: 4096 bytes
>>> 2009/12/29 04:15:55.372| storeDigestSwapOutStep: size: 50898741
>>> offset: 50896896 chunk: 1845 bytes
>>> 2009/12/29 04:15:55.372| storeDigestRewriteFinish: digest expires at
>>> -1 (-1262078156)
>>> 2009/12/29 04:16:02| Detected REVIVED Sibling: mypeer01
>>> 2009/12/29 04:16:02| Detected REVIVED Sibling: mypeer03
>>> 2009/12/29 04:16:02| Detected REVIVED Sibling: mypeer02
>>>
>>>
>>> Then start again an hour later.
>>>
>>> 2009/12/29 05:15:54.958| storeDigestRebuildStart: rebuild #277
>>>
>>>
>>> Thanks
>>>   Dusten
>>
>
Received on Wed Dec 30 2009 - 22:29:09 MST