Re: [squid-users] Squid swap.state management

Seann Clark wrote:
> Amos Jeffries wrote:
>> Seann Clark wrote:
>>> All,
>>>
>>>
>>> I have been working over the past few days to manage a rather
>>> large swap file, that isn't reducing when I attempt to rotate the
>>> file. I am sure that I can prune this file's total size down but I am
>>> not sure the best way to tackle this. It doesn't help that usually
>>> the store.log file grows the same way, the swap.state file is at 31G,
>>> and the store log until I removed it, was at 30G's, and killed the
>>> disk space on my
>>
>> Both of these files are more accurately called journals than logs.
>> They are incremental and continuously growing between reset points
>> (rotate, reconf, shutdown).
>>
>> store.log is not usually needed.
>> Unless you have one of the rare tools that need to process it. You can
>> safely turn it off with "cache_store_log none" in squid.conf.
>>
>> swap.state is a little tricky. It should have been replaced with the
>> current index contents on rotate.
>>
>> Squid-2.6 and later you can safely erase the swap.state and Squid will
>> rebuild it clean from the in-memory index on next shutdown,
>> reconfigure, or rotate.
>>
>>> system. This has been a problem I am searched on google for and
>>> haven't found anything that matches what I am looking for, most of
>>> the resource either deal with expanding the system drives out, or
>>> making the disks faster which doesn't help my situation. I am not
>>> sure what to provide in terms of information from my configurations
>>> that would help but will provide what is required.
>>>
>>> As a side note, the process is occasionally coring as well.
>>>
>>
>> Not a good sign. What version and release of Squid is this?
>>
>> Amos
> Squid Cache: Version 2.6.STABLE22
>
>
> I didn't know the none flag worked, guess I was trying the more complex
> things with the store log file. I had deleted the swap.state file and it
> took forever to rebuild (over night processing, though not bad) but
> nothing else bad happened, but I had picked up that it was bad to delete
> the file.

Well, bad in that ... if its missing when Squid starts the cache can
take forever to rebuild the index from raw data. Doing in enormous
amount of disk reading to do so. Which drags down the speed terribly
until its finished.

Doing so to a running Squid is not quite so bad, since the write-out has
to happen for a proper shutdown/restart/reconfigure anyway the
re-generate happens.

The fact that it took all night is very worrying. As I noted Squid needs
to do this write-out whenever the logs are rotated (aka daily or more),
and whenever the service is restarted. Probably caused by either huge
cache (thus huge index), nasty slow disks or worse: both.

> The version of squid is a little lagging because I need to
> upgrade the entire system and haven't had a chance yet, but I also don't
> want to port a bad config over to the new build.
>
> Seann

If you can dig into the cache.log or the core files themselves, for why
the core dumps were happening you might find the problems related.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE7 or 3.0.STABLE20
   Current Beta Squid 3.1.0.14