Re: [squid-users] COSS causing squid Segment Violation on FreeBSD 6.2S (store_io_coss.c)

On Thu, 14 Aug 2008, Adrian Chadd wrote:

> Don't load aio.ko; that way if squid tries using the POSIX AIO
> routines it'll crash.
>
> I should get around to removing those from Squid-2.HEAD (as the
> implementation isn't "good enough" for the current way Squid does disk
> IO) and reintroduce it later in a more sensibly abstracted disk IO
> layer.
>
> Yes, compile in aufs and coss together.

Ok. Thanks.
   I have both compiled in:

# squid -v | xargs -n 1 echo | grep coss
--enable-storeio=ufs diskd null aufs coss

   My COSS storage is definately less than disk size:

# diskinfo amrd1
amrd1 512 73274490880 143114240 8908 255 63

Or 69.880MB.

cache_dir coss /dev/amrd1 65000 max-size=16384 block-size=4096

And AUFS on ufs on two more identical disks:

cache_dir aufs /2 56000 16 256
cache_dir aufs /3 56000 16 256

That seems right?
   one last question.
   I've currently got /etc/libmap.conf to map threads libraries to libthr,
as I've found them more reliable for other servers:

# ldd /usr/local/sbin/squid
/usr/local/sbin/squid:
         libcrypt.so.3 => /lib/libcrypt.so.3 (0xa8163000)
         libmd.so.3 => /lib/libmd.so.3 (0xa817c000)
         libm.so.4 => /lib/libm.so.4 (0xa818b000)
         libpthread.so.2 => /usr/lib/libthr.so.2 (0xa81a1000)
         libc.so.6 => /lib/libc.so.6 (0xa81b4000)

Is that ok or should I just leave it to use libpthread?
   Cheers.

>
>
> Adrian
>
>
> 2008/8/14 Mark Powell <M.S.Powell_at_salford.ac.uk>:
>> On Thu, 14 Aug 2008, Adrian Chadd wrote:
>>
>>> I'm not sure how much time I'm going to have to look at this code right
>>> now.
>>>
>>> My priorities are elsewhere and you're the only person who is
>>> currently reporting this sort of pending relocation issues with COSS
>>> and FreeBSD-6.3.
>>>
>>> I'd suggest patching the source to propagate upward the async IO write
>>> failure (ie - the errno) and log the reason in cache.log. You may find
>>> its something simple like the disk size being different to your
>>> configured cache_dir size.. :)
>>
>> NP. If it's just me then I'll have a play around and see if I can get a
>> config that's stable.
>> Can you confirm that the KLM aio.ko should or should not be loaded?
>> But AUFS and COSS must be both be compiled into squid?
>> Cheers.
>>
>>>
>>>
>>>
>>> 2008/8/13 Mark Powell <M.S.Powell_at_salford.ac.uk>:
>>>>
>>>> Hi,
>>>> Updated to 2.7.STABLE4. Curiosity got the better of me and I re-enabled
>>>> COSS.
>>>> The same failure still seems to be occuring i.e.
>>>> 'storeCossCompletePendingReloc: got failure (-1)' and a segfault on
>>>> FreeBSD
>>>> 6.3-p4.
>>>> Not sure the bug has been looked at since I filed it?
>>>>
>>>> http://www.squid-cache.org/bugs/show_bug.cgi?id=1944
>>>>
>>>> Does this mean COSS always going to be a no-go on FreeBSD?
>>>> Many thanks.
>>>>
>>>> On Thu, 26 Apr 2007, Adrian Chadd wrote:
>>>>>
>>>>> On Thu, Apr 26, 2007, Mark Powell wrote:
>>>>>
>>>>>> It does stop the crashes, but I'm getting a lot of errors, so of which
>>>>>> appear to be new ones:
>>>>>
>>>>> Very eww. Now we need to figure out why the hell its spitting out those
>>>>> pending relocate
>>>>> IO errors as its probably playing havoc with things.
>>>>>
>>>>> Could you do me a favour and stuff all of this into a Bugzilla bug
>>>>> report?
>>>>> Bugzilla is linked off the main squid website front page.
>>>>>
>>>>>
>>>>>
>>>>> Adrian
>>>>>
>>>>>>
>>>>>> 2007/04/26 15:12:17| COSS: /dev/amrd1: Rebuild Completed
>>>>>> 2007/04/26 15:12:17| 81219 objects scanned, 144 objects relocated,
>>>>>> 555
>>>>>> objects fresher, 9346 objects ignored
>>>>>> 2007/04/26 15:12:18| Completed Validation Procedure
>>>>>> 2007/04/26 15:12:18| Validated 143160 Entries
>>>>>> 2007/04/26 15:12:18| store_swap_size = 2576724k
>>>>>> 2007/04/26 15:12:19| storeLateRelease: released 4996 objects
>>>>>> 2007/04/26 15:12:21| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:29| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:29| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:29| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:29| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:36| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:38| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:38| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:38| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:38| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:38| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:38| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:38| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:38| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:38| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:38| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:38| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:38| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:38| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:39| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:39| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:39| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:39| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:39| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:39| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:12:39| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:13:18| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:13:31| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:13:31| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:13:32| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:13:32| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:13:32| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:13:32| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:13:32| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:13:32| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:13:32| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:13:33| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:13:41| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:14:23| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:14:35| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:14:35| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:15:00| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:15:24| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:15:33| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:15:45| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:15:45| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:15:56| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:15:56| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:15:56| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:15:56| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:15:56| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:15:56| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:15:57| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:15:57| squidaio_queue_request: WARNING - Queue congestion
>>>>>> 2007/04/26 15:16:04| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:07| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:07| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:07| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:07| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:07| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:07| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:07| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:07| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:08| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:08| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:08| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:08| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:08| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:08| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:08| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:08| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:08| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:14| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:19| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:19| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:19| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:19| ctx: enter level 0:
>>>>>> 'http://www.theaa.com/css/product_styles.css'
>>>>>> 2007/04/26 15:16:19| WARNING: failed to unpack meta data
>>>>>> 2007/04/26 15:16:19| ctx: exit level 0
>>>>>> 2007/04/26 15:16:19| ctx: enter level 0:
>>>>>> 'http://www.theaa.com/css/print.css'
>>>>>> 2007/04/26 15:16:19| WARNING: failed to unpack meta data
>>>>>> 2007/04/26 15:16:19| ctx: exit level 0
>>>>>> 2007/04/26 15:16:19| ctx: enter level 0:
>>>>>> 'http://www.theaa.com/css/aa_homepage_styles.css'
>>>>>> 2007/04/26 15:16:19| WARNING: failed to unpack meta data
>>>>>> 2007/04/26 15:16:19| ctx: exit level 0
>>>>>> 2007/04/26 15:16:19| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:19| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:19| ctx: enter level 0:
>>>>>> 'http://www.theaa.com/css/tools.css'
>>>>>> 2007/04/26 15:16:19| WARNING: failed to unpack meta data
>>>>>> 2007/04/26 15:16:19| ctx: exit level 0
>>>>>> 2007/04/26 15:16:19| ctx: enter level 0:
>>>>>> 'http://www.theaa.com/css/leftNav.css'
>>>>>> 2007/04/26 15:16:19| WARNING: failed to unpack meta data
>>>>>> 2007/04/26 15:16:19| ctx: exit level 0
>>>>>> 2007/04/26 15:16:19| ctx: enter level 0:
>>>>>> 'http://www.theaa.com/css/topnav.css'
>>>>>> 2007/04/26 15:16:19| WARNING: failed to unpack meta data
>>>>>> 2007/04/26 15:16:19| ctx: exit level 0
>>>>>> 2007/04/26 15:16:19| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:19| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:19| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:19| ctx: enter level 0:
>>>>>> 'http://www.theaa.com/javascript/metafaqs.js'
>>>>>> 2007/04/26 15:16:19| WARNING: failed to unpack meta data
>>>>>> 2007/04/26 15:16:19| ctx: exit level 0
>>>>>> 2007/04/26 15:16:19| ctx: enter level 0:
>>>>>> 'http://www.theaa.com/images/topnav/mainlogo.gif'
>>>>>> 2007/04/26 15:16:19| WARNING: failed to unpack meta data
>>>>>> 2007/04/26 15:16:19| ctx: exit level 0
>>>>>> 2007/04/26 15:16:19| ctx: enter level 0:
>>>>>> 'http://www.theaa.com/images/common/bg_body.png'
>>>>>> 2007/04/26 15:16:19| WARNING: failed to unpack meta data
>>>>>> 2007/04/26 15:16:19| ctx: exit level 0
>>>>>> 2007/04/26 15:16:19| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:19| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:19| ctx: enter level 0:
>>>>>> 'http://www.theaa.com/images/topnav/meta_faq_button.gif'
>>>>>> 2007/04/26 15:16:19| WARNING: failed to unpack meta data
>>>>>> 2007/04/26 15:16:19| ctx: exit level 0
>>>>>> 2007/04/26 15:16:19| ctx: enter level 0:
>>>>>> 'http://www.theaa.com/touchclarity/logging.js'
>>>>>> 2007/04/26 15:16:19| WARNING: failed to unpack meta data
>>>>>> 2007/04/26 15:16:19| ctx: exit level 0
>>>>>> 2007/04/26 15:16:19| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:20| ctx: enter level 0:
>>>>>> 'http://www.theaa.com/touchclarity/logging-code.js'
>>>>>> 2007/04/26 15:16:20| WARNING: failed to unpack meta data
>>>>>> 2007/04/26 15:16:20| ctx: exit level 0
>>>>>> 2007/04/26 15:16:20| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:20| ctx: enter level 0:
>>>>>> 'http://www.theaa.com/touchclarity/optimise.js'
>>>>>> 2007/04/26 15:16:20| WARNING: failed to unpack meta data
>>>>>> 2007/04/26 15:16:20| ctx: exit level 0
>>>>>> 2007/04/26 15:16:20| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:20| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:20| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:20| storeCossCompletePendingReloc: got failure (-1)
>>>>>> 2007/04/26 15:16:20| ctx: enter level 0:
>>>>>> 'http://www.theaa.com/touchclarity/logging.html'
>>>>>> 2007/04/26 15:16:20| WARNING: failed to unpack meta data
>>>>>> 2007/04/26 15:16:20| ctx: exit level 0
>>>>>> 2007/04/26 15:16:20| ctx: enter level 0:
>>>>>>
>>>>>> 'http://www.theaa.com/images/homepage/travel/travel_newsflash_title.gif'
>>>>>> 2007/04/26 15:16:20| WARNING: failed to unpack meta data
>>>>>> 2007/04/26 15:16:20| ctx: exit level 0
>>>>>> 2007/04/26 15:16:20| ctx: enter level 0:
>>>>>>
>>>>>> 'http://www.theaa.com/images/homepage/travel/travel_newflash_button.gif'
>>>>>> 2007/04/26 15:16:20| WARNING: failed to unpack meta data
>>>>>> 2007/04/26 15:16:20| ctx: exit level 0
>>>>>> 2007/04/26 15:16:20| ctx: enter level 0:
>>>>>> 'http://www.theaa.com/images/homepage/travel/travel_newflash_bg.jpg'
>>>>>> 2007/04/26 15:16:20| WARNING: failed to unpack meta data
>>>>>>
>>>>>> Cheers.
>>>>>>
>>>>>> --
>>>>>> Mark Powell - UNIX System Administrator - The University of Salford
>>>>>> Information Services Division, Clifford Whitworth Building,
>>>>>> Salford University, Manchester, M5 4WT, UK.
>>>>>> Tel: +44 161 295 4837 Fax: +44 161 295 5888 www.pgp.com for PGP key
>>>>>
>>>>>
>>>>
>>>> --
>>>> Mark Powell - UNIX System Administrator - The University of Salford
>>>> Information Services Division, Clifford Whitworth Building,
>>>> Salford University, Manchester, M5 4WT, UK.
>>>> Tel: +44 161 295 6843 Fax: +44 161 295 5888 www.pgp.com for PGP key
>>>>
>>>>
>>>
>>
>> --
>> Mark Powell - UNIX System Administrator - The University of Salford
>> Information Services Division, Clifford Whitworth Building,
>> Salford University, Manchester, M5 4WT, UK.
>> Tel: +44 161 295 6843 Fax: +44 161 295 5888 www.pgp.com for PGP key
>>
>>
>
>

-- 
Mark Powell - UNIX System Administrator - The University of Salford
Information Services Division, Clifford Whitworth Building,
Salford University, Manchester, M5 4WT, UK.
Tel: +44 161 295 6843  Fax: +44 161 295 5888  www.pgp.com for PGP key