Re: [squid-users] Re: ICP and HTCP and StoreID

On 2014-02-12 06:51, Niki Gorchilov wrote:
> Rising this issue from the dead :-)
>
> On Thu, Jan 16, 2014 at 8:21 AM, Alex Rousskov
> <rousskov_at_measurement-factory.com> wrote:
>> On 01/15/2014 03:31 PM, Niki Gorchilov wrote:
>>> Actually, it is working. [...] inter cache communication is working
>>> only with
>>> altered URLs but this still does the job:
>>> - If UDP is MISS the originating peer makes a TCP connection to
>>> destination server and caches the result
>>> - if UDP is HIT, the call is forwarded via sibling with modified URL,
>>> but the sibling handles the request without problems
>>>
>>> UDP HIT request example:
>>> peer B: UDP_HIT/000 0 HTCP_TST
>>> http://c.youtube.com.squid.internal/videoplayback/09ebf166f4892e4f.140.712704-950271
>>> - HIER_NONE/- -
>>> peer B: TCP_HIT/200 237948 GET
>>> http://c.youtube.com.squid.internal/videoplayback/09ebf166f4892e4f.140.712704-950271
>>> - HIER_NONE/- application/octet-stream
>>> peer A: TCP_MISS/200 237948 GET
>>> http://r2---sn-bavc5aoxu-nv4e.googlevideo.com/videoplayback? -
>>> SIBLING_HIT/peerb application/octet-stream
>>>
>>> UDP MISS request example:
>>> peer B: UDP_MISS/000 0 HTCP_TST
>>> http://c.youtube.com.squid.internal/videoplayback/09ebf166f4892e4f.140.2138112-2375679
>>> - HIER_NONE/- -
>>> peer A: TCP_MISS/200 237938 GET
>>> http://r2---sn-bavc5aoxu-nv4e.googlevideo.com/videoplayback? -
>>> HIER_DIRECT/r2---sn-bavc5aoxu-nv4e.googlevideo.com
>>> application/octet-stream
>>>
>>> Case closed!
>>
>> Store ID is not a URL (even if it is often convenient to think that
>> way). If Squid sends requests with StoreIDs, Squid is broken (even if
>> it
>> happens to usually work in your particular case). Consider the
>> situation
>> where the peer returned UDP_HIT but then purged the cached entry. What
>> will it do with the c.youtube.com.squid.internal request?
>
> Yes, you are absolutely right. Everything "works" until the peer
> decides to refresh the entry during the TCP session. Then it tries to
> connect to the non-existent *.squid.internal host, which obviously
> fails.
>
>> Somebody who cares about this _and_ can reproduce it, should file a
>> bug
>> report, but please double check that the requests are indeed using
>> Store
>> IDs using cache.log debugging or a packet trace. Do not just assume
>> that
>> Squid will never log a Store ID instead of a URL.
>
> Bug is definitely there. Confirmed by sniffing the inter-peer
> communication. I may try to fix it but give me some high-level advice
> in which classes/files to look for. I get lost at
> http://www.squid-cache.org/Doc/code/annotated.html :)
>

IPC and HTCP are old-school C code still for the most part. There are
not exactly classes involved, except the state and packet structure i
stored in a class.

For ICP the source file icp_v2.cc contains most of the code with a
little bit in icp_v3.cc and ICP.h as a shared header. The root functions
receiving/sending ICP packets are icpUdpSend() and icpHandleUdp().

For HTCP the relevant root functions are htcpSend() and htcpRecv() in
src/htcp.cc.

Same logic applies to the URLs for all ICP and HTCP packets:
  a) queries sent to peers should send client HTTP request URL/URI (*not*
StoreID value)
  b) queries received should convert to StoreID before looking up
existence in the local cache.

(a) is the bug. (b) is the known missing operation in current StoreID
feature.

HTH
Amos
Received on Tue Feb 11 2014 - 22:08:55 MST