Re: [squid-users] Help on Squid Proxy and download manager integration

On 2014-03-30 23:10, Fahim Mohammed wrote:
> Hi all,
>
> Could you please give me some tip / point me in the direction of how
> i can achieve the following?
>
> I need squid to redirect certain download types to a download manager
> like aria2 or axel - which will download the file using segmented
> downloading and pass the file to squid or to the user who requested
> the file.
>
> Do hope that the question is clear and that someone will be able to
> help out. Thanks a bunch :)

Opening certain URLs in an application is a web browser or OS feature.
By the time Squid gets the request it is already being processed by
whatever client application the end-user has.

Also, you may seriously want to reconsider this proposal. Download
managers using "segmented downloading" (I assume that means Range
requests or some equivalent?) *break* HTTP in several ways ...

  * Other clients downloading those same URLs end up with worse response
times as they cannot get HIT responses on an earlier clients full-object
request.

  * Other clients downloading *other* URLs also get a speed/bandwidth
penalty as the download manager forces extra connections through the
proxy. The proxy will slow down when it nears TCP socket limits and as
its total connection count rises (more work to do == slower
per-connection handling for everyone).

  * The proxy may un-segment each of the N separate requests made by the
download manager in order to fetch the entire object and cache it.
Resulting in much more bandwidth consumption on the Proxy<->Server
network until the first URL request is cached. Following which the
download manager receives multiple responses in the *same* time the
proxy could have delivered as HIT. (ie. up to twice as long to deliver
the object than normal HTTP/1.1 would have. yay.)

The Axel page has a hint about other problems "It doesn't pass the
correct headers from the client to the server yet". To me that reads as
meaning that HTTP/1.1 performance features are unlikely to work properly
due to header brokenness, possibly even cross-site security issues as
well.

Download managers were never a good idea for HTTP protocol fetches. A
lot of content generation and delivery work is placed upon the
intermediaries and server under normal protocol handling. Increasing
that work N-fold to benefit a single client is a bad idea that backfires
in some obscure ways.

Amos
Received on Sun Mar 30 2014 - 20:51:46 MDT