Client Streams API

Collaboration diagram for Client Streams API:

Modules
	Client Streams Internals

Functions
void	clientStreamInit (dlink_list *, CSR *, CSD *, CSS *, ClientStreamData, CSCB *, CSD *, ClientStreamData, StoreIOBuffer tailBuffer)
void	clientStreamInsertHead (dlink_list *, CSR *, CSCB *, CSD *, CSS *, ClientStreamData)
void	clientStreamCallback (clientStreamNode *thisObject, ClientHttpRequest *http, HttpReply *rep, StoreIOBuffer replyBuffer)
void	clientStreamRead (clientStreamNode *thisObject, ClientHttpRequest *http, StoreIOBuffer readBuffer)
void	clientStreamDetach (clientStreamNode *thisObject, ClientHttpRequest *http)
void	clientStreamAbort (clientStreamNode *thisObject, ClientHttpRequest *http)
clientStream_status_t	clientStreamStatus (clientStreamNode *thisObject, ClientHttpRequest *http)

Detailed Description

Introduction

A ClientStream implements a unidirectional, non-blocking, pull pipeline. They allow code to be inserted into the reply logic on an as-needed basis. For instance, transfer-encoding logic is only needed when sending a HTTP/1.1 reply.

Each node consists of four methods - read, callback, detach, and status, along with the stream housekeeping variables (a dlink node and pointer to the head of the list), context data for the node, and read request parameters - readbuf, readlen and readoff (in the body).

clientStream is the basic unit for scheduling, and the clientStreamRead() and clientStreamCallback() calls allow for deferred scheduled activity if desired.

Theory on stream operation

• Something creates a pipeline. At a minimum it needs a head with a status method and a read method, and a tail with a callback method and a valid initial read request.
• Other nodes may be added into the pipeline.
• The tail-1th node's read method is called.

For each node going up the pipeline, the node either:

• satisfies the read request, or
• inserts a new node above it and calls clientStreamRead(), or
• calls clientStreamRead()

There is no requirement for the Read parameters from different nodes to have any correspondence, as long as the callbacks provided are correct.

Whats in a node

TODO: ClientStreams: These details should really be codified as a class which all ClientStream nodes inherit from.

Each node must have:

• read method - to allow loose coupling in the pipeline. (The reader may therefore change if the pipeline is altered, even mid-flow).
• callback method - likewise.
• status method - likewise.
• detach method - used to ensure all resources are cleaned up properly.
• dlink head pointer - to allow list inserts and deletes from within a node.
• context data - to allow the called back nodes to maintain their private information.
• read request parameters - For two reasons:
• To allow a node to determine the requested data offset, length and target buffer dynamically. Again, this is to promote loose coupling.
• Because of the callback nature of squid, every node would have to keep these parameters in their context anyway, so this reduces programmer overhead.

Function Documentation

clientStreamAbort()

void clientStreamAbort	(	clientStreamNode *	thisObject,
		ClientHttpRequest *	http
	)

Detaches the tail of the stream. CURRENTLY DOES NOT clean up the tail node data - this must be done separately. Thus Abort may ONLY be called by the tail node.

Parameters

thisObject	'this' reference for the client stream
http	MUST NOT be NULL.

Abort the stream - detach every node in the pipeline.

Parameters

thisObject	??
http	??

Definition at line 235 of file clientStream.cc.

References assert, clientStreamDetach(), dlink_node::data, debugs, clientStreamNode::head, and dlink_list::tail.

Referenced by ClientHttpRequest::freeResources().

clientStreamCallback()

void clientStreamCallback	(	clientStreamNode *	thisObject,
		ClientHttpRequest *	http,
		HttpReply *	rep,
		StoreIOBuffer	replyBuffer
	)

Call back the next node the in chain with it's requested data. Return data to the next node in the stream. The data may be returned immediately, or may be delayed for a later scheduling cycle.

Parameters

thisObject	'this' reference for the client stream
http	Superset of request data, being winnowed down over time. MUST NOT be NULL.
rep	Not NULL on the first call back only. Ownership is passed down the pipeline. Each node may alter the reply if appropriate.
replyBuffer	- buffer, length - where and how much.

Definition at line 150 of file clientStream.cc.

References assert, clientStreamNode::callback, clientStreamNode::data, debugs, clientStreamNode::next(), dlink_node::next, and clientStreamNode::node.

Referenced by esiProcessStream(), esiStreamRead(), clientReplyContext::processReplyAccessResult(), clientReplyContext::pushStreamData(), ESIContext::send(), and clientReplyContext::sendStreamError().

clientStreamDetach()

void clientStreamDetach	(	clientStreamNode *	thisObject,
		ClientHttpRequest *	http
	)

Removes this node from a clientStream. The stream infrastructure handles the removal. This node MUST have cleaned up all context data, UNLESS scheduled callbacks will take care of that. Informs the prev node in the list of this nodes detachment.

Parameters

thisObject	'this' reference for the client stream
http	MUST NOT be NULL.

Detach from the stream - only allowed for terminal members

Parameters

thisObject	??
http	??

Definition at line 192 of file clientStream.cc.

References assert, cbdataReference, cbdataReferenceDone, cbdataReferenceValid(), clientStreamNode::data, debugs, clientStreamNode::detach, RefCount< C >::getRaw(), dlink_node::next, clientStreamNode::node, clientStreamNode::prev(), and clientStreamNode::removeFromStream().

Referenced by clientReplyDetach(), clientSocketDetach(), clientStreamAbort(), downloaderDetach(), esiBufferDetach(), esiStreamDetach(), and Http::Stream::finished().

clientStreamInit()

void clientStreamInit	(	dlink_list *	list,
		CSR *	func,
		CSD *	rdetach,
		CSS *	readstatus,
		ClientStreamData	readdata,
		CSCB *	callback,
		CSD *	cdetach,
		ClientStreamData	callbackdata,
		StoreIOBuffer	tailBuffer
	)

Initialise a client Stream. list is the stream func is the read function for the head callback is the callback for the tail tailbuf and taillen are the initial buffer and length for the tail.

Definition at line 112 of file clientStream.cc.

References cbdataReference, clientStreamInsertHead(), dlink_node::data, dlinkAdd(), clientStreamNode::head, clientStreamNode::node, clientStreamNode::readBuffer, and dlink_list::tail.

Referenced by ConnStateData::abortRequestParsing(), ConnStateData::buildFakeRequest(), Downloader::buildRequest(), clientBeginRequest(), ConnStateData::parseHttpRequest(), and Ftp::Server::parseOneRequest().

clientStreamInsertHead()

void clientStreamInsertHead	(	dlink_list *	list,
		CSR *	func,
		CSCB *	callback,
		CSD *	detach,
		CSS *	status,
		ClientStreamData	data
	)

Doesn't actually insert at head. Instead it inserts one after head. This is because HEAD is a special node, as is tail This function is not suitable for inserting the real HEAD.

Definition at line 131 of file clientStream.cc.

References assert, cbdataReference, dlink_node::data, debugs, dlinkAddAfter(), RefCount< C >::getRaw(), clientStreamNode::head, dlink_list::head, dlink_node::next, clientStreamNode::node, and clientStreamNode::readBuffer.

Referenced by clientStreamInit(), and clientReplyContext::processReplyAccessResult().

clientStreamRead()

void clientStreamRead	(	clientStreamNode *	thisObject,
		ClientHttpRequest *	http,
		StoreIOBuffer	readBuffer
	)

Triggers a read of data that satisfies the httpClientRequest metainformation and (if appropriate) the offset,length and buffer parameters.

Parameters

thisObject	'this' reference for the client stream
http	Superset of request data, being winnowed down over time. MUST NOT be NULL.
readBuffer	- offset, length, buffer - what, how much and where.

Call the previous node in the chain to read some data

Parameters

thisObject	??
http	??
readBuffer	??

Definition at line 170 of file clientStream.cc.

References assert, clientStreamNode::data, debugs, RefCount< C >::getRaw(), clientStreamNode::prev(), clientStreamNode::readBuffer, and clientStreamNode::readfunc.

Referenced by constructHelperQuery(), ClientHttpRequest::doCallouts(), esiBufferRecipient(), esiProcessStream(), esiStreamRead(), Downloader::handleReply(), ClientRequestContext::hostHeaderVerifyFailed(), ClientHttpRequest::httpStart(), and Http::Stream::pullData().

clientStreamStatus()

clientStream_status_t clientStreamStatus	(	clientStreamNode *	thisObject,
		ClientHttpRequest *	http
	)

Allows nodes to query the upstream nodes for :

• stream ABORTS - request cancelled for some reason. upstream will not accept further reads().
• stream COMPLETION - upstream has completed and will not accept further reads().
• stream UNPLANNED COMPLETION - upstream has completed, but not at a pre-planned location (used for keepalive checking), and will not accept further reads().
• stream NONE - no special status, further reads permitted.

Parameters

thisObject	'this' reference for the client stream
http	MUST NOT be NULL.

Call the upstream node to find it's status

Parameters

thisObject	??
http	??

Definition at line 257 of file clientStream.cc.

References assert, dlink_node::data, clientStreamNode::node, dlink_node::prev, and clientStreamNode::status.

Referenced by esiBufferRecipient(), esiStreamStatus(), Downloader::handleReply(), and Http::Stream::socketState().