Type	NEW
Priority	HIGH
Status	CLOSED

Topic	XmlBlaster has a sophisticated client callback framework
Des cription	XmlBlaster has a callback framework to send messages to clients. Clients need to establish a callback server (in any supported protocol) to receive these asynchronous messages. Callback Features: Thread pool A callback is invoked by a thread from a thread pool. Retrieving a thread from the pool consumes approximately 180 micro seconds compared to approximately 800 micro seconds creating a thread natively. It is guaranteed that a maximum of one thread is occupied per queue/callback. Zero or once delivery (transient message) A message is usually delivered exactly once. Under certain failure circumstances a message can't be delivered or is even redelivered: If xmlBlaster is shutdown or crashes, transient messages are lost. If the client can't be reached or the client does not acknowledge the message, xmlBlaster will redeliver the message. Such messages are marked with a QoS tag <redeliver>12</redeliver> whereby 12 is the current attempt. xmlBlaster retries as configured (see below) or until the message or queue span of life expires and in this case the lost messages are published as dead letters (the login session is destroyed as well). Redelivering can be switched off, resulting in immediate dead letters and garbage collection of resources. Note: The redeliver information is recovered even on server crash for persistent messages (the redeliver counter is persistent for persistent messages). For PtP messages it could be reasonable to inform the sender of the message about this situation [needs to be specified] Once and only once delivery (persistent message) Persistent messages are guaranteed to be delivered once and only once, even if xmlBlaster crashes or is shutdown interim. Persistent messages are stored in xmlBlasters database on hard disk as soon as it arrives. Only if a client is not reachable or not acknowledging, a message can be lost (see previous paragraph). Burst mode (collect messages) XmlBlaster allows to activate a burst mode. This allows to collect messages for a client for a configurable amount of time (typically 50 milli seconds). This increases latency of message delivery for 50 msec but allows a much higher message throughput. The overall performance increases. Protect against misbehaving clients Every callback is controlled by a timer, to avoid leakage of threads and memory resources in the server when clients misconduct. ACK/NAK handling Clients shall acknowledge messages if they are capable to do so. For dumb clients or connectionless protocols like email, the xmlBlaster plugin driver can simulate the ACK. Callbacks are only as reliable as the protocol and client allows. ACK/NAK handling is achieved through return values of method invocation. Oneway invocations have no ACK/NAK support on application level, but the underlying TCP ACK/NAK makes them reliable enough for many use cases. Automatic disconnect detection Failed callback connections are caught by xmlBlaster and handled as described above. Authentication on callback The update() method invoked on client side has to accept the message and has the problem to trust the sender. Another process may otherwise misuse the callback and send unauthorized messages. Therefore the first argument in update() sends a sessionId, which must be passed to xmlBlaster initially on callback construction. Oneway invocations Oneway invocations are method invocations without a return value. The increase performance but loose ACK/NAK on application level: The publish() and update() methods support oneway variants called publishOneway() and updateOneway(). Note that only CORBA and our native SOCKET protocol support oneway invocations. XmlRpc, RMI and the like don't support this mode. Message compression Messages can be compressed with the zip format. The zip applies for the content of the message only. Zipping happens before sending a message to a callback server which requested this service. This is currently implemented with the SOCKET and EMAIL protocol only. Pinging of callback server The callback server is pinged every minute as default. Pinging with method ping() is similar implemented as the update() invocation on the client callback server. If the ping fails ('onExhaust') the session is killed - the client is auto logged out (depending on the retries setting). Error handling We distinguish recovery procedures for callback connection problems and for blocking clients and for clients sending NAK. There are different error conditions when invoking update() or ping(): Receiver is not reachable or throws an exception Depending on configuration we try to redeliver. When we give up a dead letter is published and the client is logged out. Receiver blocks Currently we can't detect this, the callback worker thread is blocked and lost. Receiver sends error in return string The return value is ignored, the message counts as delivered. In future this information may be used for transaction processing. Only exceptions thrown by a client have influence as described in requirement interface.update. A possible approach for a lost PtP messages is to send a NAK to the sender of the message. Open issues Blocking publisher XmlBlaster blocks a publish() invocation of a client until it has securely accepted the message. Is it necessary to support a blocking until e.g. a PtP message is delivered to its destinations (like a blocking RPC call)? Transactional context Transaction support (probably based on Jini) and failure recovery needs to be specified.
Example any	This shows the callback part of a connect() QoS: <callback type='IOR' sessionId='4e56890ghdFzj0' pingInterval='60000' retries='5' delay='60000' oneway='true' useForSubjectQueue='true'> IOR:00044550005... <compress type='gzip' minSize='1000'/> <burstMode collectTime='400'/> </callback> Use CORBA to callback messages, on error retry 5 times pausing 60000 millis between retries. Deliver the given sessionId with update() to the client so that the client can trust us. Ping the client every 60000 millis (one minute) to check if he is alive. The given callback may be used by the subjectQueue as well. Compress messages bigger 1000 bytes before sending them to me. Collect messages for 400 milliseconds and update them in one callback (burst mode) This update mode is a 'transaction' for all collected messages. The messages are sent oneway, there is no return value or exception transfer for this method invocation. In other words there is no application level ACK. Default is oneway='false'.
Example Java	This shows the hardcoded callback configuration of a connect() QoS for Java clients: import org.xmlBlaster.client.qos.ConnectQos; import org.xmlBlaster.client.qos.ConnectReturnQos; import org.xmlBlaster.util.qos.address.CallbackAddress; ... I_XmlBlasterAccess con = glob.getXmlBlasterAccess(); ConnectQos connectQos = new ConnectQos(glob, name, passwd); CallbackAddress cbAddress = new CallbackAddress(glob); cbAddress.setDelay(4000L); // retry connecting every 4 sec cbAddress.setRetries(-1); // -1 == forever cbAddress.setPingInterval(4000L); // ping every 4 seconds connectQos.addCallbackAddress(cbAddress); // Connect to xmlBlaster ... ConnectReturnQos connectReturnQos = con.connect(connectQos, this);
Configure	These parameters can be specified on client side with java clients (see ConnectQos API). Other programming languages need to supply these setting with a raw XML string (see example section above). Property Default Description Implemented dispatch/callback/collectTime 0 BurstMode: The time to collect messages for update dispatch/callback/pingInterval 60000 (one minute) Ping interval: pinging every given milliseconds dispatch/callback/retries 0 How often to retry if callback fails: defaults to 0 == don't retry, on failure we give up. Set to -1 to try forever dispatch/callback/delay 60000 (one minute) Delay between callback retires in milliseconds dispatch/callback/useForSubjectQueue true Shall this session callback be used for subjectQueue messages as well? dispatch/callback/compress/type "" Compress messages if set to "gzip" or "zip" dispatch/callback/compress/minSize 0 Messages bigger this size in bytes are compressed dispatch/callback/ptpAllowed true PtP messages wanted? Defaults to true, false prevents spamming dispatch/callback/receiveTimestampHumanReadable false You can force a human readable timestamp in the QoS of received messages on update(), see API of UpdateQos below dispatch/callback/sessionId unknown The identifier sent to the callback client, the client can decide if he trusts this invocation These parameters can be specified on serverside. Property Default Description Implemented dispatch/callback/threadPrio 5 The priority by which the dispatcher threads have to be run. dispatch/callback/maximumPoolSize 200 The maximum number of threads allowed to run to dispatch. This is for all clients together. dispatch/callback/minimumPoolSize 50 The minimum number of threads allowed to run to dispatch. This is for all clients together. If you have less threads when the dispatcher requests come, new threads are instantiated. You should avoid choose this value to be less than three since if a dispatcher blocks you could block the all others. dispatch/callback/createThreads 5 The maximum number of threads which have to be initially created. dispatch/callback/threadLifetime 180000 The time expressed in milliseconds a thread lives with no activity before it is removed. NOTE: Configuration parameters are specified on command line (-someValue 17) or in the xmlBlaster.properties file (someValue=17). See requirement "util.property" for details. Columns named Impl tells you if the feature is implemented. Columns named Hot tells you if the configuration is changeable in hot operation.
See API	org.xmlBlaster.engine.queue.MsgQueue
See API	org.xmlBlaster.engine.callback.CbWorker
See API	org.xmlBlaster.util.qos.address.CallbackAddress
See API	org.xmlBlaster.util.qos.storage.QueueProperty
See API	org.xmlBlaster.client.qos.ConnectQos
See API	org.xmlBlaster.client.qos.UpdateQos
See API	org.xmlBlaster.test.qos.TestCallbackConfig
See REQ	engine.queue
See REQ	engine.qos.login.callback
See REQ	util.property
See REQ	util.property.args
See REQ	util.property.env
See TEST	org.xmlBlaster.test.qos.TestCallback
See TEST	org.xmlBlaster.test.qos.TestCallbackConfig

This page is generated from the requirement XML file xmlBlaster/doc/requirements/engine.callback.xml

Back to overview