Re: [xmlblaster] losing some pub/sub messaging w/ -protocol XMLRPC (but not w/ -protocol SOCKET)

[Michele Laghi <xbl at laghi.eu>]

trying to reproduce the error, got the situation that when a client is
killed its session contiunes to live on the server side. When the client
comes again with the same port it gets also the other sessions messages.

Console1 : java org.xmlBlaster.Main -xmlBlaster/jmx/HtmlAdaptor true
Console2; java javaclients.HelloWorldPublish
-dispatch/connection/plugin/xmlrpc/hostname laptopAir -protocol SOCKET
-interactive false -sleep 500 -numPublish 10 -persistent false
Console3: java javaclients.HelloWorldSubscribe
-dispatch/connection/plugixmlrpc/hostname laptopAir -protocol XMLRPC
-dispatch/callback/plugin/xmlrpc/port 8090 -interactive false
-autoSubscribe true -runAsDaemon true -unSubscribe false -multiSubscribe
false

On Console3 the subscribing client can be killed and the session will
continue to live.

After the client has been killed and before the callback ping comes,
when more messages are publishes the following error comes:

Caused by:
java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
        at
java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
        at java.net.Socket.connect(Socket.java:525)
        at java.net.Socket.connect(Socket.java:475)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
        at sun.net.www.http.HttpClient.<init>(HttpClient.java:233)
        at sun.net.www.http.HttpClient.New(HttpClient.java:306)
        at sun.net.www.http.HttpClient.New(HttpClient.java:323)
        at
sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:860)
        at
sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:801)
        at
sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:726)
        at
sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:904)
        at
org.apache.xmlrpc.client.XmlRpcSunHttpTransport.writeRequest(XmlRpcSunHttpTransport.java:96)
        at
org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:143)
        at
org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:115)
        at
org.apache.xmlrpc.client.XmlRpcSunHttpTransport.sendRequest(XmlRpcSunHttpTransport.java:69)
        at
org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56)
        at
org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167)
        at
org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:158)
        at
org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:147)
        at
org.xmlBlaster.protocol.xmlrpc.CallbackXmlRpcDriver.sendUpdate(CallbackXmlRpcDriver.java:193)
        at
org.xmlBlaster.engine.dispatch.CbDispatchConnection.doSend(CbDispatchConnection.java:310)
        at
org.xmlBlaster.util.dispatch.DispatchConnection.send(DispatchConnection.java:251)
        at
org.xmlBlaster.util.dispatch.DispatchConnectionsHandler.send(DispatchConnectionsHandler.java:462)
        at
org.xmlBlaster.util.dispatch.DispatchWorker.run(DispatchWorker.java:115)
        at
EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:732)
        at java.lang.Thread.run(Thread.java:619)


When testing overflow exceptions came, these look like the following and
are caused by the fact that the subscribers are slow in receiving it.

2010-05-10 11:27:13.132 WARNING 15-pool-1-thread-3 RL10
org.xmlBlaster.engine.RequestBroker deadMessage: Generating dead message
'callback:/node/xmlBlaster_192_168_0_21_3412/client/michele/-5/NORM/1273483633132000000/Hello'
from publisher=/node/xmlBlaster_192_168_0_21_3412/client/michele/-6
because delivery failed: XmlBlasterException
errorCode=[resource.overflow.queue.entries] serverSideException=true
location=[callback:/node/xmlBlaster_192_168_0_21_3412/client/michele/-5:
] message=[#17773M Cache queue overflow, 1001 entries are in queue, try
increasing '-queue/callback/maxEntries' on client login: first check in
put : ] [See URL
http://www.xmlblaster.org/xmlBlaster/doc/requirements/admin.errorcodes.listing.html#resource.overflow.queue.entries]
2010-05-10 11:27:13.343 WARNING 18-XmlBlaster.SOCKET RL10
org.xmlBlaster.util.dispatch.DispatchManager
 

On 04/20/2010 03:44 PM, Gautam Thaker wrote:
> Michele Laghi wrote:
>> Hi Gautam,
>> which configuratioo of the XMLRPC Protocol are you using ? Are you
>> tunnelling back the answers or are you using it in the classical way
>> (that is separate Connections for publish / updates)
>>
>
> All my receivers were on one node, I had 4 of them. I was starting
> them w/ the shell script below, varying the port number argument as
> 8090, 8091, 8092, 8093
>
> java ght2rec -protocol XMLRPC
> -dispatch/connection/plugin/xmlrpc/hostname 10.1.1.2 
> -dispatch/callback/plugin/xmlrpc/port $1
>
> The sender was on a 2nd node, it was started with:
>
> java ght2send  -dispatch/connection/plugin/xmlrpc/hostname 10.1.1.2
> -protocol XMLRPC
>
> The xmlBlaster server runs on a host by itself (host w/ ip addr
> 10.1.1.2).
>
> I believe tihs means I am using "classical way".
>
> I can confirm that if I use this same setup and just use
> "SOCKET/socket" in place of "XMLRPC/xmlrpc" the application has run
> for 10x longer (to finish) w/o any lost replies.
>
> Gautam
>
>
>
>> Regards
>> Michele
>>
>>
>>
>> On 04/16/2010 10:45 PM, Gautam Thaker wrote:
>>> Hi:
>>>
>>> I am running a variant of HelloWorld demo pgm. My setup is as follows:
>>>
>>>
>>> sender <--->xmlBlaster_server  <----> receiver x 4
>>>
>>> There is just 1 "sender", but 4 "receivers".
>>>
>>> Sender publishes a msg on "forward" channel that is subscribed to by
>>> all the receivers. When they get a msg, they reply back to a "reverse"
>>> channel. The sender blocks till it receives 4 replies. It measures the
>>> time from its send to receipt of all the 4 replies. It then does sends
>>> a next message on "forward" channel.
>>>
>>> This goes on for 100,000 times. Message sizes are small, byte[4] or
>>> byte[8].
>>>
>>> What i have noticed is that when using
>>>
>>> -protocol XMLRPC at various times, after few 10's of thousand
>>> exchanges, the sender is stuck in a state where it has received 3 of
>>> the 4 replies it was expecting. As the sender gets multiple replies
>>> back potentially its "update()" routine may have multiple concurrent
>>> active callbacks (Is this possible?) In any event, all my variables
>>> that know how many replies have come in so far are "synchronized", i
>>> was fearing I was getting threads stomping on each other. Even w/ all
>>> variable access well synchronized I get to this state where 3 of 4
>>> replies have come in and I am hung.
>>>
>>> To repeat, just by changing command line args to use -protocol SOCKET
>>> I have never seen this happen to me.
>>>
>>> Can published messages in such simple, synchronized message exchanges
>>> where queues can never have more than 1 (or 4) messages in it, can
>>> msgs be lost by xmlBlaster?
>>>
>>> Thank for any hints. I need reliable message delivery.
>>>
>>> Gautam
>>> g
>>>
>
>