[xmlblaster] missing volatile messages

[Eugene Wu <ewu at navahonetworks.com>]

Hi, there,

We have been using xmlBlaster server for a while and it worked well. But 
after upgraded it from
version 1.0.6 to 1.1.1, we noticed volatile message losing when the 
subscribers is running.

In order to figure out what was going wrong, I wrote three shell scripts 
for testing:
pubtest.sh, subtest.sh and validate.sh, which are attached with the mail.

Test machines:
  Server machine:
     AMD Athlon(tm) 64 Processor 3000+, 1 GM RAM
     Debian GNU/Linux 3.1, kernel 2.6.11
     Java version 1.5.0_06
     xmlBlaster v1.1.1

  Client machine:
     Pentium III (Coppermine), 2 GM RAM
     Debian GNU/Linux 3.1, kernel 2.6.10
     Java version 1.4.2_09
     xmlBlaster v1.1.1

Test steps:
  1. restart xmlBlaster server in server machine using default configs.
  2. run subtest.sh in client machine.
  3. after subscription is finished, run pubtest.sh in client machine.
  4. after publishing is done, run validate.sh
  5. run diff to figure out missing messages

Testing:
  1. subtest.sh setup three subscribers by using xpath:
       "//service[.='AA_AA' and ../type[.='BB_BB']]"

  2. pubtest.sh setup two publishers, each of them published 100 
volatile messages to oid=AABB,
     by using client tags:
       <service>AA_AA</service><type>BB_BB</type><pub1-%counter/>
       <service>AA_AA</service><type>BB_BB</type><pub2-%counter/>

Test results:
  1. after testing many times, I found that most of messages(>97%) 
reached all subscribers,
     some messages(2%) reached part of subscribers, but some 
messages(<1%) completely lost.
     Checked the server's log file and confirmed that the server 
received all messages, but
     some of them were not pushed to subscribers.

  2. if run both xmlBlaster server and clients in a same machine, NO 
messages lost (testing more than 10 times).

  3. tested with xmlBlaster v1.2, and messages still got lost.

  4. changed publish number from 100 to 1000, got the followwing 
NullPointerException.
     I thought it was caused by queue's maxEntriesCache setting 
(default as 1000). After
     increasing maxEntriesCache to 2000 for both connect queue and 
callback queue, no more
     NullPointerException occurred.

[Mar 21, 2006 3:06:07 PM ERROR XmlBlaster.SOCKET.tcpListener-ewu 
AvailabilityChecker] publishArr() failed:
XmlBlasterException serverSideException=true 
node=[xmlBlaster_127_0_0_1_3412] location=[RequestBroker]
java.lang.NullPointerException
stackTrace=java.lang.NullPointerException
       at 
org.xmlBlaster.engine.TopicHandler.invokeCallback(TopicHandler.java:1350)
       at 
org.xmlBlaster.engine.TopicHandler.invokeCallbackAndHandleFailure(TopicHandler.java:1170)
       at org.xmlBlaster.engine.TopicHandler.publish(TopicHandler.java:645)
       at 
org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1677)
       at 
org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1483)
       at 
org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1477)
       at 
org.xmlBlaster.engine.XmlBlasterImpl.publishArr(XmlBlasterImpl.java:185)
       at 
org.xmlBlaster.util.protocol.RequestReplyExecutor.receiveReply(RequestReplyExecutor.java:402)
       at 
org.xmlBlaster.protocol.socket.HandleClient.handleMessage(HandleClient.java:231)
       at 
org.xmlBlaster.protocol.socket.HandleClient.run(HandleClient.java:352)
       at java.lang.Thread.run(Thread.java:534)
errorCode=internal.nullpointer message=#1.1.1 
errorCode=internal.publish: An internal error occurred
when processing a publish() request. 
/node/xmlBlaster_127_0_0_1_3412/client/ewu/-9
       at 
org.xmlBlaster.util.XmlBlasterException.convert(XmlBlasterException.java:665)
       at 
org.xmlBlaster.util.XmlBlasterException.convert(XmlBlasterException.java:654)
       at 
org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1697)
       at 
org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1483)
       at 
org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1477)
       at 
org.xmlBlaster.engine.XmlBlasterImpl.publishArr(XmlBlasterImpl.java:185)
       at 
org.xmlBlaster.util.protocol.RequestReplyExecutor.receiveReply(RequestReplyExecutor.java:402)
       at 
org.xmlBlaster.protocol.socket.HandleClient.handleMessage(HandleClient.java:231)
       at 
org.xmlBlaster.protocol.socket.HandleClient.run(HandleClient.java:352)
       at java.lang.Thread.run(Thread.java:534)
versionInfo=version=1.1.1,revision=1.1.1,os.name=Linux,os.version=2.6.11.7-zh,java.vm.vendor=Sun 
Microsystems Inc.,
java.vm.version=1.4.2_09-b05,os.arch=i386,build.timestamp=03/17/2006 
05:29 PM,build.java.vendor=Sun Microsystems Inc.,
build.java.version=1.4.2_09 errorCode description=
http://www.xmlblaster.org/xmlBlaster/doc/requirements/admin.errorcodes.listing.html#internal.nullpointer



I tryed to debug but failed, as some log messages never showed up even 
if I turned on -trace -call and -dump.

Hope the message could be helpful for your debugging.


Best Regards,

Eugene Wu

Attachment: pubtest.sh
Description: application/shellscript

Attachment: subtest.sh
Description: application/shellscript

Attachment: validate.sh
Description: application/shellscript