RE: [xmlblaster] Durable QoS performance

[Doug.Palmer at csiro.au]

> If the issue is urgent you would need to code it yourself.

It's not urgent in the sense that there's an mission critical application
waiting on it. But I've got an obvious desire to be able to say "it works
reliably with enough throughput to be usable for a distributed sensor
application, even if it is a pig to program and configure".

(Just to give you an idea if what I'm thinking of. The basic aim is to build
a middleware layer for a set of distributed environmental monitoring sensors
so that land & water scientists can build an accurate picture of the state
of an area. The data comes from various wireless links. The data needs to be
delivered to databases, web pages, other applications ... So I think you can
see why xmlBlaster looks attractive. To begin with, we're talking on the
order of 1000 sensors, each generating a reading a minute. So that's on the
order of 20 durable messages a second, with enough fat built in to cope with
expansion and traffic bursts.)

> The oid is one message, you can collect message to groups with XPath 
> subscriptions.

I've replaced Oids with 

      this.publishKey =
        "<key oid='' contentMime='application/octet-stream'>\n" +
        "   <" + this.groupName + "></" + this.groupName + ">\n" +
        "</key>";

for the publisher and

      this.subscribeKey =
        "<key oid='' queryType='XPATH'>" +
        "/xmlBlaster/key/" + this.groupName +
        "</key>";

for the subscriber. Is this correct?

This seems to be working, up to a point. (I do have to remember to turn INFO
level logging off, to stop it telling me that it's found a match for every
message.) But the overall performance seems to have taken a dive: 

Size	MPS	Sdrs	Rcvrs	Subj	Time		Sent	Rcvd	TS
TR
4096	10	10	1	1	119999	1200	1200	10	10
4096	20	10	1	1	119984	2400	2400	20	20
4096	50	10	1	1	119999	4541	4531	38	38
4096	100	10	1	1	119984	7593	7583	63	63

Part of the problem seems to be that when I stop a test and start another, I
start getting messages of the form

[Jul 8, 2002 11:14:21 AM WARN  RequestBroker-http://152.83.91.4:3412]
Generating dead letter oid=http://A.B.C.D:3412-1026090587859-72 from
publisher=guest because delivery to 'guest' cbQueue=session:23 failed. --
I've removed my IP address.

Eventually, the dead letter queue fills up and the whole thing dies in a
welter of OutOfMemoryErrors. To shut down, I'm unSubscribing the receiers,
using the Oid supplied by subcribe() method. I then disconnect() and then
shutdown(). (I've tried assorted variations on this pattern.) Is this right?
Do I have to explicitly erase every message I've sent?

The above is for non-durable messaging. With durable, I get:

id=FileUtil reason=Can't write file java.io.FileNotFoundException:
C:\Documents and
Settings\dougp\tmp\http:\A.B.C.D:3412-1026091518765-11-XmlKey.xml (The
filename, directory name, or volume label syntax is incorrect) 

Which is caused by the ':' being illegal in filenames under Win2k.
(Incidentally, why am I getting http: to begin with? I thought that IIOP was
the default. This could be an explanation for the difference in performance
between your and my benchmarks. The clients are using the inbuilt
xmlBlaster.properties file from the xmlBlaster jar.)

Another thing is that I'm getting lots of messages of the form "Looping
nanoCounter=1". The code in Timeout.java says that I should be only getting
this to avoid two similar keys and it shouldn't happen very often. Since the
oids are generated, and look pretty different, should this occur?

Last thing. Doing pub/sub via raw XML seems awfully laborious. Particularly
since the PublishKeyWrapper and SubscribeKeyWrapper effectively provide an
API for pub/sub via oids. I would have thought that, since oids seem to be
individually message-specific, the wrappers would provide an API for simple
pub/sub using topic names, which is what most people would want to do.
Instead, the Java demo examples and API seem to positively encourage a
newbie like me to use oids as a quick'n'dirty subject name. (Don't get me
wrong, I'm immensely impressed by the coolness of using XPath to allow
subscriptions to almost anything under the sun. And it's got applications in
the area I'm thinking of. It's just that I feel that the API conspires to
discourage me from using it.)

>This is strange, the default callback only gets PtP messages and  subcribed
messages
>which didn't specify an own Callback on subscription.
>If i invoke 'java HelloWorld4' this seems to work fine.

>From HelloWorld4, I get:

[8/07/2002 13:28:01 INFO  HelloWorld4] Reveiving asynchronous message
'HelloWorld4' in HelloWorld4 handler 
[8/07/2002 13:28:01 INFO  HelloWorld4] Reveiving asynchronous message
'SomeOtherMessage' in default handler 

Which I think is what's supposed to happen.

It's the 'default handler' bit that bothers me. How do I stop a connection
from receiving 'SomeOtherMessage' and having it go to the default handler,
if I don't want to see it at all? I would have thought that there's a
default 'explicit subscriptions only' option, to avoid having a client
drowning in extraneous messages.