RE: [xmlblaster] Durable QoS performance

[Doug.Palmer at csiro.au]

> say "it works
>  >reliably with enough throughput to be usable for a 
> distributed sensor
>  >application, even if it is a pig to program and configure".
>  >
> 
> :-)
> But it makes me reflective, how can we improve or simplify the
> piggy programming and configuring tasks?

Please accept what follows in the spirit of constructive critism. I don't
want to make it sound like I'm sounding off with "xmlBlaster is a load of
crap" when I think the opposite.

Actually, server configuration is pretty straightforward. Especially when
compared with some products that I've worked with. It was a bit of a scare
to realise that the config file is in the xmlBlaster jar and, therefore,
relatively inaccessable. But I can see the reasoning behind that.

The biggest problem I have is with sudden layering shifts. Keys, Qos, no
problem! However, I seem to have, on one hand, something like
PublishKeyWrapper, which looks like the kind of thing I need to set up a
publication and, on the other hand, the need to write raw XML whenever
things get a little complicated. The wrap() method is pretty obscure, and
the only example is the HelloWorld3 program, which uses oids as keys anyway.
The need to use toXml sometimes also suggests encapsulation leaks.

As a suggestion, I'd keep the Java API using objects all the time, instead
of raw XML and have a class hierarchy that allows raw XML. For example

                     PublishKeyWrapper (abstract)
                              ^
                              |
        +---------------------+--------------------------+
        |                     |                          |
PublishKeyOidWrapper   PublishKeyTopicWrapper   PublishKeyRawWrapper

PusblishKeyOidWrapper indicates publication though an explicit Oid,
PublishKeyRawWrapper through raw XML. The PublishKeyTopicWrapper publishes
under a "topic" in the ordinary pub/sub model. Topics could be strings with
a heirarchy of topic names. Each subclass has its own toXml() method that
generates the appropriate XML for lower layers. The same approach could be
used for subscribers, with SubscribeKeyTopicWrapper generating a suitable
XPath for the topic. (The class hierarchy here may be a bit off. Rather than
having separate {Publish/Subscribe/Get/Erase/Unsubscribe}KeyWrapper
heirarchies, it might be worth having a straight KeyWrapper heirarchy
separated by {Oid,Topic,Raw} approach. But that's neither here nor there.)

The mixture of paradigms also makes things tricky. Is an oid a message id or
a topic id? It's both: it's a drain cleaner and a meat substitute! Is it a
messaging system or a shared data repository? Getting caught by the default
value of isVolatile has made me realise that it's not a straight MOM system.
Nothing wrong with that, but it is a trap for the unwary. However, I can see
the benefits of having the mixture, so I've got no concrete suggestions
here.

> If you use the same oid="sensor" for all messages, it should boost
> performance.

That's true, but from a previous message:

From: Marcel Ruff [mailto:ruff at swand.lake.de]
> The oid is one message, you can collect message to groups with XPath 
> subscriptions.

Without using XPath, messages seem to be limited to a rate of one message
going through the system at a time, since the oid uniquely identifies the
message within the system.

> Dead letters are volatile messages, this memory leak was fixed
> some days ago.
> Can you please confirm that the memory exhaust disappeared in 
> the newest 
> cvs?

Will do.

> The problem is that as a default the last message of every 
> key-oid remains
> in memory. So in your case just use the same oid for all messages.
> Or erase them or set them to 'isVolatile'.
> But the best choice for you is probably to use the same oid.

OK. So the default behaviour is non-volatile

>  >wrong, I'm immensely impressed by the coolness of using 
> XPath to allow
>  >subscriptions to almost anything under the sun. And it's got 
> applications in
>  >the area I'm thinking of. It's just that I feel that the 
> API conspires to
>  >discourage me from using it.)
>  >
> 
> The main thing is NOT to use too many different oid's as they are
> performance hungry and memory hungry.

This is going to be a little difficult if you want to use durable messages,
since part of the point of using them is to get the MOM system to manage
failures in service. If you've got volatile message producers (as I have,
since the sensors can go offline quite happily at any time) I want to be
able to put a message into the MOM system and then forget about it. If a
message producer has to wait until the previous message with the same oid is
delivered, then there's a good chance that the waiting producer will go
offline before it can send its message. If you've got bursty traffic, then
things get even worse.

Not that I'm too worried about memory footprints etc. I'm used to the idea
that, if you want serious performance, you slap in a few more processors and
an extra Gb of RAM. The main thing, for me, is to get sufficient
parallelism.

>  >It's the 'default handler' bit that bothers me. How do I 
> stop a connection
>  >from receiving 'SomeOtherMessage' and having it go to the 
> default handler,
>  >if I don't want to see it at all? I would have thought that 
> there's a
>  >default 'explicit subscriptions only' option, to avoid 
> having a client
>  >drowning in extraneous messages.
>  >
> 
> You only get what you subscribed for.
> Only PtP messages arrive always in the default callback.

But HelloWorld4 doesn't set a destination. So it can't be a PtP message
(right?). So why does SomeOtherMessage turn up at the default handler?