Re: [xmlblaster] xmlBlaster scalability?

[Marcel Ruff <mr at marcelruff.info>]

Marshall Shapiro wrote:
> I'm using xmlBlaster on a project and the question of scalability has 
> come up.
> Has there been any studies or test done, or do you know, what type of 
> through-put one can reasonably attain with xmlBlaster (in it's various 
> configurations) using the socket protocol in terms of message size and 
> frequency. Especially in terms of occasional extremely large content 
> messages? (I'm using SOCKET protocol and Linux server with Windows 
> clients).
>
> Thanks for your help.
>
> Moshe
Hi Moshe,

there is no simple answer to this -
but it is simple to test yourself the scalability questions.

We have some simple clients to test manually, this is for Java
javaclients.HelloWorldPublish and javaclients.HelloWorldSubscribe.
After startup they display adjustable settings which you can
use on command line (xterm or dos box).

Example (add xmlBlaster.jar to your CLASSPATH):

1. Start the server (on demand change the peristent store in xmlBlaster.properties):

   java org.xmlBlaster.Main -call[core] true

2. Test your requirements, here are typical examples:


Q1) I need to publish transient 20 messages / second of size 10 kBytes
    is this possible?

  java javaclients.HelloWorldPublish -protocol SOCKET -oid Hello -numPublish 1000 -contentSize
10000 -persistent false

    To run not interactive add those command line args (with a time summary):

   -interactive false -sleep 0

Q2) Many Subscribers need to listen to those messages
    Can xmlBlaster handle this?

   a) Two are C embedded devices
   b) One is a java statistic client (on Win)
      In fail safe mode (to not loose messages on client crash)
   c) One is a C++ BO (on Solaris)
      Fail safe (to not loose messages on client or server crash)
   d) 5 are mobile phones
   e) 100 are Java GUI clients


a) Update xmlBlaster "cvs upd -dP" first

   build c         // compile the C client library

   cd xmlBlaster/demo/c/socket/bin

   Subscriber -session.name Subscriber-C -oid Hello -updateSleep 0


b) java javaclients.HelloWorldSubscribe -session.name StatisticServer/1 -protocol SOCKET -oid Hello
-dispatch/callback/retries -1

c) Update xmlBlaster "cvs upd -dP" first and configure build.properties

   build cpp      // compile C++ client library
SubscribeDemo -session.name BusinessServer/1 -oid Hello -dispatch/callback/retries -1 
-persistentSubscribe true -dispatch/connection/pingInterval 5000 -dispatch/connection/delay 5000

d) See http://www.xmlblaster.org/xmlBlaster/doc/requirements/client.j2me.html

e) java javaclients.HelloWorldSubscribe -protocol SOCKET -session.name GUI -oid Hello


You will find detailed instructions under
  http://www.xmlblaster.org/xmlBlaster/doc/requirements/requirement.html
to each of the above clients (and other client types).

Note that there are many ways to tune xmlBlaster
so if any of your requirement is not met
just send this case to our mailing list for discussion.
For example the logging output of the client may
consume a lot of CPU, and for persistent message performance
the bottleneck is the JDBC driver, the DB and dominantly
your harddisk performance.
Another important point is that clients can consume
a lot of CPU so if xmlBlaster is later deployed to
a dedicated server hardware you should consider to test
xmlBlaster running on an own PC (different to your test clients)
and track the CPU load there.

For the above example these statements are interesting:
As xmlBlaster uses internally reference counting for all its subscribers
sending a message to 1000 of subscribers should scale whell
as far as memory consumption is concerned (for transient messages),
note however that most JDBC drivers consume temporary about 10 to 20 of the
memory so persistent messages need temporary a lot of RAM
(choosing a good JDBC driver helps). But still this memory
consumed is only for one message, all subscribers have a
reference only even if the message is swapped to HD.


best regards

Marcel


--
http://www.xmlBlaster.org