[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[xmlblaster] xmlBlaster performance issues
We've been using xmlBlaster for over a year now in one of our projects.
We've slowly been adding connected clients, increasing the number of
messages we send, and growing the size of the messages. Just recently
we seem to have hit the limits of the xmlBlaster server. However we're
still dramatically under the performance listed on the "performance"
page of the xmlBlaster.org site.
Here's our usage model:
- 4-5 clients using XmlBlasterAccessUnparsed in the C API.
- 2-3 clients using the Java API.
- A couple of the C clients actually make more than one connection to
the server. (The system architecture for our project dictates a few
"conceptual" components that don't match 1:1 with our actual running
We have 4 high-end machines (2-4 GB RAM, dual-core, dual-processor
Xeon/64-bit Opteron depending) running both Linux and Windows. All 4
have gigabit Ethernet connected to a single switch. We spread our
applications between the machines relatively evenly.
The messages are all point-to-point save for a single "monitoring"
component that just starts up and subscribes to '*' with a graphical
front end to verify the correctness of the information we're passing
around. We believe we're using the Socket protocol in all of our
The messages have a minimal key but contain an average of maybe 1k-4k in
their content section (maxing out around maybe 10k).
- We send messages in the C client using publish (vs. publishMsgArr).
- We altered xmlBlaster.properties by uncommenting the first 7 lines,
and we saw a large performance gain.
- We defined XB_USE_PTHREADS and saw a performance gain.
- We tried to enable burst mode on a few of the C clients but didn't see
much of a difference. (The "Client features" page says that burst mode
isn't implemented in the C client though.)
We ran the first test you have listed on the performance page
You: (600 MHz K7) 672 mps.
Us: (3.0 GHz dual-Xeon) 33 mps.
Us: (after uncommenting 7 lines in xmlBlaster.properties) 300 mps.
The 7 lines did help, but in our actual system, we're still not seeing
anything close to that 300 mps. In particular, the C client responsible
for most of the messages in the system appears to sleep (CPU usage near
0) whenever delivering a lot of messages back-to-back (even with
XB_USE_PTHREADS). This causes the GUI in that app to refresh at only a
frame every second or two.
We suspected the single "subscribe '*'" client might have been the
culprit for hitting the performance, but taking it out of the equation
doesn't help much.
I was wondering if you have a list of tips for squeezing more
performance out of the client and server:
- Is there a way to enable asynchronous send?
- Should we use "publish message array" in the C client?
- Should we use "publish one way"?
- Does compiling in release mode speed things along significantly? (We
tried on the C clients, but didn't see much difference.)
- Will compression help? (I suspect the message-send overhead is worse
than any kind of bandwidth limitations we might be running into.)
- Is there anything else we're missing?
Thanks for any assistance you can offer.
piegdon at soartech.com