[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [xmlblaster] RE: Some issues with release .84



Lance Thomas wrote:

Hello XMLBlaster folks,

I'd like to thank you for contribution with XMLBlaster. It is a wonderful
product, and it seems to be on its way to becoming an industrial strength
messaging server.

We have been testing out XMLBlaster 0.84 and we ran into the following
difficulties. We were wondering if you had seen these before and if you had
any suggestions:

1) Difficulties with database persistence: we are using a Postgres database
to act as persistent storage. Every so often (around 1/100 times during our
tests), XMLBlaster will throw the following error:

java.sql.SQLException: ERROR:  Cannot insert a duplicate key into unique

This is fixed.

2) Deadlock: XMLBlaster clients occasionally appear to enter into deadlock,
under certain simultaneous usage patterns. Other non-deadlocked clients seem
to be able to connect and continue processing. One culprit may be postgres,
although I am unsure.

This should be fixed. (Please report if it should show again).


3) Difficulties with large messages: some of the messages we send are 2
megabytes in size. If there are about five clients publishing messages of
this size simultaneously, meaning a total of 10 MB in motion, the server may
throw an out of memory exception, even when the JVM is set to use 256 MB of
memory. If there are any property settings or other configuration settings
that would help with this, please let us know. If XMLBlaster is not supposed
to be used with such large messages, or if we must figure in a 26x memory
factor in deployment, please let us know.


I made some investigations and found out it is related to the used JDBC driver.
Running RAM based only we use exactly the RAM which is the size of the message.
Running persistent showed me:


1. Using the deliverd Postgres JDBC driverin in xmlBlaster/lib:
I had up to x40 memory factor used, this is temporary during JDBC operation
and there after freed.


2. I played with newer Postgres JDBC versions (latest devel, latest stable for different
JDK versions):
They behave all the same, but "only" use half the amount of memory (up to x20)
So i have updated the delivered JDBC driver in xmlBlaster/lib.
You can replace it yourself if you find a better driver.


3. Using Oracle 9i JDBC driver.
Here i found a temporar factor of x5.
Note that again after insert completion the memory consumption is down to the
expected amount.


Note: With the default setting the max . size of the cache is 2 MB, so if you send
messages bigger 1MB (until any value) only one message is kept in cache,
the others are swapped to harddisk. Every topic (key oid) has its own cache.
Those properties configure the settings (xmpBlaster.properties):
persistence/msgUnitStore/maxBytesCache= [defaults to 2 MB]
persistence/msgUnitStore/maxBytes= [The max size on harddisk, defaults to Integer.MAX_VALUE]



4) Large amounts of data in persistent storage: Suppose we publish 500 MB worth of messages into XMLBlaster with persistence flagged. XMLBlaster accepts these messages and writes them to persistent storage with no problem. However, if we stop and restart XMLBlaster, it attempts to reload the messages from persistent storage and eventually runs into memory-related errors (on our 256 MB virtual machine). Again, if there are any property settings that will help with this, please let us know.

I think this is covered by 3)


Please do a cvs update build.sh delete build.sh all to get the newest.

i hope this helps,

Marcel


Again, we thank you and commend you for a fine product, and if you have any ideas about these issues, please let us know.

Take care,

Lance Thomas