Re: [xmlblaster] RE: Some issues with release .84

[MarcelRuff at t-online.de]

Lance Thomas schrieb:
> Hello XMLBlaster folks,
> 
> I'd like to thank you for contribution with XMLBlaster.
> It is a wonderful
> product, and it seems to be on its way to becoming an
> industrial strength
> messaging server.
> 
> We have been testing out XMLBlaster 0.84 and we ran into
> the following
> difficulties. We were wondering if you had seen these
> before and if you had
> any suggestions:
> 
> 1) Difficulties with database persistence: we are using a
> Postgres database
> to act as persistent storage. Every so often (around
> 1/100 times during our
> tests), XMLBlaster will throw the following error:
> 
> java.sql.SQLException: ERROR:  Cannot insert a duplicate
> key into unique
> index pg_class_relname_index
>         at
>
org.postgresql.core.QueryExecutor.execute(QueryExecutor.java(Compiled
> Code))
>         at
> org.postgresql.jdbc2.Statement.execute(Statement.java(Compiled
> Code))
>         at
> org.postgresql.jdbc2.Statement.execute(Statement.java(Compiled
> Code))
>         at
> org.postgresql.jdbc2.Statement.executeUpdate(Statement.java(Compiled
> Code))
>         at
>
org.xmlBlaster.util.queue.jdbc.JdbcManager.update(JdbcManager.java(Compiled
> Code))
>         at
>
org.xmlBlaster.util.queue.jdbc.JdbcManager.createQueueTable(JdbcManager.java
> :1394)
>         at
>
org.xmlBlaster.util.queue.jdbc.JdbcManager.addFreeTables(JdbcManager.java:51
> 4)
>         at
>
org.xmlBlaster.util.queue.jdbc.JdbcManager.getTable(JdbcManager.java:319)
>         at
>
org.xmlBlaster.util.queue.jdbc.JdbcQueuePlugin.initialize(JdbcQueuePlugin.ja
> va:76)
>         at
>
org.xmlBlaster.util.queue.QueuePluginManager.getPlugin(QueuePluginManager.ja
> va(Compiled Code))
>         at
>
org.xmlBlaster.util.queue.QueuePluginManager.getPlugin(QueuePluginManager.ja
> va:59)
>         at
>
org.xmlBlaster.util.queue.cache.CacheQueueInterceptorPlugin.initialize(Cache
> QueueInterceptorPlugin.java:146)
>         at
>
org.xmlBlaster.util.queue.QueuePluginManager.getPlugin(QueuePluginManager.ja
> va(Compiled Code))
>         at
>
org.xmlBlaster.util.queue.QueuePluginManager.getPlugin(QueuePluginManager.ja
> va:73)
>         at
>
org.xmlBlaster.engine.TopicHandler.startupHistoryQueue(TopicHandler.java:289
> )
>         at
>
org.xmlBlaster.engine.TopicHandler.administrativeInitialize(TopicHandler.jav
> a:235)
>         at
> org.xmlBlaster.engine.TopicHandler.publish(TopicHandler.java:457)
>         at
> org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1479)
>         at
> org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1319)
>         at
> org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1313)
>         at
>
org.xmlBlaster.engine.XmlBlasterImpl.publish(XmlBlasterImpl.java:164)
>         at
>
org.xmlBlaster.protocol.xmlrpc.XmlBlasterImpl.publish(XmlBlasterImpl.java:12
> 2)
>         at
> sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
> Source)
>         at
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
> .java:40)
>         at
> java.lang.reflect.Method.invoke(Method.java:335)
>         at org.apache.xmlrpc.Invoker.execute(Unknown
> Source)
>         at
> org.apache.xmlrpc.XmlRpcServer$Worker.executeInternal(Unknown
> Source)
>         at
> org.apache.xmlrpc.XmlRpcServer$Worker.execute(Unknown
> Source)
>         at
> org.apache.xmlrpc.WebServer$Connection.run(Unknown
> Source)
>         at
> org.apache.xmlrpc.WebServer$Connection.run(Unknown
> Source)
>         at org.apache.xmlrpc.WebServer$Runner.run(Unknown
> Source)
>         at java.lang.Thread.run(Thread.java:566)

Ok, we need to dig into this one.
Can you provide any client to reproduce it?
If you switch on debugging log
  -call[core] true -trace[core] true -call[jdbc] true -trace[jdbc]
true
we have more informations to investigate
(please send the log file directly to Michele and to me).

> 
> 2) Deadlock: XMLBlaster clients occasionally appear to
> enter into deadlock,
> under certain simultaneous usage patterns. Other
> non-deadlocked clients seem
> to be able to connect and continue processing. One
> culprit may be postgres,
> although I am unsure.

This is a very important issue as well.
Can you make a dump of the JVM during deadlock?
With IBM JDK i think a `kill -3` makes a dump.
There we can investigate which threads are in deadlock.
Or even better, do you have a client to reproduce it?


> 
> 3) Difficulties with large messages: some of the messages
> we send are 2
> megabytes in size. If there are about five clients
> publishing messages of
> this size simultaneously, meaning a total of 10 MB in
> motion, the server may
> throw an out of memory exception, even when the JVM is
> set to use 256 MB of
> memory. If there are any property settings or other
> configuration settings
> that would help with this, please let us know. If
> XMLBlaster is not supposed
> to be used with such large messages, or if we must figure
> in a 26x memory
> factor in deployment, please let us know. 

This issue we know already, Michele and i will try to fix it.

> 
> 4) Large amounts of data in persistent storage: Suppose
> we publish 500 MB
> worth of messages into XMLBlaster with persistence
> flagged. XMLBlaster
> accepts these messages and writes them to persistent
> storage with no
> problem. However, if we stop and restart XMLBlaster, it
> attempts to reload
> the messages from persistent storage and eventually runs
> into memory-related
> errors (on our 256 MB virtual machine). Again, if there
> are any property
> settings that will help with this, please let us know.

Thanks for reporting this. We will try to fix it
next week.

> 
> Again, we thank you and commend you for a fine product,
> and if you have any
> ideas about these issues, please let us know.
> 
> Take care,
> 
> Lance Thomas
> 
> 
best regards (and keep on testing),

Marcel