[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [xmlblaster] Help! OutOfMemory by publishing lot big topics

Thomas, Johann wrote:
<homage>
I must say, as I ran across xmlblaster I found it more and more
interesting as I tried to dig deeper to find out it's intention and
usage. What I liked was the implemented development process, that
surfaced through the reference documentation. I liked the mailing list
quality, it assured me, that this product is well alive and has a good
foundation.

I was also astonished of the number of the clients and protocols
implemented.

I would have been happy to find a short introduction to the concepts
Yes, this is missing. 
that base the architecture of xmlBlaster, as the reference is sorted by
the requirement 'package identifier'. But I also knew, the lack of a
tutorial is a guarantee to find a community of smart and skilled
developers and users not needing such a introduction to hard ;-) So the
mailing list archive was the way to start with for the glue between the
reference pages.
</homage>

I am happy not to patch anything in the swapping code, as it would be a
real behaviour change that I expect to have unwanted side effects. I
have expected to get some more understanding of how the caching works
and find where to block the message data caching.

I would be happy if I could get some hints for a few API usage questions
I have:

1) Lot of topics:
-----------------
I have chosen to use many topics, as:
a) I (partly) need to process the messages in random order
and
b) I have not figured out how to "get" the topic history, seems I get
only the last message in the topic. I could use the iteration of the
topic history at least where I introduced the splitting of the content
in multiple publishes. There I have used the faster, because known, way
of different topics.
http://www.xmlblaster.org/xmlBlaster/doc/requirements/interface.get.html
shows with <history numEntries='20' newestFirst='true'/> how to get history entries.

If you are a subscriber (versus the above getter) you usually are not interested
anymore in the history as you will receive all messages. Don't forget to connect
the client in a fail safe mode to not loose the messages during the client is down.
2) StoragePlugin vs. QueuePlugin
------------------------------------------
I have expected to get rid of the 'meat' caching by configuring the
StoragePlugin accordingly before. I just got stock before with the
max*Cache settings. Now, from the following to configurations I expect
the first to swap the message data/meat directly to the persistent
store:
StoragePlugin[CACHE][1.0]=${JdbcStorage[HSQLDatabaseEngine]}
QueuePlugin[CACHE][1.0]=${JdbcStorage[HSQLDatabaseEngine]}
For my understanding: Is the second line suppressing the history,
callback, subject and (if client) connection queue RAM caching of only
the references into the StoragePlugin? Or is it necessary for stopping
the 'meat' caching, too?
No, this is perfectly legal as well:

StoragePlugin[CACHE][1.0]=${JdbcStorage[HSQLDatabaseEngine]}
QueuePlugin[CACHE][1.0]=org.xmlBlaster.util.queue.cache.CacheQueueInterceptorPlugin,persistentQueue=JDBC,transientQueue=RAM

Now the meat is on hard disk always, and the references from the queues
are cached (you can choose a huge RAM part to avoid swapping).
Note: All implementations 'CacheQueueInterceptorPlugin', 'RamQueuePlugin' and 'JdbcQueueCommonTablePlugin'
implement the same I_Queue interface and are therefor interchangeable, you can provide your
own implementation of I_Queue if needed.

enjoy
Marcel

Regards,

Johann Thomas


-----Original Message-----
Von: owner-xmlblaster at server.xmlBlaster.org
[mailto:owner-xmlblaster at server.xmlBlaster.org] Im Auftrag von Marcel
Ruff
Gesendet: Montag, 19. Juni 2006 15:49
An: xmlblaster at server.xmlBlaster.org
Betreff: Re: [xmlblaster] WG: Help! OutOfMemory by publishing lot big
topics

Thomas, Johann wrote:
...
- the content dumped is cached "somewhere" in the heap, but it can not

be garbage collected (each message has it' unique oid).
This is the problem: The topics have no swapping mechanism so using a
lot of different topics wont scale. Usually you should choose some well
defined topics (typically 5-100, not more than say 1000).

Each topic can hold any numbers of message instances, they will be
swapped away, depending on the configuration.
As you mentioned above - swapping does hard disk IO and is therefore not
as performing as memory based messages - but it will prevent out of
memory problems.

- I have set some properties maxEntriesCache = 0, but it had no effect

at publishing time. I have found some comments in the some of the code, saying first overload of queue is allowed.
You can just switch off the CACHE and work directly with the persistent
queue:

StoragePlugin[CACHE][1.0]=${JdbcStorage[HSQLDatabaseEngine]}
QueuePlugin[CACHE][1.0]=${JdbcStorage[HSQLDatabaseEngine]}

Now no RAM queue is used as all, the 'CACHE' instantiates the JDBC queue
implementation ...
so no patching is needed.


- I have patched CacheQueueInterceptorPlugin for me, so that it does
not
overload the cache above the specified settings. I'm not sure the
server
is behaving as stable as before with this settings, btw.
- after the restart of the xmlBlaster server the heap was mostly free.

So I suppose, there some other queues/caches were the first published
message of a given oid is held in memory, but not preloaded at a
server
restart.
We have one storage holding the message 'meat' (which is reference
counted):
  StoragePlugin
and several queues, which have a reference on the 'meat' only:
  history
  callback
  subject
  connection (on client side only)

The "callback" queue exists once for each login session, it is the most important.
The "history" queue is sort of a memory of already processed messages, and the
"subject" queue is for the seldom use case of PtP messages to a login name destination.

regards
Marcel

Here the overview of my little patch (between 2006-06-16 2101 JTh
patch
comments):

// .................... (cut for keeping the email short) package
org.xmlBlaster.engine.msgstore.cache;
public class PersistenceCachePlugin implements I_StoragePlugin,
I_StorageProblemListener, I_Map, PersistenceCachePluginMBean {

// .................... (cut for keeping the email short)

   private void assureTransientSpace(I_MapEntry mapEntry) throws
XmlBlasterException {

      while (!spaceLeft(mapEntry, this.transientStore)) {

// .................... (cut for keeping the email short)

if (oldest == null) {
if (log.isLoggable(Level.FINE)) log.fine("The RAM queue is
full, new entry '" + mapEntry.getUniqueId() + "' seems to be the first
and only one, so we accept it");
break;
}
// ++++ 2006-06-16 2101 JTh patch start
if (oldest == mapEntry) {
if (log.isLoggable(Level.FINE)) log.fine("2006-06-16 2101
JTh patch: The RAM queue will become full, new entry '" +
mapEntry.getUniqueId() + "' seems to be the first and only one, we
still
removed it, transientStore.numOfEntries now " +
this.transientStore.getNumOfEntries());
         }
         // ---- 2006-06-16 2101 JTh patch end
         if (log.isLoggable(Level.FINER)) log.finer("Swapping '" +
oldest.getLogId() + "' to HD ...");

// .................... (cut for keeping the email short)

      }
   }

   public int put(I_MapEntry mapEntry) throws XmlBlasterException {

// .................... (cut for keeping the email short)

      synchronized(this) {

// .................... (cut for keeping the email short)

// separate persistent from transient entries and store the
persistents in persistence // .................... (cut for keeping
the
email short)


assureTransientSpace(mapEntry);
numTransientPut = this.transientStore.put(mapEntry);
// ++++ 2006-06-16 2101 JTh patch start
assureTransientSpace(mapEntry);
// ---- 2006-06-16 2101 JTh patch end
} // sync(this)
// .................... (cut for keeping the email short)
}
}
Regards,
Johann Thomas

--
OpenPGP: 0x224225EA (7454 D7BB A75D EA8E 5DFF EA4E 778E A85B 2242
25EA)