Re: [xmlblaster-devel] Large-scale File issue socket and memory

[Marcel Ruff <mr at marcelruff.info>]

Hi Joseph,

Koprivnikar, Joseph (Mission Systems) wrote:

> I was probing through discussion boards hoping to find some sort of
> solution but came up empty handed. The closest issue I found to what I
> am currently experiencing I noticed you had attempted to solve, so I
> thought it best to send an email since I'm not sure what list I should
> be sending to.
> On to the problem: I seem to actually have two problems occuring
> simultaniously and the software itself is starting to behave very
> non-deterministically. On one side is a memory issue, occasionally after
> publishing a large message (in this case ~16 Meg for an initial file,
> plus whatever wrapping is done) there will be an OutOfMemory error. I
> attempted to solve this by giving some parameters in both server and
> client side. 
> 1. I appended the -xms and -xmx flags to the batch file I use to run
> xmlblaster (server) giving each parameter 128 and 768 M, respectively.
>  
Yes, if you use persistent messages depending on the quality of the JDBC 
driver they
allocate temporary up to 5 or 10 times the size of the message - this is 
no leak
as it is temporary but it needs to be addressed by setting a high -xmx 
value.

> 2. I followed a suggestion made in the boards about message cache size
> in the xmlblaster.properties file, which I upped to 20 M
> as a side note, these seemed to fix a great deal of previous errors. 
> You'll also notice that I'm sending very large messages, which brings me
> to the second part of the problem. Since it takes a good 4 to 15 minutes
> (depending on the machine I'm using) to package up these messages, as
> soon as the message is sent, I immediately get a "connection reset"
> message. This indicates to me that the client was unable to respond to
> server pings while packaging the message to send. So I was wondering if
> there was some xmlblaster setting I could do that would allow for a much
> longer timeout, or perhaps some client side switch that would allow the
> connection to stay alive while building the message.
>
>  
Adding

dispatch/callback/burstMode/maxEntries = 1
plugin/socket/responseTimeout=3600000

to xmlBlaster.properties should do it.

The first option will send queued messages separately and not in a bulk 
(thus reducing the chance
of a timeout).

The second is the  maximum time for a client to finish the 
callback/update method (here set to one hour,
assuming you use the SOCKET protocol).

If the problems remains please send another mail to our mailing list,
sorry for the late response,

regards,
Marcel